Click AI: Open-Domain Regional Image Animation Using Brief Prompts  

[Paper will push to ArXiv soon]     [Github Repo]     [Twitter(X)]
[Pump.fun💊]

Click AI's vision is to expand the boundaries of memes. We hope to cooperate with pumpfun to turn static memes into dynamic memes for dev use and jointly build meme culture.
😊 The code and checkpoints is coming soon！

Click to play result from Click AI!

Gallery

Here we demonstrate more animation results via our framework.

Click to play result from Click AI!

Comparisons

Here we demonstrate the animations generated by different methods.
We qualitatively compare our approach with the most recent open-sourced state-of-the-art animation methods, including Animate anything, SVD, Dynamicrafter and I2VGen-XL. We also compare our approach with commercial tools such as Gen-2, Genmo, and Pika Labs.

Click to play the following animations!

---

Motion Strength Control

Here we demonstrate the comparisons between our optical flow motion magnitude control (OFM) and FPS-based motion magnitude control (FPS).

Click to play the following animations!

---

Ablation Study

Here we demonstrate the qualitative results of ablation the constructed short prompt dataset (D) and motion-augmented module (M). The motion prompt is “running”.

Click to play the following animations!

Limitation

Our approach is limited in generating large and complex human motions, as shown in the video.This maybe due to the complexity of the action and the rareness of related training samples.

Click to play the following animations!

Project page template is borrowed from DreamBooth.