Can Stable Diffusion Make Videos? Yes when used with the Deforum tool

Could AI video creation be at your fingertips?

Ian Spiller

Share this article...

• Follow us

Models like Stable Diffusion and Midjourney have transformed AI image generation in the dynamic field of AI. The prospect of producing videos, though, poses a bigger obstacle.

While Stable Diffusion has demonstrated its ability to produce incredibly lifelike and detailed still images, the question of whether it can also produce videos still remains. In this piece, we explore the potential and question if Stable Diffusion has developed to the point where it can produce engaging videos.

How to Make Videos with Stable Diffusion

To create videos with Stable Diffusion, users can take advantage of the new img2img features. These features allow for the generation of short videos or GIFs that mimic the appearance of actual videos.

Moreover, Stable Diffusion now offers animated frames, enabling the creation of captivating short videos. Users can utilize the Deforum tool, an extension of the Web UI, to direct and generate MP4 videos. With Deforum’s power and reliability, users can explore the exciting possibilities of video creation within the realm of Stable Diffusion. To get started, follow the steps below:

Installation

To install the Deforum extension and enable its functionality, follow the steps below based on your operating system.

For Windows:

1. Open the command prompt (cmd) and navigate to the stable-diffusion-webui directory. If you have followed the installation guide, run the command: `cd %userprofile%\stable-diffusion-webui`
2. Install the Deforum extension by executing the following command: `git clone https://github.com/deforum-art/deforum-for-automatic1111-webui extensions/deforum`
3. After restarting the AUTOMATIC1111 GUI, you should see the Deforum tab.

For Mac:

1. Open the Terminal App and go to the installation directory of stable-diffusion-webui. If you have followed the guide, run the command: `cd ~/stable-diffusion-webui`
2. Install the Deforum extension by executing the following command: `git clone https://github.com/deforum-art/deforum-for-automatic1111-webui extensions/deforum`
3. After restarting the AUTOMATIC1111 GUI, the Deforum tab will be visible.

Adjusting the Settings

To generate a test video and familiarize yourself with the settings, follow these steps:

• Step 1: Open AUTOMATIC1111 GUI and navigate to the Deforum page.
• Step 2: Go to the keyframes tab, located at the bottom half of the page. Adjust the camera parameters in the Motion tab according to your preferences. The Max frames setting determines the length of the video.
• Step 3: Move to the Prompts tab, where you will find a list of prompts with corresponding frame numbers. Each prompt becomes effective at a specific frame. By default, the provided prompts are already set for the video.
• Step 4: Click on the Generate button to initiate the video generation process.
• Step 5: Once the generation is complete, click on the button above the Generate button to view the video. To save the video, click the three vertical dots in the bottom right corner or locate it in the output directory under the img2img-images folder.