Video Diffusion Models

AI绘画模型—

盾灵安全导航

One of our main innovations is a new conditional generation method for unconditional diffusion models. Our new conditioning method, which refer to as the gradient method, modifies the sampling procedure of the model to improve a conditioning loss on denoised data using gradient-based optimization. We find that the gradient method is more capable than existing methods in ensuring consistency of the generated samples with the conditioning information.

We use the gradient method to autoregressively extend our models to more timesteps and higher resolutions.

Frames from our gradient method (left) and a baseline "replacement" method (right) for autoregressive extension. Videos sampled using the gradient method attain superior temporal coherence compared to the baseline method.