Phenaki

 Phenaki is a cutting-edge model that offers a breakthrough in video synthesis technology. By allowing users to generate realistic, variable-length videos from textual prompts, Phenaki opens the door to a whole new era of creative possibilities. Despite the challenge of generating videos from text due to the computational cost, limited quantities of high quality text-video data and variable length of videos, Phenaki proves to be up to the task. Its advanced tokenizer, which uses causal attention in time to work with variable-length videos, and video encoder-decoder, which outperforms all per-frame baselines in the literature in terms of spatio-temporal quality and number of tokens per video, combine to create stunningly realistic video representations.

To generate video tokens from text, Phenaki uses a bidirectional masked transformer conditioned on pre-computed text tokens. This tokenizer compresses the video to a small representation of discrete tokens and allows Phenaki to generate arbitrarily long videos conditioned on a sequence of prompts in open domain, making it the first model to do so. To address data issues, Phenaki demonstrates how joint training on a large corpus of image-text pairs as well as a smaller number of video-text examples can result in generalization beyond what is available in the video datasets. Additionally, Phenaki presents interactive examples that showcase its capabilities, allowing users to choose one combination of context words for creating a video about an astronaut.

With its groundbreaking technology, Phenaki offers a new level of creativity for users looking to bring their stories or ideas to life. Whether it's creating a video from a still image or generating a 2-minute story using a long sequence of prompts, Phenaki provides the tools to create stunning, realistic videos that will captivate and engage audiences. Explore the possibilities of video synthesis with Phenaki today.