Adobe has previewed its upcoming generative AI video tools, showcasing new features that convert text and images into video clips. These tools, part of Adobe’s Firefly video model, build on the company's earlier demonstrations and promise significant advancements in video and audio editing.
These details were revealed in a Jess Weatherbed news piece that was published on The Verge on September 11.
The new tools allow users to generate video clips from text descriptions and adjust them with simulated camera controls. These controls can modify camera angles, motion, and shooting distances. Another feature can create videos from specific reference images, useful for adding B-roll or filling gaps in production (Weatherbed, 2024).
Adobe’s preview footage suggests that the video quality rivals that of OpenAI’s Sora model, which Adobe is considering integrating into Premiere Pro. However, Adobe's Firefly model has a limit of five seconds per video clip. Alexandru Costin, Adobe’s VP of generative AI, noted this constraint but emphasised Firefly’s advantage in being "commercially safe" due to its training on licenced and public domain content (Weatherbed, 2024).
The new text-to-video and image-to-video features will be available later this year in beta form as a standalone Firefly application. They will eventually be integrated into Adobe’s Creative Cloud, Experience Cloud, and Adobe Express (Weatherbed, 2024).
Additionally, Adobe previewed a “Generative Extend” feature for Premiere Pro, which will extend existing video footage similar to Photoshop’s Generative Expand tool. This feature will also be released later this year.
