AI Video Is Ready for Its Close Up

Get Ready for AI Video to Be as Wild as Images and Text

AI video has quietly taken a leap forward, and the results could make the generative imaging and text bots that took the world by storm last year look dull by comparison.

Without much fanfare, and mostly locked away in R&D labs, Google, Meta and others have steadily advanced the state of the art for text-to-video inch by inch. Using tools like Phenaki, Dreamix and Make-a-Video it's already possible to create simple scenes from scratch with text and image prompts, much in the same way Stable Diffusion generates novel images.

Now the same company that created Stable Diffusion's core code has unveiled next generation level stuff – and it's impossible not to immediately appreciate that a ChatGPT-style tsunami is headed our way as soon as this stuff goes public.

Like some of its rivals, AI startup Runway's Gen-1 algorithm automatically changes the style, color, texture and other visual features of an existing video clip using nothing more than a simple text prompt or image. Samples released by the company this week are ridiculously good, and that's just working off the artistic inspiration of a handful of software engineers.

What'll happen when millions of people give it a shot? We're about to find out.

Why It Matters

Generative AI is sweeping through all of the creative arts and it's now crystal clear that video will be no exception. As the ease and quality of producing novel and startling effects drop, this will have a big ripple effect through social media, where it will almost certainly become a tidal wave that swamps Hollywood and its splashy big budget productions and hard to produce special effects.

James Cameron's Avatar 2 cost $250 million and used bleeding edge tech to capture the performances of real actors then transform them digitally using a virtual camera and loads of post production work. It's not hard to imagine being able to do almost as well soon with an iPhone and a future video AI.

iPhones are already making spectacular use of NeRF (near radiance field) technology to deliver eye-popping effects, so this is not at all far-fetched.

Unlike Meta and Google, which have developed most of their AI behind closed doors (Google only just released a test version of its consumer AI chatbot, Bard), a consumer ready version of Gen-1 is on the launchpad. Runway worked closely with video makers to gain insight into how filmmakers and VFX editors actually work on post-production, so ordinary people will be able to use it very soon.

To understand why that's important, look no further than OpenAI's ChatGPT. The technology behind it was semi-public for more than 2 years. It took a consumer release to burn it into the global psyche overnight.

What's Next

There's been some blowback over what some see as hasty consumer AI roll outs, as we pointed out last week. After first partnering with Stability.ai, which took over training Stability Diffusion and wound up in lawsuits over alleged copyright infringement, Runway has distanced itself with its pivot to video.

In the case of video-to-video tools, it's not clear how these same issues might play out. Transforming a selfie for TikTok might be uncontroversial, transforming a Hollywood classic probably not.

Gen-1 is going into private beta for a small group of testers and has opened a waitlist. The company then plans to release it publicly, likely as a limited pro version, but it seems like just a matter of time before we see a free to use version for everyone.

Runway CEO Cristóbal Valenzuela said putting Gen-1 into the hands of creative professionals will show the world what's possible, and spark another viral moment for AI.

“We’re really close to having full feature films being generated,” he told MIT Tech Review. “We’re close to a place where most of the content you’ll see online will be generated.”

Update: Brands 1, NFT Artists 0

Mason Rothschild’s fuzzy MetaBirkins project featuring digital images of a popular line of Hermes bags coifed with fur went too far, a jury has found in the first trial to consider the line between creative license and trademark protections in the wild west of NFTs.

For now, fashion house Hermes is on top, winning two key arguments: that Rothschild profited off the fashion house's name without permission, and that he was not entitled to a legal exception for freedom of expression because he is an artist.

The verdict is notable mostly for being what lawyers call "a case of first impression," meaning the legal issues at stake had not been litigated before. It is the first verdict on NFTs and trademarks but it is not a verdict that now applies generally to other NFT trademark cases, such as Yugalabs v Ripps, another closely watched case that currently going through pretrial motions – yet.

Don't be surprised if this gets appealed.

“I’m not terribly surprised the jury found for Hermes here and I think it was probably the right result,” says intellectual property attorney Michael Kasdan, who spoke with Meteor about the trial at length last week. “It would have been more surprising and a 'bigger deal' in terms of changing the status quo if Rothschild had won.”

For now, Rothschild has to pay $133,000 in damages, the 450 NFTs he sold can't be traded (legally that is, there may be workarounds for determined buyers) and the 550 unsold NFTs in his collection will stay on the shelf.

Even after the opening salvo, this will likely remain a closely watched case with big implications for digital goods, freedom of expression, the Metaverse and the future of art. Given the heavyweights involved it could well set the law of the land eventually as it continues to work its way through the courts, a process that could take years.

Meanwhile, it’s a chilling result for artists who make statements about big name brands and for NFT projects hoping their tokens might be protected under the First Amendment.

Andy Warhol biographer Blake Gopnik, whose testimony was not allowed to be entered into evidence in the trial by Judge Jed Rakoff, sees the verdict as a blow for artists.“Crazy,” the art critic wrote on Twitter. “So artists aren’t allowed to have luxury goods as a subject, and make that clear in the title?”

Craig Wright's case against Bitcoin developers is going to trial
The man who claims to be Satoshi Nakamoto, pseudonymous developer of Bitcoin, will get his day in court. After initially being thrown out by a UK court, Wright won the right to trial on appeal. Sometime next year he’ll make his case that he’s entitled to access to two wallets with over 100,000 Bitcoin between them.

What rares? Open editions fuel some renewed buzz in NFTs
Open edition NFTs come with no limit on the amount that can be minted and tend to come with low price tags. The success of VV Checks, in particular, has already led to a number of derivative open edition projects.

ChatGPT and other chat bots can spread conspiracy theories and other misinformation
A team of researchers, including from ChatGPT creator OpenAI, found that the chat bot could be prompted to repeat popular conspiracy theories and other forms of mis- and dis-information. Not a total surprise, but certainly cause for concern heading into election campaign season.

Startup Neeva was already doing AI-forward search before Microsoft and Google
Meteor has been hip to Neeva for a while. It's search that provides AI-generated answers with citations. Not bad.

Forget geoengineering, ‘astroengineering’ offers an audacious alternative to slow global heating
Scientists propose blasting lunar dust from the moon to a point in space between Earth and the sun where it could act as a shade to block 1 - 2 per cent of our star’s radiation and slow the advance of climate change. It’s not yet clear how long the dust shield would stay in place or how well it would work, but researchers assure us that the approach would not overcorrect and risk triggering a new ice age.

The first-ever firing of all 33 engines on the new biggest rocket ever happens Thursday
SpaceX President Gwynne Shotwell said the company hopes that a long-awaited hold-down test firing in Texas finally happens this week. The Super Heavy booster is set to send Elon Musk's moon and Mars spacecraft, Starship, on its first trip to orbit as soon as next month.

Thanks for reading!  Please take just 30 seconds to tell us what you love or hate about this issue.