Is deepfake truly changing VFX?

Diederik Veelo: The king is dead, long live the king!

Bas Moonen: Wait, are you talking about VFX? Is that a statement you actually want me to respond to?

DV: It’s not my opinion.

BM: Well for the record it’s obviously not true. It's kind of a cliché, but it's true that AI and deepfake have changed VFX, by introducing a bunch of new tools. It has opened the field to different disciplines and perspectives, which also allows us to explore new aesthetics. The developments in AI are moving faster than we can anticipate, but we should take a playful, experimental approach that enhances creative output.

This G-Star Raw spot, Wear Your Denim Till The End, sees a man’s jeans age at a slower rate than he does.

DV: Just for the record, I disagree with the statement too! At the end of the day, we use any tool – audio, visual, AI or traditional VFX – to craft our narratives. The medium is secondary; storytelling is what binds us.

We now frequently use deepfake in various projects.

BM: Our G-Star project illustrates AI’s role in storytelling. Working closely with [director] Paul Geusebroek, we crafted the narrative using deepfake tech, building the dataset ourselves from hundreds of photos taken before the shoot. With proprietary software we taught an AI model what our actor should look like throughout his life, which we then applied to body doubles. We used VFX to enhance the realism of human expressions. 

This technology has evolved, and we now frequently use deepfake in various projects. For instance, we 'shoot' celebrities in unvisited locations using body doubles after creating our own dataset. This technique is particularly useful when we have limited time with an actor.

DV: It’s good you mention datasets Bas. As we navigate these innovations, it’s crucial to keep talking about the ethical and transparent use of AI. I think the focus for now is on responsibility, rights and IP; and knowing what data is used to train models and not using it for misinformation. Right now, there is a lot of uncertainty for many people, and that's not fair to those putting in effort and time into this industry.

We're constantly discovering new things and having happy accidents everywhere.

DV: Maybe the new technologies will herald even more creativity. I'm not sure if it will be better per se, but we might see more of it. I think, Bas, as a creative that’s up to you. The artists are the ones who need to create something special out of all of this. AI is currently struggling with any kind of art direction. It's cool to create something one-off but if you want consistency, it requires close supervision.

BM: I'm a big fan of the experimental part of this revolution. We're constantly discovering new things and having happy accidents everywhere. We’re using more and more AI and deepfake in our post-production and you see so much amazing content coming out of it. The deepfake principle is at the heart of many new applications of AI that we’re currently developing. Teach the AI what something specific should look like and apply it to something else, faces, environments or objects. The principle is the same.

The A.I.dentical Fans campaign, created by TBWA Peru for BCP, Peru's official soccer team sponsor, saw the creation of 50 deepfake identities generated by images of millions of Peruvians. Any Peruvian who resembles one of these AI images has the chance to win tickets for an upcoming soccer game. 

DV: You still need to work in tandem with traditional VFX techniques to really tell the story well. That said, we can do so much more – such as changing the weather or season, modifying tree foliage, making lighting continuous, or extending the golden hour. We're expanding our capabilities with AI, exploring both predictable and surprising outcomes. This opens up more opportunities for virtual production and the impact on traditional workflows and roles in production is huge. 

Even when the techniques improve, AI-generated films will never fully replace traditional filmmaking – they won’t become the norm, but they might become a genre.

It’s fascinating to see the evolving landscape of streaming services with giants like Disney, Hulu, and Warner Brothers' MAX potentially partnering up. Initially, streaming drew viewers away from TV, but now these platforms may start bundling their services, mirroring the cable TV model. This reflects significant industry shifts and underscores the ongoing need to deliver content that resonates with and captivates audiences – whatever the techniques used to create it. 

This might one day all be categorised under 'visual effects' or perhaps a new term, but the differentiators will always be the quality of storytelling, the ability to art-direct moving images, and simplifying production for clients.

Director Paul Trillo’s promo for Washed Out’s track The Hardest Part is the first music video to be generated entirely with OpenAI’s text-to-video generator.

BM: Take the style of this amazing music video made by Paul Trillo using Sora, it deserves a recognition in content creation. But even when the techniques improve, AI-generated films will never fully replace traditional filmmaking – they won’t become the norm, but they might become a genre. 

Generative AI will always be somewhat whimsical. This is both its strength and weakness. 

Generative AI will always be somewhat whimsical. This is both its strength and weakness. It’ll get better at outputting exactly what’s required. We will get better at directing it. But when a project demands something very specific, there’ll always be a need for human craft alongside it. I see a lot of potential growth in task-specific AI tools within VFX. 

Deepfake/ageing, mocap, gaussian splatting [a recent volume rendering method useful for capturing real-life data into a 3D space and rendering it in real-time] – they’re all over my Instagram feed. Our team is developing tools to change weather or light in much the same way as deepfake is changing faces. 

It’s these AI tools that as an industry, we need to adopt into our way of thinking. It’s in the synergy between VFX and AI that we can create truly exceptional work.