Digital Domainulls Making of 'TRON: Legacy' Goes Way Beyond Capture

The "TRON:Legacy" lightcycles played a big part in the 3-minute prototype for the film.

"TRON: Legacy" is a movie that's been years in the making.  Recently, Digital Domain's Ed Ulbrich spoke with  SourceEcreative to discuss some of the processes that the studio went through in pulling off this achievement.
 
Many of the key contributors to the making of "TRON" hail from the Digital Domain (DD) commercials division, says Ulbrich, a familiar presence in the commercials industry who is President of Digital Domain's Commercials Division and Executive Producer of its transmedia production arm, Mothership. 
 
"TRON: Legacy" Director Joe Kosinksi, who's repped for TVCs by Anonymous Content, has worked with DD in the past on commercial assignments, and the film's DP, Claudio Miranda, is another longtime collaborator.  All of them have at one time or another worked or been associated with director David Fincher, a co-founder of Anonymous. This list includes Jim Haygood of Union Editorial, who cut a number of Fincher's features and was the editor on "TRON: Legacy." 
 
"They're all 'Team Fincher' people," says Ulbrich, "and they all have deep roots in commercial work."  The same holds for Eric Barba, Digital Domain's Visual Effects Supervisor on the film.  And Darren Gilford, the film's production designer, is a former DD staffer who previously ran the studio's art department.  "It was kind of like friends and family on this project," Ulbrich says.  "Everyone shared a sphere of experience."

Actor Jeff Bridges, in headset cam and motion markers, acting out scenes as Clu.

In commercials, Kosinski has worked with DD on a number of TV spots, including work for Hummer, Lincoln (click the titles to see "Lift Off" and "Effects") and Xbox (see "Distress Call" and "Rendezvous").  For "TRON: Legacy," Ulbrich says the studio took the team they'd built around Kosinski for his prior work; that group formed  the  nucleus of collaborators who produced the short that was used to sell Disney on the "TRON: LEGACY" concept back in 2008. 
 
That short, which Ulbrich describes as a prototype, was initially pegged at a minute in length but eventually extended to three minutes, based on the reaction executives at Disney, which produced the first "TRON,"  had to what they were seeing.  It provided just a glimpse of what the updated film's overarching narrative would entail, with an obvious focus on the elements like the light cycles and the computer game grid environment that play such key roles in the story. 
 
For those who are not in on the story, "TRON: Legacy" takes place about 30 years after the original film, in which a young tech magnate, Kevin Flynn, played by Jeff Bridges, gets essentially sucked into a computer game and has to battle the evil Clu and a pernicious software program in order to escape.
 
In the 2010 version, Flynn's son has been similarly transported into the  "TRON" world - a now much-advanced gaming environment - where he's not only re-united with his father, played now by a 60-year old Bridges, but also goes up against Clu, who's played by an eerily lifelike digital simulation of Bridges from 30 years ago.

A double mimicked Bridges' performance, and a digitally-created head was placed on his body.

The short film that DD produced with Kosinski was shown initially at Comic-Con on the West Coast in 2008, and caused a sensation among techies, sci-fi enthusiasts and graphic novel fans. The response helped seal Disney's backing for the project.  "Many of the people in the group that worked on the short never really wrapped on that project," says Ulbrich.  "They went straight from that into the feature, and only finished up just a few weeks ago," Ulbrich says.
 
The final film is an extended mix of live action, digital production and CGI, with its crowning achievement being the extended use of DD's digital human synthesis, the process they originally developed when making Fincher's "The Curious Case of Benjamin Button," and has since evolved further.  All of which was overseen by DD's Eric Barba and Head of Animation Steve Preeg.  The tricks the studio went through to enable Bridges to reprise both his roles from the original 1982 film and today's were extensive, Ulbrich adds. "It was sort of Benjamin 2.0," he quips. 

The challenge loomed large on several fronts.  The DD team had to forego doing any digital human synthesis work on the short prototype film, for reasons of time and expense, Ulbrich explains, so the key to getting Bridges and others to approve the project was something of a leap of faith.  They got around it by bringing Bridges in and showing him what they were doing with Brad Pitt on "Button."  His reaction, says Ulbrich, "blew his mind. It was sort of the tipping point for us." 

The final version, featuring the 'digital human synthesis' avatar based on Bridges' performance.

But transforming Brad Pitt into an 85-year old man and 'de-aging,' or reversing the process, on a 60-year old Jeff Bridges are two very different things, he adds.  "No one knows what they're going to look like when they're 85," he says, "but everyone knows what Jeff Bridges looked like when he was 35. " They had much less creative leeway in how they were going to interpret Bridges as Clu in "TRON: Legacy."
 
There were other wrinkles (or 'de-wrinkles,' if you will) that they had to factor in: Clu, as a character, is not exactly Jeff Bridges, Ulbrich explains. "He's an avatar, a digital character from a videogame, one that's mutated and evolved from the character from the original film." So the appearance had to bear the likeness of  Bridges, but still be different in subtle yet unmistakable ways.  On top of that, the overall film had to build on the ground-breaking look of the original film, which is experiencing renewed interest now that its sequel has debuted.
 
Bridges also wanted to play the role of Clu as though he were "in the moment," says Ulbrich - that is, in the scene and on the set with other actors.  This added an entirely new level of R&D to the work the studio had done on "Benjamin Button," since in that film Pitt did not have to be physically in the same scene with other actors.  That meant that DD would have to find a way to capture Bridges' performance as Clu, then drop in a digital version in place of the real thing that could adequately reflect his body language and expressions.
 
The solution relied on making some extensive volumetric scans of Bridges' facial expressions, coupled with performance capture, while he was acting out the Clu scenes on set, combined with enhanced digital human synthesis work.   Performance capture was facilitated by the use of a headset camera system that Bridges wore while acting out scenes on set with the other cast members; it captured his facial  performance from four different angles. In addition, his face had performance capture markers placed on it. But what was done went way beyond marker-based capture, Ulbrich explains.

After Bridges performed scenes as Clu, a mid-30s body double mimicked his performances.  Separately, famed sculptors Rick Baker and Kazu Tsuji took a life-cast of Bridges' face and sculpted it to create a three-dimensional scale model that was 'de-aged' to approximately 35, which was then scanned/digitized into a 3D CG model. The CD model was further sculpted digitally by DD's artists, to result in the desired look for the Clu character.

Extensive CGI work went into creating the videogame environments for the film.

Then, to create a true-to-life digital model of Bridges as Clu, Bridges' face was digitally scanned as he assumed a range of facial expressions that follow the guidelines of FACS, or the Facial Action Coding System, which was developed by the psychologist Paul Ekman back in 1976.  FACS is a means of classifying human facial expressions that has something of a roadmap, providing insight into human reactions to physical expressions of emotion and has been widely used by animators, social scientists, psychologists, even human resource professionals. This process enabled the  DD team to create a digital library of facial expressions as performed by the 60-year old Bridges. 

The digital data from Bridges' on-set performance was then matched with the library of his facial expressions, using a mathematical model developed by the studio, and this performance and expression data was re-targeted to the digital model of Clu's head that was created from the scanned model. Animators then went in and finessed the anatomy to match the expressions that Bridges recorded to Clu's physiology using hand animation. 

DD then had to track the exact movements of the body actor and the camera, to integrate the CG head precisely with the body, in full stereoscopic 3D. Finally, all of Clu's elements were composited in minute detail to integrate animation, textures and lighting, and create the final shot.
 
"What we ended up with was not animators interpreting Jeff's performance, nor was it just marker-based capture, which doesn't provide you with enough information in order to achieve subtlety," says Ulbrich, but rather something that transcended both techniques.
 
The entire film includes over 1500 VFX shots, says Ulbrich, and tapped the resources of six companies working in Asia and North America in order to pull it off. While DD oversaw all of the work, the studio only directly produced the most challenging sequences, such as the digital human synthesis work, light cycle and disc game sequences and other key scenes.  In the process, says Ulbrich, DD became more than just a VFX studio, but rather a manager of mass-scale digital production.  And this has applications for the ad world, he adds, as global digital production becomes more prevalent across both feature films and advertising content.
  
And what about the digital human synthesis process?  Will DD's 'de-aging' of Jeff Bridges create a flood of advertisers looking to rejuvenate their long-dead celebrity spokespersons? Believe it or not, some are already thinking of doing just that.
 
"I've spent a lot of time trying to talk people out of that," Ulbrich laughs.  "For a lot of reasons it's a really bad idea - the main reason being, if the actor isn't actually there, the performance is just not going to be able to give you that person.  We need the actor's soul, in order to give it back to an audience this way." But he goes on to point out, in all seriousness, that the process is extremely time-consuming and labor-intensive, which of course are code words for expensive. And that's something that should make most clients think twice.

Published 29 December, 2010