By Tiana Cline
One of the most data-intensive industries is entertainment. Streaming services like Netflix or Apple TV are driven by data, collecting information about what you watch and for how long and using it to make personalized recommendations to keep you online. But that is only one side of how artificial intelligence (AI) and machine learning (ML) is used.
Filmmakers today use data science to forecast box office success and, on the rare occasion, a TV show can be picked up before a pilot has been showed – all thanks to technology.
But there is another part of the movies where data matters – visual effects (VFX). Even though computer generated images have been popular since the 1990s when films like Jurassic Park and Terminator were
released, the amount of data generated today by production houses is unfathomable.
Gretchen Libby is an American visual effects artist who worked at Industrial Light & Magic, the VFX company founded by Star Wars writer and director George Lucas. While she is known around the world for her work on award-winning films like Star Wars: Attack of the Clones, Iron Man, and Avatar, she decided to join Amazon Web Services (AWS) in 2020 as the director of visual computing to help those who create film and television content do so in the cloud.
Avatar: The Way of Water, most recently, was rendered in AWS. The movie was so computer-intense
it couldn’t be stored in the production company’s data center. “Many of us who know about visual effects, computer-generated water is some of the hardest work [you can do],” said Libby at an Amazon’s re:Invent panel.
“My first meeting on Avatar was in 2013,” explains David Conley, an executive VFX producer of Wētā FX, to FORBES AFRICA Conley says that Avatar: The Way of Water pushed Wētā FX to the edges of what computers cand deliver.
“We got to the point where the requirements for rendering this movie were we needed to partner with
someone. It has been a huge transition in the history of Wētā FX because we’ve had one of the largest data centers to render all the movies that we’ve been making over the last six years, seven years,” he adds. “And we make about 10 to 15 movies a year.”
With Avatar: The Way of Water, the render workload doubled. The VFX team went from shooting 24 to
48 frames per second, never mind the complexity of the water or other simulations. According to Jon Landau, the film’s producer, most of the scenes in the first Avatar film had two characters.
“Now we have five, 10, 12 characters with hundreds of background extras that need to be generated and rendered,” he says. While someone watching the movie will see a final shot that’s been rendered, behind the scenes, every single shot averages around 500 iterations. “All of those need to be rendered, one after the other… and [Wētā FX] outgrew their capacity to do that on the ground. And it’s not just their capacity – we blew the power grid!” laughs Landau.
GLOBAL PRODUCTION
There are, of course, security concerns that come with having such a big IP in the cloud. “We care very much about protecting our IP at all costs,” explains Landau. “There was a learning curve at the beginning for everybody to understand how to feed the machine and once we got there, the throughput transformed and we were able to – even in the later stages of the film – turn around iterations that
really just upped the ante on everything.”
For Landau, the door of possibility has been opened for what the cloud can really do. “It is important to
understand the artistry of requiring a project like this is wholly dependent on large amounts of teams coming together and providing lots of different iterations and solutions,” says Conley. “It requires a huge amount of compute power.”
Because Avatar: The Way of Water came together during the pandemic, cloud solved Wētā FX’s big data
problem. “We couldn’t architecturally expand our data center because that may require infrastructure that
would go to the city council and we all know what it’s like to deal with local government,” Conley says.
Cloud was the only solution – especially considering the stringent content security protocols Disney has on an IP like Avatar. “And I knew that a single frame of this getting out at the wrong time would be devastating not only to Wētā FX but to the project,” he adds. “It took about 14 months to get the first frame and then in eight months, we delivered 3.3 billion thread hours’ worth of content.”
“It’s hard to put in context… there were shots that are so complex, and I’m talking about a four second shot, not an hour-long shot, that would take two weeks to render.
The intense work was daunting and the difference when we went to the cloud was truly powerful,” says Landau. To make movies at a global scale with teams around the world requires combing big data together in a way that connects post-production with distribution and rendering. Through a cloud-based infrastructure, the Wētā FX team have found a way to bring all the entertainment expertise together
from inception to script writing, shooting and using the metadata that’s captured on set to deliver final visual effects.
“And that’s what’s exciting for me, those are the next steps. We’re paying the future as opposed to reacting,” says Conley. “What we’ve done here is create a system that allows us to be able to expand the physical and capacity constraints that we want and it gives us the flexibility to move beyond the traditional mortars that restrict us in the post-production space.”