How Pixar Used AI to Make 🔥

Welcome to another issue of VP Land - you’re daily update of all things new and interesting in the virtual production industry (with a bit of generative AI).

Special thanks to this email’s sponsor, Adspective, an AI platform that automates ad design and placement in existing multimedia content.

How Pixar’s Elemental used NST to animate fire

Pixar’s Elemental director Peter Sohn faced a problem - how to animate a character made entirely of fire without looking absolutely terrifying. Fortunately, Paul Kanyuk, a crowds technical supervisor, had a possible solution from a research paper from Siggraph Asia.

The full story and sample animation are in this Wired article, but here’s the breakdown.

Challenges with Naturalistic Animation: Animating natural elements such as fire can be particularly challenging due to their inherent unpredictability and ever-changing nature. It's especially hard when these elements need to be anthropomorphized into characters that can convey specific emotions and facial expressions, as with Ember in Pixar's "Elemental."

Existing Tools and Techniques: While Pixar had existing tools to create flame effects, these tools struggled to balance realism with the need to shape fire into a character. Too much realism created a disturbing effect, while too little undermined the fiery essence of the character.

AI and Machine Learning Solutions: AI and machine learning, specifically neural style transfer (NST), offered a promising solution. NST, often used to apply the artistic style of one image to another, was proposed to manage the voxel movements in the animation. This approach aimed to give the fire character a distinct look, balancing between the dynamic nature of fire and the character's shape and sensibilities.

Artistic Control in Animation: The use of AI didn't remove the need for artistic input. Instead, it combined swirly, hand-drawn cartoonish flames (dubbed "fleur-de-lis") combined with ‘blobbier fire’ from the original simulation to get the natural movement of fire and the distinct Pixar style.

Processing Power: The drawback to this is treating all 1600 shots as a VFX shot with machine learning required a lot of computing power. But, they figured out a solution. Kanyuk: “We figured out a way to virtualize the GPU and take half of it to use overnight, making the time to render a frame go from about five minutes to one second.”

Promising Future of AI in Animation: The success of this approach demonstrates the immense potential of AI and machine learning in animation, providing solutions for complex challenges and enhancing the creative process. It also shows that these technologies can help reduce the time and cost associated with traditional animation processes.

Sponsor Message

Looking to revolutionize your advertising strategy? Discover Adspective, the innovative solution that automatically adds advertising content to your existing photos and videos.

With Adspective, you can breathe new life into your visual content, making your ads more engaging and effective than ever before.

But that's not all. Adspective also offers new cost-effective ways to publish ads, making it a game-changer for businesses of all sizes. Whether you're a startup on a tight budget or a large corporation looking to maximize your advertising ROI, Adspective is the solution you've been waiting for.

Learn more about how Adspective can transform your advertising strategy.

Using Photoshop as the hub to aggregate different gen AI models:

Around the Web

• Zeroscope, a new, free model for text to video

• Recap of the Hollywood Innovation and Transformation Summit on virtual production

• Send and receive large files quickly with MASV*

• Design comparison breakdown of Apple Vision Pro vs Meta Quest Pro

• More XR labs opening up in colleges

• ControlNet, Qualcomm’s new generative AI tool

* Sponsored Link