 |
| ILM's Christopher Jones (left) and creative director Michael Koperwas with Real-Time Rocket in the background (source: Industrial Light and Magic). |
Industrial Light and Magic made a splash last year with Real-Time Rocket, an immersive digital puppetry experience for fans of Marvel Studios that was featured at Disney's D23 Expo.
Marvel fans at D23 were able to directly interact with Rocket Raccoon from Marvel's Guardians of the Galaxy, thanks to a collaboration between Marvel Studios, Disney’s StudioLab, and ILM, that enabled improv actors to perform Rocket using markerless motion capture technology.
ILM showcased an updated version of the activation at the Disney Accelerator Demo Day earlier this month:
Real-Time Rocket won a VES Award for Outstanding Visual Effects in a Special Venue Project, which is impressive, although having live audiences interact with a digital character is far from a new idea. Real-time digital characters have been featured in attractions like Turtle Talk With Crush at the Disney theme parks for over twenty years.
The specific challenge for ILM here was that due to the inherent limitations of markerless motion capture, they had to engineer a way to take a relatively limited amount of performance data from a performer and translate that into natural, biologically accurate poses appropriate for Rocket's raccoon anatomy.
@ign Rocket doesn’t know what IGN is, but you can interact with Rocket at the TVA at D23! #disney #d23 #marvel #marvelstudios #guardians #rocketraccoon #gotg #guardiansofthegalaxy #tva #loki #ign #mcu #movie ♬ original sound - IGN Entertainment
Amidst all the hype about employing Unreal Engine, markerless motion capture and the use of a LIMINAL 3D display, it's easy to forget that while the technology behind Real-Time Rocket is remarkable, what audiences responded to so strongly has very little to do with a technology pipeline, and everything to do with the performers using it.
Real-Time Rocket really works because of the classic Disney character performer training behind the scenes. Actors were trained extensively on Marvel lore, Rocket's speech patterns, and improvising with a live audience. The performers had a two-way video feed, which enabled them to spot specific people in the crowd to react to. That is just the latest iteration of what puppeteers have been doing digitally in various forms going all the way back to Jim Henson and Waldo C. Graphic in the 1980s.
So the actual magic at work here isn't digital - it's human.
Sergei Obraztsov famously said that his hand was the soul of his puppet. Here, the performer provides the character's soul. ILM's digital pipeline is, ultimately, just the puppet.
