Flash Animation Using Kinect

The Dead Man and the Lawyer is a short film and experiment by Nick Fox-Gieg, a Canadian animator and all-round interesting guy I've had the pleasure of getting to know over the past few months. Although this might look like relatively straight forward 2D animation, he actually made it using the Kinect and I think this opens up some exciting possibilities for creating Kinect animation.

To make the film Nick used OSCeleton to record Kinect data to an .xml file via a utility he's written called Processing Sketch. The beauty of this approach is that the motion capture information was written to an .xml file in real-time without any need for video capture. That way the animation data can be read back from .xml in Flash, tweaked and then rendered in full HD. A short tutorial and download links for all the necessary utilities and libraries you need to try this yourself can be found on Nick's website.

Nick's also a semifinalist in YouTube's NextUp competition right now. If you like his work, show him some love and vote for him.

First Kinect Motion Capture Series - Under the HUD

Under the HUD is what looks to be the first motion capture web series created using the Kinect. It was made using the OpenNI and Brekel Kinect hacks, as well as Autodesk's MotionBuilder and Maya software. Although the animation isn't completely fluid or elegant, it's not bad either. The important point is that, as Fast Company points out, until recently you needed to spend thousands (not the millions claimed by FC) to do this.

Episodes are coming soon; you can follow @thwackers on Twitter and visit the Thwackers website for updates.

Via Kinect Hacks.

New Sesame Street Game For Kinect

Here are some digital puppets in a very literal sense...Sesame Street: Once Upon a Monster is a new game for Xbox Kinect by acclaimed games studio Double Fine. The family-friendly game promises to take players into a living storybook populated with famous Muppet characters like Elmo, Cookie Monster and Oscar the Grouch. Gamasutra has an interview with Double Fine's Nathan Martz and Tim Schafer discussing the development of the game in depth.



The team at Double Fine is obviously having fun with this project (and who wouldn't?). Have a look at this video of Tim Schafer showing off some "fur rendering" technology.

Once Upon a Monster looks like it will be a lot of fun. It's due for release this fall.

Via Tough Pigs.

We Be Monsters: Multi-Person Digital Puppets

Carnegie Mellon University has been a leader in interactive studies (including lots of digital puppetry work) for years now so it's not surprising that CMU students have been busy dreaming up some pretty cool projects using the Kinect. A good example of this is We Be Monsters, a Kinect hack by students Caitlin Rose Boyle and Asa Foster that turns participants into a two-person puppet, “inspired by multi-person Chinese dragon costumes and (Mr.) Snuffleupagus.” It's a cool project that's somewhat similar to the previously-blogged Animata software.

You can see more examples of the great work that Carnegie Mellon students are cooking up over at Create Digital Motion.

Chris O’Shea's Little Magic Stories

The more artists explore new frontiers in interactive art, the more I often mind myself wondering where exactly to draw the line between what is animation and what is puppetry. A good example of this is the augmented shadow puppetry that I recently wrote about over at PuppetVision, which is sort of shadow puppetry, but also sort of not.

Another good example is this performance system created by Chris O’Shea called Little Magic Stories. It was made using the X-Box Kinect (see previous post) and the old Pepper's Ghost Vaudeville trick.

So is this animation? Digital puppetry? Or something else entirely?

Via CrunchGear.

Why Realism Doesn't Work in Animation (or Machinima)

Salon.com has a great interview with character designer Shannon Tindle. The article mostly discusses character design and why so many animated characters (like the ones in the recent "Gnomeo and Juliet") look terrible, but also touches on the subject of motion capture animation vs. animator-driven key frame animation.

Shannon is very diplomatic throughout the interview, but makes his feelings known about movies that skew towards realism in terms of both their design and their use of motion capture. Here's what he had to say about "motion capture" animated movies like The Polar Express and A Christmas Carol:

"All I know is that the studio that Disney formed to produce those films, Image Movers, doesn't exist anymore. "Christmas Carol" is one of my favorite stories of all time. But with those films, it doesn't look exactly like a real person and so it becomes something in between. In any animated film -- stop-motion, CGI and 2-D, and I've worked in all of those mediums -- you need to make a clear statement. Any time you waffle, if you're somewhere in between reality and stylization, a straight line and a curve, people feel it and they tend to have a bad reaction to it."

To me this is also one of the biggest barriers preventing real-time animation from reaching a more mainstream audience. Machinima might be getting more and more popular, but it's appeal is still very niche compared to mainstream animation. The movement in most real-time animation is driven by "realistic" mocap and/or skeletal tracking. At the same time, it's difficult to find an example of a real-time animated character that exhibits as much personality as Mickey Mouse did in the early Disney cartoons of the 1930s. I think there's a direct connection there; in order to be successful in the mainstream, audiences have to be able to connect emotionalally with the characters they're watching. As Shannon points out, it's much harder to do that using realistic motion or design.

Many Machinima creators are really innovative and do an excellent job of working with and around the limitations in the software they use to make their movies, but what they really need is a platform that allows them to create better, more emotive and relate-able characters so that their imaginations can be set free.

Salon.com article via Cartoon Brew.

Kinect Digital Puppetry: It's Here and it's Real

Kinect hacking has come a long way in a few short months and as most people following it already know, the potential implications for digital puppetry are huge. I was at a small meet-up for Kinect hackers in Toronto last night and tried a very rudimentary version of "Kinect puppetry" that an animator I know has been working on and it was very cool. Affordable, accessible digital puppetry is here, it's real and if you already have a computer and a little technical know-how you can do it yourself for under $200.

Here's an impressive demo of Kinect puppetry using the Japanese "Vocaloid" Hatsune Miku using the Kinect and free software from the VPVP Project:



Aside from the Kinect itself (which retails for around $150), most of the software needed to try this yourself can be downloaded for free. Of course you need a 3D program like Blender along with some sort of 3D game engine. Fortunately "lite" or "learning" versions of commercial engines like Motionbuilder and Unity can be downloaded for free and some people are using Ogre 3D, which is open source. The exact methods to make your own digital puppets will vary depending on which software you use, but lots of tutorials can be found by searching online.

Not all digital puppetry with the Kinect has to be three dimensional though. Here's a neat 2D demo done in the style of shadow puppetry. It's part of a project called Virtual Marionette:



Although I do think this is huge step forward for open, accessible digital puppetry there are still some problems to solve before Kinect can be used to create truly emotive, expressive characters. For one thing, the skeletal tracking in these demos is very simple without individual finger movements or any kind of facial expression. Those problems aren't unsolvable though and I have some ideas on how to tackle them, but that's another post.

Cross-posted from PuppetVision.