Kinect Digital Puppetry: It's Here and it's Real

Kinect hacking has come a long way in a few short months and as most people following it already know, the potential implications for digital puppetry are huge. I was at a small meet-up for Kinect hackers in Toronto last night and tried a very rudimentary version of "Kinect puppetry" that an animator I know has been working on and it was very cool. Affordable, accessible digital puppetry is here, it's real and if you already have a computer and a little technical know-how you can do it yourself for under $200.

Here's an impressive demo of Kinect puppetry using the Japanese "Vocaloid" Hatsune Miku using the Kinect and free software from the VPVP Project:



Aside from the Kinect itself (which retails for around $150), most of the software needed to try this yourself can be downloaded for free. Of course you need a 3D program like Blender along with some sort of 3D game engine. Fortunately "lite" or "learning" versions of commercial engines like Motionbuilder and Unity can be downloaded for free and some people are using Ogre 3D, which is open source. The exact methods to make your own digital puppets will vary depending on which software you use, but lots of tutorials can be found by searching online.

Not all digital puppetry with the Kinect has to be three dimensional though. Here's a neat 2D demo done in the style of shadow puppetry. It's part of a project called Virtual Marionette:



Although I do think this is huge step forward for open, accessible digital puppetry there are still some problems to solve before Kinect can be used to create truly emotive, expressive characters. For one thing, the skeletal tracking in these demos is very simple without individual finger movements or any kind of facial expression. Those problems aren't unsolvable though and I have some ideas on how to tackle them, but that's another post.

Cross-posted from PuppetVision.