Digital Shadow Puppet Installation

A nice look at an interactive, Kinect-based digital shadow puppet installation created by New York based motion designer Yang Yang.

Digital Guignol Theater

I've been doing a lot of Punch and Judy research lately and I stumbled across this Digital Guignol Theater that was created by Wizarbox in 2009. A proof-of-concept demo intended to aid in children during rehabilitation, it allows them to control a digital Guignol (the French equivalent of Punch) show using their voice and one or more Wiimotes.

Digital Wayang Kulit Music Video

We don't see a lot of music videos that feature digital puppetry, so when one comes along it's worth noting!

This video for Boshra Al Saadi's Snowyman was created using a modified version of Antonius Wiriadjaja's Java-based Wayang Kinect project. Hit the link to learn more about his work and download the digital Wayang Kulit applet so you can try it out yourself!

Flaming Skull Face Tracking Demo

Some clever face tracking from a Spanish studio called Paradox D&D. They've written a custom application that positions a 3D skull over top of the user's head, capturing the user's body movements and using them to control a 3D model in real-time. This is an example of a "virtual dresser" that allows a user to "wear" 3D elements in a manner very similar to augmented reality.

Via KinectHacks.

Researchers Demo New Kinect-based 3D Puppetry System

This video highlights a new Kinect-based 3D puppetry system developed by researchers at the University of Washington and the University of California, Berkeley that allows users to create 3D animation with minimal experience.

What's somewhat unique about this system is that unlike most Kinect-based systems, instead of the using body movement to control an onscreen character or object, users manipulate physical objects to control corresponding 3D models on a virtual set. I like this approach to digital puppetry a lot, because it retains the basic concept of puppetry (real-time manipulation of a physical object). To demonstrate how effective they believe this approach is, the researchers used novices in puppetry and animation to test the system.

Although the system these researchers have developed appears to be somewhat limited - note that the objects in the demo are solid objects without any kind of skeleton or use of Inverse Kinematics or Forward Kinematics - systems like this are very positive developments for digital puppetry. It would be great to see this get out of the lab and be developed in to some kind of open source platform.

A technical paper on this project can be found here.

Multitouch Puppetry

There has been an explosion of experiments with various forms of digital puppetry in the past couple of years, but we're still lacking a true "killer app" that makes real-time manipulation of digital characters simple an intuitive. That doesn't mean that progress isn't being made though; little breakthroughs are being made all the time. I've seen a number of interesting experimental interfaces in the past year, including this one created by Quan Nguyen and Michael Kipp, who have attempted design a simple, user-friendly multitouch system that can be used to control the complex movement of the human arm.

From their research:

"Controlling a high-dimensional structure like a 3D humanoid skeleton is a challenging task. Intuitive interfaces that allow non-experts to perform character animation with standard input devices would open up many possibilities. Therefore, we propose a novel multitouch interface for simultaneously controlling the many degrees of freedom of a human arm. We combine standard multitouch techniques and a morph map into a bimanual interface, and evaluate this interface in a three-layered user study with repeated interactions. The multitouch interface was found to be as easy to learn as the mouse interface while outperforming it in terms of coordination...Our results show that even complex multitouch interfaces can be easy to learn and that our interface allows non-experts to produce highly coordinated arm-hand animations with subtle timing." 

Quan and Michael are members of EMBOTS (Embodied Agents Research Group), a research group based in Germany. They've worked on a number of interesting projects with digital puppetry applications; you can find an overview of their work here.

Kinect Digital Puppetry Experiment

Here's a recent experiment in 2D digital puppetry, created using the Kinect and KinectSDK block for Cinder / C++ (compiled using Microsoft Visual Studio 2010). If you're able to program, you can find the code that was used to create this digital puppet here and try it yourself!