Digital Wayang Kulit

Here's a nine minute demo of a Digital Wayang Kulit (Indonesian shadow puppetry) program developed at the MSc in Digital Education program at University of Edinburgh. The 2D figure is controlled by the digital Dalang (puppeteer) using a Gametrak controller and a Wiimote.

There have been a lot of shadow puppet inspired digital puppetry demos created over the years (this one is from 2013), but I love how fluid the movement of this one is.

Special thanks to Jane for submitting this!

This is a simple, but nonetheless very effective example of 2D digital puppetry created using Unity and a Kinect.

Digital Puppeteer Mario Mey

This is a new demo reel for Argentinian digital puppeteer Mario Mey that shows off his digital characters performing en Español at various live events (his character Pinokio 3D was mentioned here back in 2010) . He creates and performs his "Marionetas Digitales" (digital puppet) characters using Blender 3D and PureData, a real-time graphical dataflow programming environment for audio, video, and graphics.

You can see Mario at work and get a look at his production process in this video, however it was recorded in Spanish.

A digital dragon puppet

A nice example of a digital shadow puppet, made by Luis Leite using Kinect and Unity 3D. To animate the puppet, a human body is tracked in real-time using the Kinect sensor, with one hand controlling the head and the other controlling the tail. The physical movement of the performer's body is remapped on to the virtual shadow puppet using Inverse Kinematics via Unity's Mecanim animation system.

Luis was also responsible for a Kinect-based digital puppet that was mentioned in a post about Kinect-based digital puppetry on Machin-X two years ago.

Karagoz goes digital

Indonesian shadow puppetry has gone digital, so why not Turkish shadow puppetry too?

iKaragoz is an app for iPhone and Android developed by a Turkish firm called Anakule. They are promoting it as the first puppet application for mobile devices, but it's definitely not (several others including iPuppeteer and Pollock's Toy Theatre app have been on the market for years).

The app allows the user to control the characters onscreen intuitively by simply moving their smart phone or tablet.In addition to the traditional Turkish Karagoz puppets, additional packs with characters from Cambodian, Chinese, Indian, Indonesian, Thai and Greek puppetry traditions are also available.

Here's the Wayang Kulit version in action:



iKaragoz was designed by Uğur Doğan with the assistance of Turkish puppeteer Mehmet Saylan. It's available to download from the iTunes Store and Google Play. I haven't had a chance to try it out myself yet, but if someone does please let me know what you think!

Via Puppetry News.

Digital Guignol Theater

I've been doing a lot of Punch and Judy research lately and I stumbled across this Digital Guignol Theater that was created by Wizarbox in 2009. A proof-of-concept demo intended to aid in children during rehabilitation, it allows them to control a digital Guignol (the French equivalent of Punch) show using their voice and one or more Wiimotes.

Multitouch Puppetry

There has been an explosion of experiments with various forms of digital puppetry in the past couple of years, but we're still lacking a true "killer app" that makes real-time manipulation of digital characters simple an intuitive. That doesn't mean that progress isn't being made though; little breakthroughs are being made all the time. I've seen a number of interesting experimental interfaces in the past year, including this one created by Quan Nguyen and Michael Kipp, who have attempted design a simple, user-friendly multitouch system that can be used to control the complex movement of the human arm.

From their research:

"Controlling a high-dimensional structure like a 3D humanoid skeleton is a challenging task. Intuitive interfaces that allow non-experts to perform character animation with standard input devices would open up many possibilities. Therefore, we propose a novel multitouch interface for simultaneously controlling the many degrees of freedom of a human arm. We combine standard multitouch techniques and a morph map into a bimanual interface, and evaluate this interface in a three-layered user study with repeated interactions. The multitouch interface was found to be as easy to learn as the mouse interface while outperforming it in terms of coordination...Our results show that even complex multitouch interfaces can be easy to learn and that our interface allows non-experts to produce highly coordinated arm-hand animations with subtle timing." 

Quan and Michael are members of EMBOTS (Embodied Agents Research Group), a research group based in Germany. They've worked on a number of interesting projects with digital puppetry applications; you can find an overview of their work here.

Kinect Digital Puppetry Experiment

Here's a recent experiment in 2D digital puppetry, created using the Kinect and KinectSDK block for Cinder / C++ (compiled using Microsoft Visual Studio 2010). If you're able to program, you can find the code that was used to create this digital puppet here and try it yourself!

Power Puppets

Power Puppets are a series of digital puppet characters produced by Little Mountain Productions, a firm that provides a variety of design and production services to Christian churches.

There are a variety of Power Puppet 2D and 3D characters available. The characters are controlled using an X-Box controller. The software is available for both Windows and Mac and is sold as a standalone product, although Little Mountain also builds complete hardware systems as well.

SoftKinetic Digital Puppetry Demo

SoftKinetic used a digital puppetry demo to show off some new firmware for their DepthSense cameras at last week's CES 2012 trade show. It highlights their new support for finger recognition, something that's difficult to accomplish with the X-Box Kinect without a lot of hacking. The firmware is still in the alpha stage, but it looks like SoftKinetic is putting together a gesture recognition system that could be really useful for digital puppetry applications.

Via The Verge.

Digital Trekkie Monster

Yoke created this very cool interactive display as a publicity stunt to help promote performances of Avenue Q at the Fredericia Theater in Denmark. Passersby are able to directly control a digital version of Trekkie Monster (one of the puppet characters who star in Avenue Q) via an X-Box Kinect. A great example of how to use interactive technologies to market a product and, if you ask me, an all-around brilliant idea!

Via Kinect Hacks.

Digital Felix The Cat from the 1980s

I only discovered this recently while doing some research for PuppetVision: The Movie (see previous post)...did you know that Felix The Cat: The Movie was one of the first films to use motion capture technology to create a computer generated character?

This digital Felix head was performed by Eren Ozker (the primary female puppeteer during the first season of The Muppet Show) and is a pioneering example of digital puppetry. Felix The Cat: The Movie was made in Europe in 1987, which means that it predates even Waldo C. Graphic. "Digital Felix" might be the very first digital puppet to ever appear in a feature film!

Puppet Time

Puppet Time is a new project from students at Georgia Tech that aims to "bring the art of digital puppetry to a wide range of users by taking advantage of the motion tracking capabilities of Android smart phones".

There have been a lot of projects very similar to this in the last few years so this isn't terribly new, although the use of Android smart phones as controllers is an interesting twist. Puppet Time was built using Unity 3D; it works by having Android phones send sensor data that controls the onscreen digital puppets to a computer running the main Unity application over a wifi network.

If you want to try it out for yourself, both the Unity and Android apps are available to download for free.

Jim Henson Google Doodle and Digital Puppet Karoke

If you visit the Google homepage tonight or tomorrow the "Google Doodle" features six digital puppets that you can perform with your mouse. It's the result of a collaboration between The Jim Henson Company and Google to commemorate what would have been Jim Henson's 75th birthday tomorrow.

What I love most about this is that such a clever, but simple thing is unleashing people's creativity. Within hours of the Google Doodle appearing "digital puppet karoke" videos of the puppets lip syncing to songs started popping up on YouTube.

Here's one of The Black Eyed Peas' Boom Boom Pow:



...and another of "Earth Angel":



Neat, huh?

Another Real-time Digital Puppetry Demo

This is a fun little demo of a digital puppet that Seth Hunter is working on with puppeteer Dan Butterworth at the MIT Media Lab. Like a lot of digital puppetry projects such as Animata, it draws a lot of inspiration from shadow puppetry techniques. The software works by tracking points on a puppet that correlate in real-time to a digital version of the character. Users have the ability to change costumes and record animations and there are future plans to enable character customization as well as remote play.

We Be Monsters: Multi-Person Digital Puppets

Carnegie Mellon University has been a leader in interactive studies (including lots of digital puppetry work) for years now so it's not surprising that CMU students have been busy dreaming up some pretty cool projects using the Kinect. A good example of this is We Be Monsters, a Kinect hack by students Caitlin Rose Boyle and Asa Foster that turns participants into a two-person puppet, “inspired by multi-person Chinese dragon costumes and (Mr.) Snuffleupagus.” It's a cool project that's somewhat similar to the previously-blogged Animata software.

You can see more examples of the great work that Carnegie Mellon students are cooking up over at Create Digital Motion.

Kinect Digital Puppetry: It's Here and it's Real

Kinect hacking has come a long way in a few short months and as most people following it already know, the potential implications for digital puppetry are huge. I was at a small meet-up for Kinect hackers in Toronto last night and tried a very rudimentary version of "Kinect puppetry" that an animator I know has been working on and it was very cool. Affordable, accessible digital puppetry is here, it's real and if you already have a computer and a little technical know-how you can do it yourself for under $200.

Here's an impressive demo of Kinect puppetry using the Japanese "Vocaloid" Hatsune Miku using the Kinect and free software from the VPVP Project:



Aside from the Kinect itself (which retails for around $150), most of the software needed to try this yourself can be downloaded for free. Of course you need a 3D program like Blender along with some sort of 3D game engine. Fortunately "lite" or "learning" versions of commercial engines like Motionbuilder and Unity can be downloaded for free and some people are using Ogre 3D, which is open source. The exact methods to make your own digital puppets will vary depending on which software you use, but lots of tutorials can be found by searching online.

Not all digital puppetry with the Kinect has to be three dimensional though. Here's a neat 2D demo done in the style of shadow puppetry. It's part of a project called Virtual Marionette:



Although I do think this is huge step forward for open, accessible digital puppetry there are still some problems to solve before Kinect can be used to create truly emotive, expressive characters. For one thing, the skeletal tracking in these demos is very simple without individual finger movements or any kind of facial expression. Those problems aren't unsolvable though and I have some ideas on how to tackle them, but that's another post.

Cross-posted from PuppetVision.

More Kinect Digital Puppetry Developments

New X-Box Kinect developments seem to keep coming fast and furious. At CES 2011, Microsoft has been demoing it's forthcoming Avatar Kinect for X-Box Live; essentially it's a platform that allows you to control a 3D avatar of yourself in real-time via body and facial movements. I suspect that the technology isn't quite ready to go yet (Microsoft CEO Steve Ballmer's demo above was canned and not done live), but Engadget has a video of a live demo of Avatar Kinect here.

Meanwhile, in other Kinect news, someone has hacked together some impressive-looking real-time motion capture using the Kinect:



(Motion capture video via Blendernation)

Puppetry on the iPad

I'm really intrigued by the possibilities that Multi-touch devices like the iPad might offer puppeteers in the near future. Several developers are exploring exactly that and looking for ways to combine puppetry and interactive media. One of these is iPuppeteer, which allows you to control a simple, marionette-like puppet on an iPad, iPhone or iPod Touch. Another is Pollock's Toy Theatre, a new iPad app that rides the wave of resurgent interest in toy theatre that's been happening during the past few years and tries to take it in to the digital age.

While both of these apps seem to offer only basic functionality (for now), it will be interesting to watch the app market and see what develops over the next year or so.

Cross-posted from PuppetVision.

Pinokio 3D

There seems to be a lot of interesting things brewing on the digital puppetry front these days. In fact, I'm going to go out on a limb and suggest that 2011 might be a breakthrough year for real-time animation.

I hope to get back to sharing some of the exciting things that are happening out there in the new year, but until then enjoy "Pinokio 3D" a digital puppet created using Blender that debuted at the 14th Annual Circus, Clowns and Street Shows Argentine Conference in Argentina. The video is in Spanish, but more info is available (in English) in this thread on the BlenderArtists Forum.