Video and Blender's Game Engine

Work continues with Panda Puppet slowly, but surely. I haven't been able to share much of what I have been doing lately because my work is focused on two client projects that I can't really blog about at this point, but I am starting up my Bear Town web series project again and plan to be using Panda Puppet heavily on that in the coming months.

What has me most excited right now though is that "Ashsid" - who wrote a lot of the original Blender scripts that much of Panda Puppet is based on - has been working on a cool new video texture plugin for Blender's game engine that allows you to combine real-time graphics with various other sources like video files, live video, rendered 3d scenes, etc. inside Blender.

This has all sorts of interesting applications, the coolest of which is tracking the movement of a camera and applying that movement to a Blender object. Here's a demo of it in action with Suzanne (the famous Blender monkey head) combined with live webcam video:



What excites me most about this is that it enables puppeteers working with monitors to have physical puppets and digital ones interact in real-time. That isn't a new idea of course, but what's great about this is that now anyone can download the software and try it themselves.

Nice work Ashsid!

Head tracking with the Wiimote

Johnny Chung Lee (see previous post) has done it again! In his latest video he explains how to use the Wiimote and Wii sensor bar to make a head tracking system. As always, you can download the software to do this yourself from Johnny's site (link goes directly to a .zip file).

The most obvious application that I can see for this in digital puppetry is a quick and easy way to track a camera in 3D space. It could also be used to allow a puppeteer to see through a character's "eyes" in a virtual environment/set.

So many things to do with the Wiimote, so little time!

More About New Henson/PBS Digital Puppetry Show

Sid The Science Kid is the star of a forthcoming digital puppetry series for kids.

The Muppet Newsflash reports that work is continuing on the forthcoming Jim Henson Company/PBS Kids digital puppetry series What's The Big Idea? (see previous post), but the show has been retitled Sid The Science Kid. The series' concept is using short sketches to teach children lessons about science and discovery and is being produced using the Henson Company's HDPS Digital Puppetry System. Forty half-hour episodes of the show are currently in production; there's no word whether or not it has been affected by the current writer's strike in the United States.

Look for Sid The Science Kid on PBS sometime in 2008.

Multi-touch Interface using the Wii Remote

This is another incredibly cool Wiimote hack from Johnny Chung Lee (see previous post) that suggests all kinds of interesting possibilities for controlling digital puppet characters in real-time. Since the Wii tracks IR lights, Johnny has discovered that almost any surface - a projector screen, a tabletop, or an LCD - can be turned into a multi-touch screen that you can use like the iPod touch or iPhone. All you need is the Wiimote, a ball point pen equipped with an LED and some free software that can be downloaded here. Watch the video for details.

And as cool as it is, I'm even more excited to see the results of what Johnny is working on next - 3D tracking and head tracking with the Wiimote!

Special thanks to Brian for sending this to me!

Track Fingers With The Wiimote

Here's another potentially cool user interface for creating digital puppetry and Machinima...Johnny Lee at Carnegie Mellon University has worked out a method for tracking his fingers using the Wiimote's infrared camera. The basic idea is very similar to the iPhone and the Gesturetek technology I wrote about back in March.

Via MAKE.

Virtual Puppeteer Goes In To The Dragon's Den

Virtual Puppet, a UK company that "specializes in motion capture technologies and involves itself with the production of advertising, face to face experiential marketing and performance for theatre and live interactive television," was featured this past weekend on the British edition of Dragon's Den. For those unfamiliar with the show, the idea behind Dragon's Den is that entrepreneurs pitch their ideas to secure investment from a panel of business experts and self-made millionaires ("The Dragons"). The rules of the show stipulate that an entrepreneur must secure all of the financing they request from one or more Dragons or they go home with nothing. Virtual Puppet's Creative Director David Field appeared on the show to pitch the Dragons for £200,000 (roughly $410,000 US).

Virtual Puppet performs digital puppets at trade shows and retail locations using Alias' MotionBuilder software and the Gypsy motion capture system manufactured by Animazoo. You can see some examples of Virtual Puppet at work in the video above.

Wondering how David made out? Find out for yourself this clip from the show (sorry, but IE on a Windows machine required to view the clip).

Low Budget Real-Time Animation System

This is a very cool demonstration of a low-budget realtime animation system that Roland Dell'Mour developed for his diploma-thesis. The system uses Maya, PIE, a midi-motion-capture-server and several other tools. The neat-looking controllers are Saitek Cyborg EVO joysticks with game pads attached. I use Cyborg EVOs myself now for digital puppetry and I like them a lot.