Machin-X: Digital Puppetry

Power Puppets

2012-01-27T04:46:00.000-05:00

Power Puppets are a series of digital puppet characters produced by Little Mountain Productions, a firm that provides a variety of design and production services to Christian churches.

There are a variety of Power Puppet 2D and 3D characters available. The characters are controlled using an X-Box controller. The software is available for both Windows and Mac and is sold as a standalone product, although Little Mountain also builds complete hardware systems as well.

Machinima vs. Anymation

2012-01-19T15:00:00.000-05:00

Have you heard of "Anymation"?

The term first emerged a few years ago as way of classifying the work of digital creators who were willing to embrace any and all available tools in order to express themselves digitally. It was originally coined by Croatian animation artist Tom Jantol and seems to have emerged during online debates over what does and doesn't constitute Machinima.

From Tom's Anymation Manifesto:

What, then, is Anymation? And where does it fit into this multi-faceted world of Computer Generated Imagery?

Quite simply, Anymation is the embracing of any and all available tools in order to create a digital expression. The spirit of Anymation is not a rejection of any one of the tool-centric genres, nor is it a desire to reform them. Rather, Anymation is a conscious decision to be LESS CONSCIOUS of the barriers typically seen between the different types of digital expression, coupled with a "renaissance man" kind of willingness to harness ANY available tool or method to create that expression.

So, essentially, Machinima is Anymation, but Anymation isn't necessarily Machinima.

I like the idea of using whatever tool you want to achieve the effect you need. That's really what Machin-X has always been about, the blending of technologies and techniques to create new and interesting work.

It's a brave new creative world.

SoftKinetic Digital Puppetry Demo

2012-01-17T16:51:00.000-05:00

SoftKinetic used a digital puppetry demo to show off some new firmware for their DepthSense cameras at last week's CES 2012 trade show. It highlights their new support for finger recognition, something that's difficult to accomplish with the X-Box Kinect without a lot of hacking. The firmware is still in the alpha stage, but it looks like SoftKinetic is putting together a gesture recognition system that could be really useful for digital puppetry applications.

Via The Verge.

Kinect Puppet Parade

2012-01-12T10:56:00.000-05:00

Theo Watson and Emily Gobeille have taken their Kinect Shadow Puppet concept to the next level. Puppet Parade is an interactive Kinect installation that uses two Kinects to track the arm movements of children, enabling them to not only control on screen digital puppets, but also step in to a virtual environment and interact with them directly.

Have a look at a non-edited version of the video above so that you can hear the live sound and see how the hand movements are tracked and translated in to creature movements:

Via PSFK.

Facial Capture Using Blender 3D

2011-12-30T12:42:00.000-05:00

The latest version of Blender 3D was released recently and has a bunch of great new features. One of the most impressive is motion tracking, which enables you to reconstruct camera animation from video and film footage, and composite 3d rendered object into movie clips. The Blender community has been busy exploring the possibilities of this new feature and several people have been attempting to use it for rudimentary motion capture.

As you can see above and below, the results so far look pretty good!

If you're feeling ambitious and want to try this for yourself you can find a tutorial here.

I've always believed that Blender had the potential to become an ideal (and totally free) professional tool for creating digital puppetry. Although I don't believe that this works in real-time - I haven't had a chance to try it myself yet - this is a huge step in that direction and suggests that we can look forward to a lot more amazing things in the future.

Via BlenderNation.

Digital Trekkie Monster

2011-12-28T16:37:00.000-05:00

Yoke created this very cool interactive display as a publicity stunt to help promote performances of Avenue Q at the Fredericia Theater in Denmark. Passersby are able to directly control a digital version of Trekkie Monster (one of the puppet characters who star in Avenue Q) via an X-Box Kinect. A great example of how to use interactive technologies to market a product and, if you ask me, an all-around brilliant idea!

Via Kinect Hacks.

"The Internet is For Porn" screened before Congress

2011-12-17T02:15:00.000-05:00

If you haven't been following the debate over the controversial Stop Online Piracy Act (SOPA) in the United States, well, let's just say I have to hand it to the United States Congress. It takes a whole lot of stupid (or maybe just a whole lot of money from Hollywood studios and record labels) to come up with copyright legislation that is even more asine and one-sided than what they already have on the books in the U.S., but with SOPA they seem to have done it.

In any case, Rep. Jared Polis, a Democrat from Colorado with a refreshingly intelligent take on internet policy, injected a little levityby in to a debate on the bill by screening the Machinima mash-up The Internet Is For Porn before the House Judiciary committee. Rep. Polis used the video as part of an effort to add an anti-pornography amendment to SOPA, a tactically brilliant back-door legal manuevere that potentially could have made the entire piece of legislation unconstitutional.

The ammendment was ultimately defeated by a vote of 18 to 9, but not only was a Machinima film screened for the House Judiciary committee, but the entire lyrics of The Internet Is For Porn are now officially part of the congressional record. How silly (and kind of cool) is that?

Via BoingBoing.

Digital Felix The Cat from the 1980s

2011-11-29T14:05:00.004-05:00

I only discovered this recently while doing some research for PuppetVision: The Movie (see previous post)...did you know that Felix The Cat: The Movie was one of the first films to use motion capture technology to create a computer generated character?

This digital Felix head was performed by Eren Ozker (the primary female puppeteer during the first season of The Muppet Show) and is a pioneering example of digital puppetry. Felix The Cat: The Movie was made in Europe in 1987, which means that it predates even Waldo C. Graphic. "Digital Felix" might be the very first digital puppet to ever appear in a feature film!

PuppetVision: The Movie

2011-11-27T06:02:00.009-05:00

I’ve been writing my other blog, PuppetVision, about puppetry in all its forms since 2004. During that time puppetry has undergone an incredible renaissance. My original goal for PuppetVision was to share and help promote some of the incredible puppetry work that was being done by puppeteers all over the world. I'm proud so say that over the past seven and a half years close to half a million individual people have been able to discover and enjoy thousands of puppet films, videos and performances by hundreds of different artists thanks to the site.

What I want to do next is take what I've been doing with PuppetVision - and also here at Machin-X - to a whole new level.

For the past several years as I’ve been filming interviews with and performances by puppeteers in different cities and different countries literally all over the world. This is all going to come together in a documentary next year called PuppetVision: The Movie that will feature interviews with and performances by over sixty different puppeteers from about fifteen different countries, working in just about every style of puppetry that exists...including digital puppetry and Machinima.

My goal is to create a film that can show a mainstream audience (that is, people who don't know much about puppetry, digital or otherwise) just how much amazing work is being done around the world. I also want to give the artists who appear in the film as much exposure and publicity as possible. The finished film will be screened at film, animation and puppetry festivals internationally.

To that end I’ve launched a campaign on IndieGoGo to finish PuppetVision: The Movie. Funds raised will be used to license archival and stock footage that is needed to tell the film's story and to pay for professional sound mixing and colour correction. Best of all, if the campaign reaches its goal, I will be able to release the finished film online for free under a Creative Commons license!

PuppetVision: The Movie will be exploring the past, present and future of puppetry and as you know, I think digital technology is a big part of that future. If you’ve enjoyed Machin-X or the PuppetVision Blog for all these years, please support PuppetVision: The Movie. You’ll get some great perks and I’ll work very hard to not only make a great film, but also introduce you to some incredible puppeteers and show you some things that you’ve never seen before.

You can support this project by becoming a funder right now at IndieGoGo.com/puppetvision.

Thank you so much for your support!

Virtual Marionette Project

2011-11-09T22:33:00.005-05:00

Luis Miguel Barbosa is doing some interesting experiments with digital puppetry. His Virtual Marionette project uses an iPad to control a digital puppet character inside Animata (see previous posts). This takes advantage of the iPad's multi-touch interface to control multiple control points simultaneously in real-time.

He's also experimenting with something he calls WIIMATA, which uses two WIImotes to control a character inside Animata:

Neat stuff.

Puppet Time

2011-10-04T10:00:00.000-04:00

Puppet Time is a new project from students at Georgia Tech that aims to "bring the art of digital puppetry to a wide range of users by taking advantage of the motion tracking capabilities of Android smart phones".

There have been a lot of projects very similar to this in the last few years so this isn't terribly new, although the use of Android smart phones as controllers is an interesting twist. Puppet Time was built using Unity 3D; it works by having Android phones send sensor data that controls the onscreen digital puppets to a computer running the main Unity application over a wifi network.

If you want to try it out for yourself, both the Unity and Android apps are available to download for free.

Motion Capture, Performance and Avatar

2011-09-27T15:41:00.001-04:00

Some excerpts from a "making of" documentary about Avatar and its pioneering use of motion capture.

Jim Henson Google Doodle and Digital Puppet Karoke

2011-09-23T23:56:00.009-04:00

If you visit the Google homepage tonight or tomorrow the "Google Doodle" features six digital puppets that you can perform with your mouse. It's the result of a collaboration between The Jim Henson Company and Google to commemorate what would have been Jim Henson's 75th birthday tomorrow.

What I love most about this is that such a clever, but simple thing is unleashing people's creativity. Within hours of the Google Doodle appearing "digital puppet karoke" videos of the puppets lip syncing to songs started popping up on YouTube.

Here's one of The Black Eyed Peas' Boom Boom Pow:

...and another of "Earth Angel":

Neat, huh?

Facial Puppetry: Face tracking and replacement

2011-09-21T12:56:00.010-04:00

I've been casually doing some research in to face tracking and face substitution techniques for a film I'm currently directing and producing. I'm exploring the idea of blending conventional "Bunraku-style" puppetry techniques with real-time animation. What I'm hoping to be able to do is replace the face of a physical puppet with an animated one (preferably in real-time, although I might have to settle for some kind of post-production process).

The basic concept is to take a person’s expressions and map them in real-time to either a digital model of a face or match the expression with a photo of another face in an image database. There are a number of people doing some interesting work in this area, especially Jason Saragih who has created the FaceTracker library; you can see an example of what it can do in the video embedded above (I mentioned this briefly in my previous post).

This isn't really a new idea. Companies like ILM and Rhythm and Hues pioneered similar post production techniques back in the 1990s and they've been used for years to create talking animals in commercials, movies and TV shows. There has also been similar commercial software like CrazyTalk around for several years. What is relatively new is that the technology now works in real-time and is accessible to anyone with a decent computer and some basic programming knowledge.

Technology like this further blurs the line between puppetry and animation (which has been happening for awhile now) and offers artists the chance to have the best of both worlds. I really love the idea of being able to endow conventional puppets with a level of expression that just isn't possible in a physical puppet. Exciting stuff.

Digital Puppetry Round-up

2011-09-14T13:24:00.000-04:00

It's been far too long since I've updated Machin-X, so to try to catch up here's a look at bunch of interesting things that have been happening in the past few months:

Carnegie Mellon University and Disney Research are working on a system for "motion capture on the go" that utilizes body mounted cameras, which opens up some intriguing technological possibilities (via Tech Crunch).

In Australia, Jason Saragih is working on an interesting system for doing facial puppetry. His work on facial expression recognition looks particularly impressive so far.

Check out the visual insanity that is the Unreal 3 graphics engine.

Using the Kinect for digital puppetry keeps getting easier.

Digital puppetry with Wiimotes, using the Unity 3D engine.

Blender continues to be expanded and refined. The latest stable version (and last of the 2.5 series) Blender 2.59 was recently released and a number of exciting new features are being planned for future releases, including better support for motion capture and an improved real-time animation system.

Finally, old puppetry meets new technology with the "Naked Puppet" digital sock puppet.

Another Real-time Digital Puppetry Demo

2011-05-12T13:46:00.000-04:00

This is a fun little demo of a digital puppet that Seth Hunter is working on with puppeteer Dan Butterworth at the MIT Media Lab. Like a lot of digital puppetry projects such as Animata, it draws a lot of inspiration from shadow puppetry techniques. The software works by tracking points on a puppet that correlate in real-time to a digital version of the character. Users have the ability to change costumes and record animations and there are future plans to enable character customization as well as remote play.

Flash Animation Using Kinect

2011-04-08T02:22:00.006-04:00

The Dead Man and the Lawyer is a short film and experiment by Nick Fox-Gieg, a Canadian animator and all-round interesting guy I've had the pleasure of getting to know over the past few months. Although this might look like relatively straight forward 2D animation, he actually made it using the Kinect and I think this opens up some exciting possibilities for creating Kinect animation.

To make the film Nick used OSCeleton to record Kinect data to an .xml file via a utility he's written called Processing Sketch. The beauty of this approach is that the motion capture information was written to an .xml file in real-time without any need for video capture. That way the animation data can be read back from .xml in Flash, tweaked and then rendered in full HD. A short tutorial and download links for all the necessary utilities and libraries you need to try this yourself can be found on Nick's website.

Nick's also a semifinalist in YouTube's NextUp competition right now. If you like his work, show him some love and vote for him.

First Kinect Motion Capture Series - Under the HUD

2011-03-10T10:20:00.002-05:00

Under the HUD is what looks to be the first motion capture web series created using the Kinect. It was made using the OpenNI and Brekel Kinect hacks, as well as Autodesk's MotionBuilder and Maya software. Although the animation isn't completely fluid or elegant, it's not bad either. The important point is that, as Fast Company points out, until recently you needed to spend thousands (not the millions claimed by FC) to do this.

Episodes are coming soon; you can follow @thwackers on Twitter and visit the Thwackers website for updates.

Via Kinect Hacks.

New Sesame Street Game For Kinect

2011-03-04T10:00:00.000-05:00

Here are some digital puppets in a very literal sense...Sesame Street: Once Upon a Monster is a new game for Xbox Kinect by acclaimed games studio Double Fine. The family-friendly game promises to take players into a living storybook populated with famous Muppet characters like Elmo, Cookie Monster and Oscar the Grouch. Gamasutra has an interview with Double Fine's Nathan Martz and Tim Schafer discussing the development of the game in depth.

The team at Double Fine is obviously having fun with this project (and who wouldn't?). Have a look at this video of Tim Schafer showing off some "fur rendering" technology.

Once Upon a Monster looks like it will be a lot of fun. It's due for release this fall.

Via Tough Pigs.

We Be Monsters: Multi-Person Digital Puppets

2011-03-02T04:26:00.016-05:00

Carnegie Mellon University has been a leader in interactive studies (including lots of digital puppetry work) for years now so it's not surprising that CMU students have been busy dreaming up some pretty cool projects using the Kinect. A good example of this is We Be Monsters, a Kinect hack by students Caitlin Rose Boyle and Asa Foster that turns participants into a two-person puppet, “inspired by multi-person Chinese dragon costumes and (Mr.) Snuffleupagus.” It's a cool project that's somewhat similar to the previously-blogged Animata software.

You can see more examples of the great work that Carnegie Mellon students are cooking up over at Create Digital Motion.

Chris O’Shea's Little Magic Stories

2011-02-24T00:05:00.003-05:00

The more artists explore new frontiers in interactive art, the more I often mind myself wondering where exactly to draw the line between what is animation and what is puppetry. A good example of this is the augmented shadow puppetry that I recently wrote about over at PuppetVision, which is sort of shadow puppetry, but also sort of not.

Another good example is this performance system created by Chris O’Shea called Little Magic Stories. It was made using the X-Box Kinect (see previous post) and the old Pepper's Ghost Vaudeville trick.

So is this animation? Digital puppetry? Or something else entirely?

Via CrunchGear.

Why Realism Doesn't Work in Animation (or Machinima)

2011-02-13T23:45:00.011-05:00

Salon.com has a great interview with character designer Shannon Tindle. The article mostly discusses character design and why so many animated characters (like the ones in the recent "Gnomeo and Juliet") look terrible, but also touches on the subject of motion capture animation vs. animator-driven key frame animation.

Shannon is very diplomatic throughout the interview, but makes his feelings known about movies that skew towards realism in terms of both their design and their use of motion capture. Here's what he had to say about "motion capture" animated movies like The Polar Express and A Christmas Carol:

"All I know is that the studio that Disney formed to produce those films, Image Movers, doesn't exist anymore. "Christmas Carol" is one of my favorite stories of all time. But with those films, it doesn't look exactly like a real person and so it becomes something in between. In any animated film -- stop-motion, CGI and 2-D, and I've worked in all of those mediums -- you need to make a clear statement. Any time you waffle, if you're somewhere in between reality and stylization, a straight line and a curve, people feel it and they tend to have a bad reaction to it."

To me this is also one of the biggest barriers preventing real-time animation from reaching a more mainstream audience. Machinima might be getting more and more popular, but it's appeal is still very niche compared to mainstream animation. The movement in most real-time animation is driven by "realistic" mocap and/or skeletal tracking. At the same time, it's difficult to find an example of a real-time animated character that exhibits as much personality as Mickey Mouse did in the early Disney cartoons of the 1930s. I think there's a direct connection there; in order to be successful in the mainstream, audiences have to be able to connect emotionalally with the characters they're watching. As Shannon points out, it's much harder to do that using realistic motion or design.

Many Machinima creators are really innovative and do an excellent job of working with and around the limitations in the software they use to make their movies, but what they really need is a platform that allows them to create better, more emotive and relate-able characters so that their imaginations can be set free.

Salon.com article via Cartoon Brew.

Kinect Digital Puppetry: It's Here and it's Real

2011-02-09T19:08:00.012-05:00

Kinect hacking has come a long way in a few short months and as most people following it already know, the potential implications for digital puppetry are huge. I was at a small meet-up for Kinect hackers in Toronto last night and tried a very rudimentary version of "Kinect puppetry" that an animator I know has been working on and it was very cool. Affordable, accessible digital puppetry is here, it's real and if you already have a computer and a little technical know-how you can do it yourself for under $200.

Here's an impressive demo of Kinect puppetry using the Japanese "Vocaloid" Hatsune Miku using the Kinect and free software from the VPVP Project:

Aside from the Kinect itself (which retails for around $150), most of the software needed to try this yourself can be downloaded for free. Of course you need a 3D program like Blender along with some sort of 3D game engine. Fortunately "lite" or "learning" versions of commercial engines like Motionbuilder and Unity can be downloaded for free and some people are using Ogre 3D, which is open source. The exact methods to make your own digital puppets will vary depending on which software you use, but lots of tutorials can be found by searching online.

Not all digital puppetry with the Kinect has to be three dimensional though. Here's a neat 2D demo done in the style of shadow puppetry. It's part of a project called Virtual Marionette:

Although I do think this is huge step forward for open, accessible digital puppetry there are still some problems to solve before Kinect can be used to create truly emotive, expressive characters. For one thing, the skeletal tracking in these demos is very simple without individual finger movements or any kind of facial expression. Those problems aren't unsolvable though and I have some ideas on how to tackle them, but that's another post.

Cross-posted from PuppetVision.

The N-Controller Avenger

2011-02-08T00:42:00.005-05:00

The N-Controller Avenger is a clip-on product for standard X-Box controllers "designed for harcore gamers who want to take their abilities to the next level." Essentially, it's a casing that fits over a regular controller so that you can keep your thumbs on the controller's analog sticks while simultaneously having all of the buttons within reach. You can see more videos of it in action on the Avenger Controller website.

Although I'm admittedly not a hardcore gamer, the N-Controller seems a bit like overkill for most video games to me, but I do wonder about it's potential for digital puppetry applications. I've tried doing digital puppetry with game pads in the past and never liked it. At just $40 each I'm tempted to order one of these and try it out to see if it's useful.

Via Joystiq.

Machinima.com Gets Techcrunched

2011-02-01T19:23:00.008-05:00

Techcrunch - one of the web's leading tech industry blogs - has an interesting post about Machinima.com today talking about the huge audience numbers the website has been racking up; 2.3 billion video views last year (350 million in of them in December alone)and over 45 million unique visitors. The article also mentions that Machinima.com has over 90 staff in L.A. producing their various online shows every week.

Part of the article that will especially warm the hearts of longtime Machinima creators is this:

He (Machinima.com owner Allen DeBevoise) tried to raise money back then (in 2005) but the audience was seen as too niche, maybe 5 million people if Machinima was lucky. Clearly everyone underestimated what was brewing here.

I think that's a huge understatement. Machinima isn't quite mainstream yet, but these numbers prove that it's already taken off as a medium in a very big way.

Kinect-Powered Hologram

2011-01-25T21:51:00.002-05:00

Another amazing Kinect hack...researchers at MIT built a rudimentary holographic video system using an X-Box Kinect. They even re-create the famous hologram of Princess Leia from Star Wars in this demo video.

Real-time, 3D holographic digital characters anyone?

Via Fast Company.

More Kinect Digital Puppetry Developments

2011-01-07T10:02:00.003-05:00

New X-Box Kinect developments seem to keep coming fast and furious. At CES 2011, Microsoft has been demoing it's forthcoming Avatar Kinect for X-Box Live; essentially it's a platform that allows you to control a 3D avatar of yourself in real-time via body and facial movements. I suspect that the technology isn't quite ready to go yet (Microsoft CEO Steve Ballmer's demo above was canned and not done live), but Engadget has a video of a live demo of Avatar Kinect here.

Meanwhile, in other Kinect news, someone has hacked together some impressive-looking real-time motion capture using the Kinect:

(Motion capture video via Blendernation)

Puppetry on the iPad

2011-01-03T10:15:00.002-05:00

I'm really intrigued by the possibilities that Multi-touch devices like the iPad might offer puppeteers in the near future. Several developers are exploring exactly that and looking for ways to combine puppetry and interactive media. One of these is iPuppeteer, which allows you to control a simple, marionette-like puppet on an iPad, iPhone or iPod Touch. Another is Pollock's Toy Theatre, a new iPad app that rides the wave of resurgent interest in toy theatre that's been happening during the past few years and tries to take it in to the digital age.

While both of these apps seem to offer only basic functionality (for now), it will be interesting to watch the app market and see what develops over the next year or so.

Cross-posted from PuppetVision.

Pinokio 3D

2010-12-17T11:35:00.004-05:00

There seems to be a lot of interesting things brewing on the digital puppetry front these days. In fact, I'm going to go out on a limb and suggest that 2011 might be a breakthrough year for real-time animation.

I hope to get back to sharing some of the exciting things that are happening out there in the new year, but until then enjoy "Pinokio 3D" a digital puppet created using Blender that debuted at the 14th Annual Circus, Clowns and Street Shows Argentine Conference in Argentina. The video is in Spanish, but more info is available (in English) in this thread on the BlenderArtists Forum.

Kinect Interactive Puppet

2010-11-20T22:56:00.003-05:00

Long (very long) time no post here...I've been very excited about the new Kinect for Xbox 360, which seems to offer all kinds of interesting possibilities for digital puppetry, especially since it's already been hacked.

Two designers, Emily Gobeille and Theo Watson, have whipped up this digital puppet prototype with it. It's a 2D puppet that uses skeleton tracking on an arm to control the movement and posture of the puppet. They hacked it together in a day using open-source Kinect drivers, which is pretty cool.

I think we'll see a lot more stuff like this using the Kinect very, very soon.

Blender Game Engine Evolution

2009-06-13T12:42:00.002-04:00

As you can see in this demo, Blender's Game Engine has come a long way in a short time (the video above was rendered in real-time, not frame-by-frame). Actually, the GE is capable of even more than what this video demonstrates; the latest version (2.49) was recently released and includes a bunch of cool features including video textures and real-time dome rendering as well as improved modifier support, speed, logic and physics.

As the GE gets better and better, more and more people seem to be interested in using Blender to create Machinima and digital puppetry. You can download the latest version of Blender from Blender.org.

The next release of Blender, version 2.5 is a major overhaul of the program's underlying code, some of which dates back to 1994(!). I'm especially excited about the proposed new animation system “Animato” and the new event system, which you can see demonstrated below:

Although I don't think the GE is the focus on the 2.5 release, you can bet that at least a few improvements are on the way. Experimental builds of Blender 2.5 can be downloaded from Graphicall, but they may not be stable and aren't necessarily suitable for production use yet.

Terminator Salvation: The Machinima Series

2009-06-02T12:45:00.002-04:00

I haven't had time to check out Terminator Salvation: The Machinima Series or the feature film from whence it sprung for that matter, but it certainly looks good. The six-part series serves as a prequel to the feature film and features Blair Williams, played in the film by Moon Bloodgood, who voices the character in the Machinima series.

The series was made entirely using in-game assets from the Terminator Salvation video game and was overseen by Terminator Salvation director McG, who's building an innovative little empire for himself in Hollywood these days. This is the first official studio-produced Machinima series and it's such a great example of cross-platform synergy that it's easy to foresee a lot more of these kinds of projects in the future.

Terminator Salvation: The Machinima Series is available to download from iTunes and the X-Box Live Marketplace. Free recaps of each episode are available to watch on Machinima.com.

Bryn Oh Second Life Machinima

2009-05-11T00:20:00.002-04:00

I love Bryn Oh's trippy, fantastical Second Life Machinima. Read more about her on PuppetVision and watch her latest work, filmed in the Immersiva sim in Second Life.

An Animator Praises Sid the Science Kid

2009-04-11T23:35:00.005-04:00

Keith Lango thinks the use of digital puppetry on Sid the Science Kid makes sense and I have to agree (but then you knew that, didn't you?). More and more I can see how puppets and pixels really are coming together. Check out the comments for the above-linked post to read some interesting discussion on puppetry, mo-cap and animation.

BTW, new episodes of Sid The Science Kid start the week of May 4th on PBS.

Creature Shop Open House Recap

2009-01-15T17:16:00.001-05:00

Debra Kaufman has written a nice recap of the previously-blogged Siggraph/Visual Effects Society event at the Creature Shop (see previous post). A couple people who were there Tuesday night wrote in to tell me about it yesterday; it sounds like it was a great event.

Digital Puppetry Open House at Jim Henson's Creature Shop

2009-01-10T14:51:00.007-05:00

Puppeteers bring characters to life with the Jim Henson Company's patented Henson Digital Performance Studio.

The Visual Effects Society and LA ACM SIGGRAPH are hosting a very cool-sounding open house of sorts at Jim Henson's Creature Shop next week called Innovation in Puppetry: From Animatronics to CGI next Tuesday, January 13th at 6 p.m. The program includes a presentation about the Creature Shop, a live demo of the Henson Digital Performance Studio, an audience Q & A, guided tours of Jim Henson's Creature Shop and self-guided tours of the rest of the Henson lot.

This event is free to LA ACM SIGGRAPH and Visual Effects Society members and $20 for non-members. New members who sign up on site and pay the $40 annual membership fee will be admitted for free. If you live in or near L.A. and have an interest in the Creature Shop's work or have always wanted to see the Henson lot, this sounds like an opportunity you shouldn't miss.

Note: You're supposed to RSVP at the Visual Effects Society web site, but I couldn't figure out how to do it so if you're planning to attend, it's probably best to email them directly to sign up.

Digital Puppetry with Animata

2008-12-14T11:57:00.006-05:00

Animata is open source real-time animation software, "designed to create animations, interactive background projections for concerts, theatre and dance performances". The goal of Animata is to make it really easy to create scenes with digital puppets that can be controlled via physical sensors, microphones or cameras.

Animata is another very promising digital puppetry system and it's already been used for a couple of interesting projects. Animata Jazz Pub is an interesting demo that shows how the Animata software can be used to react to live audio input. Reverse Shadow Theatre (seen above) is an installation that was created with Animata, inspired by Wayang Kulit, an Indonesian form of shadow puppet theatre.

If you want to try playing around with the software yourself, the Animata software can be downloaded for free for both the Mac and PC. The source is licensed under the GPL and can also be downloaded from Google Code.

Reverse Shadow Theatre link via Mary Robinette-Kowal.

How Waldo Worked

2008-10-15T04:37:00.004-04:00

In this clip from Secrets of the Muppets Jim Henson explains how Waldo C. Graphic - one of the first digital puppet characters - was created and performed (the Waldo segment clicks in at about the 5:20 mark).

Digital Puppetry with Microsoft's Touchless

2008-10-08T18:42:00.005-04:00

Touchless is the name of new multitouch software from Microsoft Office Labs. Much like the recently-blogged CamTrax software (see previous post), it allows you to use almost anything to create input for a computer via an ordinary webcam. MS Labs has released a software developer kit under an open-source Microsoft Public License that can be used to create custom apps, like controllers for digital puppetry purposes.

I really think that the concept of optically tracking everyday objects (I believe the technical term is "colour blob tracking") has enormous potential for (affordable) digital puppetry; much more so than motion capture. I've added this to my already-lengthy list of technologies to check out as soon as I have some time to spare.

Via TechCrunch.

A Wii For Everyone

2008-09-30T10:50:00.000-04:00

Okay, this is very exciting...a company called Cam-Trax Technologies has developed software that allows almost any PC webcam (they say 95% of webcams are supported) to track up to four objects in real-time and with very high accuracy and reliability. Essentially, this allows you to "build your own Wii" (minus the $250 console and controllers) and opens up lots of very exciting possibilities.

The way it works is that you download an emulator to your PC (it only works with Windows unfortunately) and build a controller for your favourite video game. While the four objects is a little limiting, I don't see why this couldn't be used for conrolling digital puppets.

I have to play around with this sometime in the next week or two.

Via TechCrunch.

Behind the Scenes of Sid the Science Kid

2008-09-28T14:25:00.010-04:00

The digital puppetry series Sid the Science Kid (see previous post) is now on the air and TV Guide recently went behind the scenes to see how the Jim Henson Company creates it. The Wall Street Journal has also been covering the show's unique production process.

Speaking of which, Brian Henson was in Silver Spring, MD at the AFI Silver Theatre yesterday and gave a presentation about the history of puppetry and how the Henson Co. has evolved over the years from Muppets to animatronics to it's latest work on Sid the Science kid using the Henson Digital Performance System.

TV Guide video via Puppeteers Unite.

29/09/08 Update: Brian Henson's presentation was at the AFI Silver Theatre in Silver Spring, MD and not the Centre for Puppetry Arts as I originally reported so I've updated the post accordingly. Brian actually spoke at the Centre for Puppetry Arts on Sunday and Harvey Deneroff has some coverage on his blog.

Panda Puppet Update

2008-09-23T16:12:00.006-04:00

It's been quite awhile since I updated Machin-X and for that I apologise; I'm hoping to be able to do more frequent updates in the future. I've gotten dozens of emails about Panda Puppet over the past few months so I wanted to post an update about the project, where it's at and what is (and isn't) happening with it right now.

I have not done any serious work on Panda Puppet for most of this year. This is due in part because I've been busy on other puppetry-related projects, but also because I have been waiting for a number of new features in Blender that I believe will improve it's usability for Machinima/digital puppetry purposes.

I do currently have a very crude working version of Panda Puppet (for Blender 2.46),but it's extremely basic, hacked together and not really suitable for public release (browse the Panda Puppet posts to see some of the old demos). There have been lots of great improvements to Blender's game engine this year (most of them developed for the Apircot video game project). The two huge new features that I've been waiting for are full character animation in real-time and real-time softbodies in Bullet, Blender's physics engine. All of the new features developed for Apricot will be included in Blender 2.48, which is tentatively scheduled for release next week. Once that is out and Ashid's great video texture plugin is updated to work with 2.48 I think it'll be time to get back to serious work on Panda Puppet again.

There seems to be serious interest in putting together a small developer's site for the project and I'm also applying for a grant that would help speed along development on Panda Puppet. In the meantime, I would love to get some help with the project. Is anybody out there handy with rigging armatures in Blender? If you are drop me a line at puppetvision {at} gmail dot com.

Second Life Lip Sync

2008-03-24T14:00:00.006-04:00

My significant other and I hosted a small dinner party over the weekend attended by a number of fantastically talented and creative folks, one of whom is an artist who has been deeply involved in Second Life for awhile now. I talked about Panda Puppet a bit with him and we had a long discussion about Second Life and it's potential. I had rejected the idea of using Second Life for digital puppetry/Machinima purposes, but his enthusiasm for it coupled with the recent announcement of a basic lip sync tool for Second Life makes me want to reconsider (helpful lip sync tips can be found on The Second Life Wiki).

Moviesandbox Looks Promising

2008-02-08T10:22:00.000-05:00

For quite awhile now Friedrich Kirschner has been developing Moviesandbox, a really promising set of open source tools for creating real-time animation. In the video above Friedrich demonstrates some of Movesandbox puppeteering possibilities using a Gametrak controller and a Wiimote. You can find more videos documenting the features of Moviesandbox here.

The latest beta version of Moviesandbox is available to download for free, but it requires you to have Unreal Tournament 2004 installed on your computer in order to make it work.

Video via Puppeteers Unite.

Video and Blender's Game Engine

2008-01-02T04:05:00.001-05:00

Work continues with Panda Puppet slowly, but surely. I haven't been able to share much of what I have been doing lately because my work is focused on two client projects that I can't really blog about at this point, but I am starting up my Bear Town web series project again and plan to be using Panda Puppet heavily on that in the coming months.

What has me most excited right now though is that "Ashsid" - who wrote a lot of the original Blender scripts that much of Panda Puppet is based on - has been working on a cool new video texture plugin for Blender's game engine that allows you to combine real-time graphics with various other sources like video files, live video, rendered 3d scenes, etc. inside Blender.

This has all sorts of interesting applications, the coolest of which is tracking the movement of a camera and applying that movement to a Blender object. Here's a demo of it in action with Suzanne (the famous Blender monkey head) combined with live webcam video:

What excites me most about this is that it enables puppeteers working with monitors to have physical puppets and digital ones interact in real-time. That isn't a new idea of course, but what's great about this is that now anyone can download the software and try it themselves.

Nice work Ashsid!

Head tracking with the Wiimote

2007-12-27T22:07:00.000-05:00

Johnny Chung Lee (see previous post) has done it again! In his latest video he explains how to use the Wiimote and Wii sensor bar to make a head tracking system. As always, you can download the software to do this yourself from Johnny's site (link goes directly to a .zip file).

The most obvious application that I can see for this in digital puppetry is a quick and easy way to track a camera in 3D space. It could also be used to allow a puppeteer to see through a character's "eyes" in a virtual environment/set.

So many things to do with the Wiimote, so little time!

More About New Henson/PBS Digital Puppetry Show

2007-12-21T11:49:00.000-05:00

Sid The Science Kid is the star of a forthcoming digital puppetry series for kids.

The Muppet Newsflash reports that work is continuing on the forthcoming Jim Henson Company/PBS Kids digital puppetry series What's The Big Idea? (see previous post), but the show has been retitled Sid The Science Kid. The series' concept is using short sketches to teach children lessons about science and discovery and is being produced using the Henson Company's HDPS Digital Puppetry System. Forty half-hour episodes of the show are currently in production; there's no word whether or not it has been affected by the current writer's strike in the United States.

Look for Sid The Science Kid on PBS sometime in 2008.

Multi-touch Interface using the Wii Remote

2007-12-13T12:57:00.000-05:00

This is another incredibly cool Wiimote hack from Johnny Chung Lee (see previous post) that suggests all kinds of interesting possibilities for controlling digital puppet characters in real-time. Since the Wii tracks IR lights, Johnny has discovered that almost any surface - a projector screen, a tabletop, or an LCD - can be turned into a multi-touch screen that you can use like the iPod touch or iPhone. All you need is the Wiimote, a ball point pen equipped with an LED and some free software that can be downloaded here. Watch the video for details.

And as cool as it is, I'm even more excited to see the results of what Johnny is working on next - 3D tracking and head tracking with the Wiimote!

Special thanks to Brian for sending this to me!

Track Fingers With The Wiimote

2007-11-26T11:12:00.000-05:00

Here's another potentially cool user interface for creating digital puppetry and Machinima...Johnny Lee at Carnegie Mellon University has worked out a method for tracking his fingers using the Wiimote's infrared camera. The basic idea is very similar to the iPhone and the Gesturetek technology I wrote about back in March.

Via MAKE.

Virtual Puppeteer Goes In To The Dragon's Den

2007-11-20T02:07:00.000-05:00

Virtual Puppet, a UK company that "specializes in motion capture technologies and involves itself with the production of advertising, face to face experiential marketing and performance for theatre and live interactive television," was featured this past weekend on the British edition of Dragon's Den. For those unfamiliar with the show, the idea behind Dragon's Den is that entrepreneurs pitch their ideas to secure investment from a panel of business experts and self-made millionaires ("The Dragons"). The rules of the show stipulate that an entrepreneur must secure all of the financing they request from one or more Dragons or they go home with nothing. Virtual Puppet's Creative Director David Field appeared on the show to pitch the Dragons for £200,000 (roughly $410,000 US).

Virtual Puppet performs digital puppets at trade shows and retail locations using Alias' MotionBuilder software and the Gypsy motion capture system manufactured by Animazoo. You can see some examples of Virtual Puppet at work in the video above.

Wondering how David made out? Find out for yourself this clip from the show (sorry, but IE on a Windows machine required to view the clip).

Low Budget Real-Time Animation System

2007-10-24T11:36:00.000-04:00

This is a very cool demonstration of a low-budget realtime animation system that Roland Dell'Mour developed for his diploma-thesis. The system uses Maya, PIE, a midi-motion-capture-server and several other tools. The neat-looking controllers are Saitek Cyborg EVO joysticks with game pads attached. I use Cyborg EVOs myself now for digital puppetry and I like them a lot.

Halloween Themed Digital Puppets Available To Download

2007-09-29T10:08:00.001-04:00

Now this is a pretty cool use of digital puppetry and something fun for Halloween; a company called ImaginEERIEing sells some basic computer programs that display a digital puppet on a Windows of Mac computer that can be controlled using a keyboard or microphone. It's also possible to playback a prerecorded performance too. It looks especially effective when the computer screen is hidden in a facade like it is in the video above.

There are currently three different characters available. The "Magic Mirror" character seen in the video can be downloaded for free; there's also a skull and a talking pumpkin available for $15 US each.

Via The Odd-Lot Puppetry Company.

Henson Announces New Digital Puppetry Series

2007-09-05T20:10:00.000-04:00

The Jim Henson Company is partnering with PBS’ KCET to produce a new educational digital puppetry series entitled What's the Big Idea? using Henson's HDPS real-time animation/digital puppetry system. The goal of the series is to promote science readiness in preschool children. Forty-eight episodes of the series have been ordered. The show is scheduled to debut in the United States on PBS stations next year.

Quick Experiment With Panda Puppet

2007-09-04T06:26:00.000-04:00

This was just a quick experiment that I did yesterday...I took a head that was modeled and rigged by Calvin over at BlenderArtists.org and set it up so that it could be puppeteered right inside a standard Blender window (i.e. not using the game engine) with a joystick. It was quick to set-up and fairly easy to do, I may experiment a bit more with this.

BusinessWeek on the Future of Machinima

2007-09-01T11:14:00.000-04:00

BusinessWeek recently ran a story on Machinima and how more and more game developers are packaging their titles with Machinima creation tools. It's a good read that outlines the history of Machinima and ponders where the artform is headed in the future.

More Skrumps Stuff

2007-09-01T00:01:00.000-04:00

This is kind of old news, but there's some new-ish content featuring the Henson Company's digital puppetry characters The Skrumps (see previous post) available on Yahoo Kids. The new material includes four new Skrumps videos, ecards and a few other downloadable goodies. Also, The Muppet Newsflash recently posted an interview with Victor Yerrid, who is one of the puppeteers working on The Skrumps.

Brian Windsor wins Innovation Award for Digital Puppetry

2007-07-25T12:02:00.001-04:00

This is just a quick post to say congratulations to Brian Windsor, who won the Puppeteers of America Jim Henson Award for Innovation for his work in digital puppetry at the recent Puppet Rampage 2007 festival in St. Paul, MN. Awarded by Puppeteers of America biannually, The Jim Henson Award for Innovation is given to recognize "innovation in puppetry that is technological, dramaturgical or collaborative in nature".

Brian and I held an impromptu demo of Brian's impressive digital puppetry work and the still work-in-progress Panda Puppet system at Puppet Rampage last week and I got to know him a bit. He's a great guy who's doing really great work and his award is well-deserved.

Congrats Brian!

Digital Puppeteer Interview: Julianne Buescher

2007-07-08T03:45:00.000-04:00

The Muppet Newsflash has an interview with puppeteer Julianne Buescher, who specializes in performing with the Henson Company's HDPS animatronic/digital puppetry control system. In the interview Julianne talks about the work she's done on a variety of Jim Henson company projects over the years, including performing digital puppet characters.

You can also learn more about Julianne on The Muppet Wiki.

Nintendo to Launch Indie Game Development For the Wii

2007-06-29T11:00:00.000-04:00

GigaOM reports that Nintendo wants to open up the Wii platform to indie developers next year. Dubbed "Wii Ware", this new service will be designed to enable developers to create small, downloadable games for the Wii in a manner similar to Microsoft's XNA Game Studio for the XBox 360. Considering the Wii's motion-control capability I think this is great news for digital puppetry and suggests all kinds of exciting developments could be on the horizon. It's really great to see Nintendo embracing the potential of user generated content.

Thanks to Brian for the heads up about this!

Quick Panda Puppet Update

2007-05-23T00:01:00.000-04:00

There's been some renewed interest in Panda Puppet recently in the blogosphere and I've been getting emails asking about the status of it so I wanted to post a quick update. Long story short I had a somewhat unexpected death in my family back in April and some of the things I was working on (primarily Panda Puppet) had to be set aside for a little while, which I'm sure everyone can understand. I'll be returning to work on Panda Puppet more or less full-time in a few weeks and when that happens I'll bring everybody up to speed on all the latest.

For those of you visiting Machin-X for the first time, read these posts to get a general overview of the project.

CalArts Teaches MoCap

2007-05-17T02:50:00.000-04:00

It's not secret that most traditional animators hate motion capture (often with good reason). Well, the good traditional animation minded folks at CalArts decided to have some fun and produced this video as the opening for the school's annual showcase of student showcase last week. It's pretty funny and apparently got more laughs than just about anything else that was screened.

Via Cartoon Brew.

Michael Bay Gets In To Machinima

2007-05-16T13:46:00.000-04:00

Looks like Michael Bay and his Digital Domain FX shop are the latest Hollywood players to jump aboard the Machinima bandwagon:

As foreshadowed by the studio's "Mad World" Gears of War TV spot, Digital Domain won't be using game engines for just games. Also on the agenda will be a series of teen-targeted animated features, which will utilize ~~machinima~~ er, a "new" cost-effective animation method. Does Hollywood actually have anything to offer the games industry? Or are Bay & co. just hopping over the fence 'cause the grass smells greener? "I make world-class images," says Bay. "Why not put those images into a game?"

It'll be interesting to watch this play out. Thanks to Brian for the link!

Conan O'Brien Gets Motion Captured

2007-05-07T11:52:00.000-04:00

Conan O'Brien recently paid a visit to Lucasfilm and among other things spent time goofing around with ILM'sreal-time motion capture system , which looks really impressive (the motion capture segment start about halfway through the video above).

Link via Cartoon Brew.

Why Machinima Is Different

2007-05-01T19:19:00.000-04:00

Peter Rasmussen - an indie filmmaker who crossed in to Machinima and is now producing a feature length Machinima film called Stolen Life - has written an interesting piece called Why Machinima Is Different that's worth a read. I especially liked the way he compares the current state of Machinima to the miliseconds after the Big Bang, full of pieces that are very basic but very powerful.

Via Overman's Blog.

ANIMAC: Digital Puppetry in the 1960s

2007-04-28T15:18:00.000-04:00

A dancer performs using ANIMAC - likely the world's first digital puppetry system - in 1962.

ACCAD at Ohio State University offers a course called A Critical History of Computer Graphics and Animation (link via BlenderNation). As you might expect, the course examines the history of the computer graphics, which stretches back much further than you might expect. The course section that I found to be the most interesting-looking was part twelve which looks at the work of Lee Harrison III, an computer graphics pioneer who in the 1960s developed ANIMAC, which was probably the world's first digital puppetry system.

A .pdf file with additional information about Lee Harrison and ANIMAC is available here.

The End of Red Vs. Blue

2007-04-04T23:06:00.000-04:00

Pioneering Machnima creators Rooster Teeth Productions have announced that they will be ending their insanely popular series Red vs. Blue with the series' upcoming 100th episode. Red vs. Blue is significant for many reasons, not the least of which was that it was a huge viral hit that put Machinima on the map. Six episodes left to go and if you've never seen the show before now seems like a great time to go check it out and see what you've been missing.

Via Thinking Machinima.

One Puppeteer's Journey From Puppets to Pixels

2007-03-30T01:55:00.000-04:00

The fence that divides puppetry and animation is a pretty low one and lots of artists like Phil Vischer, Karen Prell, Mike Quinn, and Nate Pacheco (to name just a few) find themselves jumping from one side to the other mid-career and Craig Crane is another of those puppeteer/animator hybrids.

Craig started his career in the 1990s doing a lot of ground-breaking work in real-time animation and performance capture as well as conventional puppeteering on various projects in England, France and Isreal, but these days he's an animator for Double Negative, a British effects shop. Craig discussed his transition from puppetry to animation in a two part interview with 3D World Magazine last fall (click to read part one and part two).

Craig seems to be pretty happy in the animation world these days, but I really hope he keeps a hand (no pun intended) in the puppetry world. He really is a brilliant puppeteer; you can check out is Real-Time animation demo reel above and his preschool puppet reel is a lot of fun too.

Cross-posted from The PuppetVision Blog.

Gameplay Boogie

2007-03-27T00:32:00.000-04:00

Gamers will be able to use the Wii's innovative controllers and some digital puppetry techniques to help an alien bust a move in Boogie.

EA regularly gets critiqued in the video game world for a lack of innovation. It's not hard to see why; the company seems to just churn out one relatively uninventive version of Madden Football after the other. That may finally be about to change though thanks to Boogie, one of their upcoming titles for the Wii.

On the surface Boogie sounds like another Dance Dance Revolution clone, but what makes it different is that instead of pushing buttons to make an onscreen character "dance", players have to make their character move in time with music using the Wii Remote and its Nunchuk attachment. The Nunchuk controls the character's body while the Wiimote controls its head. Players have to invent moves and dance routines with minimal guidance from the game and they succeed based on the variety of their moves and how "stylish" they are.

Sounds suspiciously like a digital puppetry game, doesn't it?

James Cameron Talks Avatar

2007-03-23T17:19:00.000-04:00

Businessweek posted an interview with James Cameron today. In it he talks about his new in-production movie Avatar, which will be utilizing a brand new motion capture-based, real-time animation system that Cameron has been developing for a few years. Interesting stuff.

Via CinemaTech.

Cool Real-Time Rube Goldberg Device

2007-03-23T08:40:00.000-04:00

A physics engine is a type of computer program used in video games and real-time animation to simulate different aspects of real-world physics like gravity, mass, velocity, and friction. A relatively new and fairly powerful physics engine called Bullet is being used in the latest version of Blender and to help promote its capabilities a Rube Goldberg Device contest was recently held with Blender users modeling elaborate machines using Bullet's Rigid Body physics.

You can see all of the entries here, but my favourite by far is the one above by Chris Plush, who's better known in Blender circles as "Blengine". When you watch this remember that the animation is generated in real-time. Don't believe me? If you have Blender on your computer you can download Chris' entry (link goes directly to a .blend file) and see it in real-time for yourself.

Now while this is real-time animation it's really more of a physics simulation than an example of digital puppetry. Although you can do some really basic things using Bullet's Ragdoll Physics, decent character control isn't yet possible with Bullet. It is on the developer's to-do list though and once it's implemented I think all sorts of new and interesting possibilities will open up.

Do-It-Yourself Cintiq Display

2007-03-21T21:45:00.000-04:00

I have to look in to building a dedicated digital puppetry workstation for work which has me thinking about all sorts of possibilities for controllers and input devices. One thing I would absolutely love to incorporate is a Cintiq. If you aren't familiar with these, Cintiqs are hybrid monitors/graphics tablets that allow you to "draw" on a screen and interact with a computer using a stylus. I think this is a much more elegant interface for a production environment than having puppeteers fiddle with a keyboard and mouse while they are putting together a performance. There's only one problem - money. The Cintiq costs about $2,500 US which is a difficult cost to justify for a somewhat unnecessary (albeit very nice) peripheral.

Luckily, the underlying technology in the Cintiq isn't new or terribly expensive, you just have to know how to put it together and of course some enterprising artist has already built their own Cintiq for roughly $150 US. The preceding link goes to their build log and you can watch a video of it in action above. It looks pretty nice!

This is slightly beyond my current level of electronics experience, but I just may try building one as an experiment if I can round up the necessary parts.

Further Blurring The Line Between Animation and Puppetry

2007-03-20T04:28:00.000-04:00

One of the interesting things I've discovered while discussing some of the concepts I am working on with Panda Puppet is that the idea of digital puppetry or utilizing puppetry techniques in animation seems to get a much warmer reception from most animators than motion capture, which has been the subject of fierce discussion and debate lately. Personally, I think the fuss over motion capture has been much ado about nothing and there are lots of ways that all these different techniques can co-exist and be used in tandem with another.

As an example, animator Michael Duffy read about Panda Puppet over at Keith Lango's blog on the weekend and pointed me to Timothy Albee's Facial Animation, a plug-in for Lightwave that allows animators to do lip sync in real-time and includes a somewhat limited 'puppeteering' mode. I don't know how useful it would be for real-time performances, but it's interesting to see another example of someone blurring the lines between puppetry and animation. You can read more about the software here. A free demo can be found at the link above along with some videos showing it in action.

First Look At Clone Wars

2007-03-16T07:37:00.000-04:00

A first look at Yoda, from the new Star Wars: Clone Wars television series.

This picture of Yoda was recently unveiled by StarWars.com. It's the first official image that has been released from Lucasfilm's upcoming animated Clone Wars TV series. The show reportedly makes extensive use of Zviz, ILM's new proprietary Machinima/real-time animation system.

Via Flooby Nooby.

Panda Puppet: Head Control Test #3

2007-03-14T23:10:00.000-04:00

Another Panda Puppet head control test; this time I wanted to experiment by mapping controls to a pre-existing character rigged for conventional animation so I used a fun looking mouse character created by a Blender user named "Clean3D" (the original mouse rig can be downloaded here). In the video I first demonstrate some very basic actions and then show how they can be mixed together to create different types of movements and expressions. I couldn't get as much control out of the rig as I would have liked, simply because the way it was build isn't ideal for Panda Puppet, but it still turned out pretty well.

Of course these real-time graphics leave something to be desired. I am still working on the programming that will record real-time performances to Blender animation, but here's what a final render of the real-time frame above would look like:

Not too bad. Thanks again to Clean3D for the great Mouse rig!

Controlling Video With Hand Gestures

2007-03-13T10:39:00.000-04:00

A company called Gesturetek has some very interesting looking technology that allows you to control video using hand gestures. It's essentially a more sophisticated version of Puppet Show, which I've mentioned here before. As you can imagine from this brief video, there are lots of potential puppetry applications for something like this.

What you could do in Machinima

2007-03-12T19:29:00.000-04:00

Brian Stokes, whose blog I always enjoy reading, linked to the fantastic French student animation film Burning Safari a little while ago alluded to the fact that it's difficult to do something like it in Machinima. While I agree completely that it would be nearly impossible to make Burning Safari with most existing Machinima and digital puppetry tools, after spending sometime looking at the films in terms of what I am working on with Panda Puppet I think this type of animated film could definitely be done using real-time techniques.

As a theoretical exercise I broke down the main sections of the film and I'll describe here how would approach them using digital puppetry techniques:

The Spaceship Landing - Easy enough to do, lots of video games already have vehicles like this in them. `Nuff said.

The Little Robots - These guys sure are cute. There are two possibilities for performing them that I can imagine; the simplest way would be to create walk cycles for them and then having a puppeteer simply control their X-Y movement within a scene. Another possibility would be to use a data glove on a flat surface like a table with each leg controlled by a different finger. This would allow more spontaneous and specific control, but running around might be difficult if you didn't have a large enough space to perform in.

"Robo-Vision" - To be perfectly honest, I would cheat this shot in After Effects.

The Monkey - Boy, I love monkeys. There's just something funny about them. In fact, he bares a striking resemblance to "Suzanne" the Blender monkey-head primitive. Controlling a character like this will be relatively straight forward with a control system like Panda Puppet.

See Monkey Get Mad - The only thing funnier than a monkey is an angry monkey. This type of moment in a scene seemed like a real challenge at first, but I have been playing with an idea I call "Emotion State Control" which would influence the movements and poses of a character according to the character's emotion state. If the emotion state "happy" is assigned to a joystick's trigger as long as the trigger is pressed all of the "happy" versions of different movements and poses would be used.

See Monkey Run - Once again, running is done using simple walk cycles. Leaping, jumping, etc. is fairly easily using a combination of walk-cycles and physics. Even the squash and stretch is fairly easy to achieve.

See Monkey On Fire - Ragdoll physics would work nicely in this situation. I would do the energy squares or whatever the robots are chucking at the monkey and the fire on the monkey's face in post using After Effects or something similar.

The other big problem that needs to be addressed is image quality. The graphics in real-time rendering are tied to a computer's ability to draw polygons onscreen. The more polygons a computer can draw and the faster it can draw them, the better the graphics. So having high resolution real-time graphics is really just a matter of having enough computational power. With enough of it, theoretically, you can render graphics of this quality in real-time. In fact I've seen real-time 3D demonstrations that exceed the quality of Burning Safari. Another work around is to simply do performances in real-time at low resolution and then render out higher quality resolution frame by frame afterwards.

There you have it, easy-peasy. Well, almost.

I don't mean to suggest that something being technically doable means it's easy artistically. Making great films is hard no matter how you do it. And even when I get Panda Puppet or someone else gets another digital puppetry system to the point where it can do all this all it will be is just a great tool. Having great tools is awfully nice, but what ultimately makes great art is a great artist.

Blender Gets Even Better

2007-03-11T07:53:00.000-04:00

Plans are afoot to add motion tracking capability to Blender. This is still in the earliest stages of development, but it looks like the idea is to add the ability to both match-move camera shots and track motion on a object that can be assigned to Blender controls. These were two huge features I really wanted for Panda Puppet (and dreaded the prospect of having to code on my own) and now it looks like they'll end up built right in to Blender.

How cool is that?

Panda Puppet: Head Control Test #2

2007-03-10T06:55:00.000-05:00

A second head control test, just some basic left/right movement, but now with an articulated jaw for extremely basic lip sync is possible. The second clip utilizes deformation to create a squash/stretch effect as the mouth opens. The third clip has two different keyposes triggered by pressing different joystick buttons. I also smoothed out the surface of the head for the second and third clips. It's all very crude still, but coming along nicely I think.

Work on this will continue through the weekend...

Nintendo - A Digital Puppetry Pioneer

2007-03-09T23:30:00.000-05:00

Most people don't know this, but Nintendo is something of a pioneer in the field of digital puppetry. Since 1994 they have been using a system they call MIRT ("Mario in Real-Time") to have many of their iconic characters like Mario appear at trade shows and in-store promotional events. The clip of MIRT in action above shows everyone's favourite Italian plumber bashing Sony, doing a Jack Nicholson impersonation and re-enacting a scene from Star Wars.

The earliest version of MIRT was a facial tracking system that ran on a Silicon Graphics super computer, but over the years the technology has been refined to the point where puppeteer Charles Martinet (the voice of Mario, Luigi and many of Nintendo's other characters) is able to work off a laptop at home and virtually perform characters all over the word. Charles briefly discusses his career as Mario & the gang in this short interview with GT.tv from last summer.

Panda Puppet: Head Control Test #1

2007-03-08T02:15:00.000-05:00

Success!

After spending the day working out several bugs, here is a quick test of the head control system I am programming for Panda Puppet. It's nothing fancy, just some basic head movement (up/down, right/left, tilt), but it works quite nicely. The head is controlled using a joystick, but almost any kind of device imaginable could be used as a controller. If you've read my last post on the control system, this is an example of direct control.

In the morning I'm going to tinker with this a bit to get some smoother animation and add some eyes and basic facial expressions so I can start playing with poses.

Controlling Digital Puppets

2007-03-07T03:39:00.000-05:00

Since a few people have emailed me asking about this, here is some more info on the control system I am developing for Panda Puppet and how it works, or at least how I am currently trying to make it work.

Panda Puppet is shaping up as a Python plug-in for Blender. At this stage I am not trying to make it do anything that isn't already possible in Blender's Game Engine, I just want to streamline the way characters can be set-up and controlled in real-time. The core of Blender's GE is Logic Bricks (sometimes called Logic Blocks) which are used to set up and control interactions between different game elements like characters, props, etc.

This is what Blender's Game Logic Panel looks like:

Once again I won't go in to all the technical nitty-gritty of Logic Bricks here, but if you want to learn more just click on the link above.

Panda Puppet's control system is a simplified interface for quickly setting up Logic Bricks in way that's ideal for controlling digital characters. I am still in the early stages of programming this so all of it may change, but this is how I am designing the control system right now:

Click on the diagram to enlarge it

Sensor Type

The sensor type is the type of device used by the puppeteer to control on an on screen character or object. This can be almost any kind of device imaginable, including a joystick, keyboard, gamepad, dataglove or even a Wiimote. I want Panda Puppet to be as flexible and "controller agnostic" as possible. Rather than forcing a puppeteer to adapt to a specific type of input device, I want a control system can be customized to the needs and preferences of a puppeteer and I want different puppeteers performing in a scene together to use different types of controls if they want.

Sensor Input

Sensor inputs are specific types of inputs from an input device. For example, the press of a button on a keyboard or the x-y axis movement of a joystick.

Sensor Action

Sensor Actions are control types that govern a character's action and determine what happens when a puppeteer uses a specific sensor input. There are three different types of Sensor Actions - direct controls, pose controls and emotion controls.

Direct Control
Using direct control a puppeteer has direct control over the movement of a specific part of a digital character. For example, The x-y "walking" movement of a character can be assigned to the x-y axis movement of a joystick so that when the joystick moves in a specific direction the character walks in a corresponding direction.

Pose Control
Pose Control is used to assign specific poses or movements to a specific sensor input. For example, an elaborate shriek or a comedic double-take could be triggered by a joystick button.

Emotion State Control
Emotion State Control influences the movements and poses of a character according to the character's emotion state. An emotion state can be assigned to a specific sensor input and triggered when that control is activated and will influence the character as long as the input control remains active. If the emotion state "happy" is assigned to a joystick's trigger as long as the trigger is pressed the character will remain "happy".

Typically, there are six primary emotional states - anger, disgust, fear, joy, sadness and surprise. Just as primary colours can be modified and mixed to create every other imaginable colour, primary emotional states can also be mixed to create every emotional state imaginable. For example, Anger + Disgust = Rage. Emotion state combinations can be triggered by activating a combination of sensor inputs (when primary emotion states are assigned to individual sensor inputs) or mixed emotion states can be grouped and triggered by a single sensor input.

If anyone out there has thoughts, ideas and/or suggestions about all of this please feel to drop me a line at puppetvision {at} gmail dot com.

Rigging a Monkey for Panda Puppet

2007-03-06T15:03:00.000-05:00

One of the features that Blender's Ketsji game engine is noticeably lacking is the ability to use shape-keys. In case you're not familiar with them, shape-keys (sometimes called "blend shapes" or "morph targets" in other 3D software packages) allows you to save and store different shapes of your character. In animation this is very useful for doing things like lip sync and creating facial expression.

Working on Panda Puppet I was stuck for quite awhile trying to figure out a work around for this problem until I stumbled across an article on bone based facial animation. I won't bore you with all the details (if you really want them, click the link) but essentially the idea is to stick a bunch of bones in a character's head and have each one control a different area of the character's face.

Here's how I am currently rigging using the default monkey head in Blender:

The bones work as follows:

B/C - Ears
D/E - Brow
F - Nose
G - Snout
H - Jaw

Here is a side view, in which the "A" bone (the primary head bone) is visible.

I am working with a low-poly head, made up of about 500 polygons. The poly count could be higher (more polys = smoother shape), but I am keeping it simple for now. Hopefully this "bone-based approach" will work. I need a more or less functional monkey head by March 15th to meet the deadline I am working with. The next week and a half is going to be very, very busy!

Wii Power Glove Controller

2007-03-02T22:20:00.000-05:00

A little over year ago I was speculating about using the old NES Power Glove to perform digital puppetry. I haven't heard of anyone doing that just yet, but someone in Japan has hacked together a custom controller using the Wii remote and an old NES Powerglove. The modded glove has specially mounted triggers on the thumb and middle finger to simulate the A and B buttons and an opening for the wii attachment in the back. It seems to work really well, you can see it in action in the clip above. More pics and details are available at Slash Gear.

The obvious digital puppetry application for this would be to make Machinima on the Wii and since so many people have already hacked the Wii remote to use it on their PCs, with a little technical know-how you could really start to have some fun with this thing.

What you can't do in Machinima (for now)

2007-02-13T20:33:00.000-05:00

This animation test above is a fantastic example of pose-to-pose animation (even if it is a "test") from Pixar. As I've mentioned many times here before, one of the inherent limitations of Machinima as it exists today is that the available authoring tools are really unintuitive and don't allow for much expression. Even the best examples of Machinima puppeteering like the ILL Clan's Trash Talk With ILL Will - which won a Mackie for best virtual performance at 2006 Machinima Awards - are ridiculously crude compared to what most puppeteers and animators can do.

I also like this fantastic clip from Aardman. It's done with Claymation of course, but it's so simple there is no reason something very similar couldn't be done in real-time. There is absolutely no reason high-quality animation work can't be done in real-time. The only problems is that a) you need fast enough hardware to render enough polygons on screen and b) there aren't simplified tools that allow you to create great performances.

All it takes is a little money to solve the first problem and, well, I'm working on the second.

Two and a Half Virtual Men

2007-02-09T16:14:00.000-05:00

Many viewers in the U.S. who watched the Super Bowl pre-game show last weekend got their first look in to the virtual world of Second Life thanks to this short promo for the CBS sitcom Two and a Half Men that was created using SL. The big-time exposure is something of a landmark achievement for Machinima, although unfortunately this is a pretty weak piece, no doubt due in part to the fact that it was reportedly produced for CBS in big a rush.

The Two and a Half Men promo was created by The Electric Sheep Company, which usually does better work than this. Visit their Machinima portfolio for some much nicer Machinima examples.

Where's Waldo?

2007-02-08T22:52:00.000-05:00

I'm putting together an educational presentation about digital puppetry and Machinima and need a little help. I'm looking for a high quality picture of of Waldo C. Graphic. Any chance any Machin-X readers have a nice, high quality picture of him? Say something 640x480 pixels or bigger maybe? I've done all the usual Google searching and all I've turned up is this one on the right, which is a little too small.

If you've got a pic that you can share please email me at puppetvision [at] gmail dot com. Thanks!

Panda Puppet Progress

2007-02-08T09:40:00.000-05:00

"Sales Guy" is the talking head being used to program Panda Puppet right now.

Just a quick update about ongoing developments with Panda Puppet. The image above is "Sales Guy" the nickname that's been given to the head being used to map and program the facial controls for Panda Puppet. I was hoping to have some video of Sales Guy in action to post here by now, but Sales Guy kind of, uh, broke earlier this week. I'm still trying to get to the bottom of the problem, but I hope to have it resolved before the end of the day today.

What's always interesting about a project like this is that things that you expect to be difficult turn out to be relatively simple while the little things that you expect to be easy end up being very, very hard.

Last week I came to the conclusion that Blender's internal game engine is not really up to snuff. The 3d engine and can't meet all of Panda Puppet's needs (there are plans to overhaul Blender's GE using OGRE, but there's no confirmed time line on when that will happen). I've been exploring using Crystal Space as Panda Puppet's 3d engine via a nifty Blender plug-in called Blender2Crystal. Unfortunately, the plug-in doesn't seem to be widely used and there is limited documentation available so things are slow going on that front.

The other thing I have been looking in to is the Sunflow renderer, which looks promising. Although Panda Puppet will work in real-time, one it's key features is that performances can be tweaked, edited out and rendered out as conventional animation. Sunflow isn't natively supported by Blender right now, but there does seem to be some interest in getting more support for it from the Blender community.

I have learned more about programming and 3D graphics in past three weeks than I have in the previous ten years.

Back with more soon!

Making Love Not Warcraft

2007-01-29T09:39:00.000-05:00

South Park's Stan, Kyle, Cartman, and Kenny as World of Warcraft characters in the South Park episode Make Love Not Warcraft.

One of the biggest things to happen to Machinima last year was the use of it to create Make Love Not Warcraft, an episode of South Park that parodied the popular massive multiplayer online role-playing game (MMORPG) World of Warcraft and gamers in general. Machinima.com has an interview with South Park's Frank Agnone, J.J. Franzen, and Eric Stough explaining how Machinima was used to create in-game sequences in the episode.

One of the things that impressed me about Make Love Not Warcraft was the relatively high degree of lip sync in the Machinima sequences, which I didn't realize could be done in a game like Warcraft. Turns out that it can't; Warcraft's publisher Blizzard helped the show cheat a little bit, providing South Park's animators with Maya models of the characters to animate conventionally for close-ups. Even if it's not technically 100% "true" Machinima though, Make Love Not Warcraft is a great example of how Machinima and real-time animation can be incorporated in to a professional production pipeline.

Bigger (and more expensive) isn't better

2007-01-27T00:54:00.000-05:00

While discussing my work thus far on the Panda Puppet system earlier this week, a colleague who I respect and love dearly scoffed at the notion that you could build a decent, high-end digital puppetry system using a free open source program that has a relatively tiny 7 mb installer.

"You need Motionbuilder," he told me. "You need Maya. Don't waste your time on this open source stuff." So we had to agree to disagree, but to underscore my position that bigger (and more expensive) isn't better I direct you to the story of the spoon and the jackhammer.

Note that the point of the story isn't that the spoon is the easier tool to use, just that it's ultimately the better one.

Disruptive Animation Technology

2007-01-24T00:39:00.000-05:00

The Oscar nominations were announced today and the fact that two of the three nominees for Best Animated Film relied heavily on motion capture has not escaped the attention of many blogs on the internet. Some animators think this is the end of an era.

This isn't really anything new. Anytime a new technique or technology comes along and disrupts an existing art form the old guard get bent out of shape about it for a decade or two until things settle down and a new equilibrium is esthablished. In the 1990s puppeteers griped about the popularity of the then-new computer graphics in special effects work. Traditional animators griped about rise in popularity of 3D animation. Now computer animators are complaining about the rise of motion capture.

This seems silly to me. Photography wasn't the end of painting, electronic devices won't replace books anytime soon and despite the popularity of computer animation, old fashioned stop-motion animation is undergoing something of a renaissance. The tools and process of animation may change, but it will never go away any more than puppetry will.

I have no doubt that when (and I believe it's a matter of when, not if) real-time animation/digital puppetry/Machinima/whatever you want to call it really takes off there will be a whole new round of moaning and groaning about how it will be the end of this artform or that technique, which of course is rubbish. There is always opportunity for smart and skilled artists in these types of transitions. For example, any decent real-time animation system is virtually impossible to put together without good traditional animation skills. Ditto for motion capture, which looks awful in it's raw form.

BTW, work on Panda Puppet this week is coming a long very nicely and much quicker than I expected. I hope to have some demos or screen shots up sometime next week.

Panda Puppet: Day One

2007-01-22T08:17:00.000-05:00

This morning I am sitting down at the computer for my first official day working at Panda Puppet. If you've followed Machin-X for a long time you'll know that Panda Puppet has been in the works for awhile. It grew out of some work I did about a year and a half ago developing a system called Flash Puppet that was used to produce a demo for a Flash Animated children's series in the U.S. When the system is finished, Panda Puppet will be a "studio in a box" that will allow you to produce real-time animation via a puppetry interface. It's being built using Blender, the leading open source 3d modelling and animation software package.

Various work has been going with Panda Puppet part-time off and on for about a year, a lot of it focused on experimenting with different ways to allow puppeteers to control characters in a digital 3D environment. I wasn't expecting to start full-time work on Panda Puppet until sometime in the summer, but as luck would have some basic real-time animation would be very useful for an upcoming project so getting the system up and running is now a priority. The specs for everything are pretty much in place and I think I've got the hardware issues I mentioned last year licked. I'm committed to delivering a proof-of-concept in about eight weeks so I'm going to be very, very busy writing and testing computer code for the next two months.

I'll be blogging my progress on Panda Puppet and explaining more about the system and how it works in the coming days and weeks. Assuming everything gets working properly this is going to be really, really nice to play with.

How to Puppeteer a Head in Machinima

2007-01-20T17:48:00.000-05:00

Lip sync and effective control of a characters' head on screen has been a difficult challenge for Machinma creators. Unless you count the high-end real-time animation being done by Disney and Henson, I haven't seen any Machinima with effective animation of a character's head. There have been some noble attempts, but all of them look pretty stiff and boring to me.

An interesting solution to this problem that I have been toying with for awhile is to have the computer visually track the movement of a puppeteer's hand, much like the way that the Nintendo Wii tracks the movements of players holding a Wii remote. Michael Nitsche, an Assistant Professor at Georgia Tech recently emailed me about PuppetShow, a very cool looking system some of his students have put together to control the heads of digital puppets with the Unreal Game Engine.

As you can see in the video, PuppetShow tracks the coloured object and the digital puppet moves on screen accordingly. This isn't new technology of course, but what I think is really cool about it is how accessible it is...all you need is a coloured piece of paper, a USB webcam and some free software. It's a limited system right now, but I like the way it puts head movement back (literally) in the hand of the puppeteer.

Basic lip sync can be achieved in a system like this through the movement of the puppeteer's thumb, but the puppet's mouth can't enunciate and the results are still pretty stiff. Many Machinima creators have suggested using some kind of voice analysis or lip sync software like Magpie as a way to create lip sync and enunciation, but I like the idea of a puppeteer having direct control over all aspects of the puppet's head.

In conventional puppetry puppeteers do not actually articulate each word that a character says. The puppet's mouth usually only opens once for each syllable that the character speaks; the mouth is open on vowel sounds (a-e-i-o-u) and closed on consonants. Therefore to achieve realistic lip sync a puppet doesn't have to be able to enunciate perfectly, it just has to be rigged to be able to open and close it's mouth and make these four different shapes:

Each of these mouth shapes could be pre-established as a keypose on a digital puppet and triggered by the puppeteer pressing a button (possibly located inside some kind of data glove or control mitt) or making a specific gesture.

By combining this approach to lip sync with the visual tracking of PuppetShow, Machinima creators could really, finally have a great system for puppeteering their characters' heads in real time.

More on Jeff Han's Multi-Touch Interface

2007-01-20T03:17:00.000-05:00

Following up on my last post about Jeff Han's incredible multi-touch computer interface, Fast Company Magazine has an article about Jeff in their latest issue. You can read it online at fastcompany.com.

Updated: Futuristic Multi-Touch Interfaces

2007-01-16T03:51:00.000-05:00

This is old (from last summer), but extremely cool. A prototype multi-touch user interface demonstrated by Jeff Han. At about eight minutes in he demonstrates some potential puppetry applications for it. Think of the possibilities!

Come to think of it, the new iPhone will have a multi-touch interface too. Hmmm...

Henson debuts The Skrumps

2007-01-10T19:53:00.000-05:00

The Skrumps is a new digital puppetry series from The Jim Henson Company.

The Jim Henson Company is developing a new animated property called The Skrumps, based on a line of colourful, monster-like toy characters created by artist John Chandler. The series is being produced using the Henson Digital Performance System and looks very promising. John Chandler, Skrumps puppeteer Victor Yerrid and Henson's head of Children's Programming Halle Stanford discuss The Skrumps and their development in the latest episode of the Henson.com podcast.

The Skrumps debuted today in four new videos available at Yahoo! Kids for a limited time. Be sure to take a look!

Via The Muppet Newsflash.

--

13/01/07 Update: A behind-the-scenes video about the making of The Skrumps is now available on Henson.com. It's fairly short, but gives a basic overview of how their HDPS works and shows the performers in action.

A Merry Machinima Christmas

2006-12-20T16:13:00.000-05:00

Visiting Scrooge puts a Machinima spin on a Christmas classic.

In the mood for a little holiday Machinima? Take a look at Visiting Scrooge, a two-part film based on Dickens' A Christmas Carol and made using The Sims 2. A behind-the-scenes page about the film is also available.

Acting and "virtual puppeteering"

2006-12-13T15:08:00.000-05:00

Bill Nighy's motion capture performance was used to create Davy Jones in Pirates of the Caribbean: Dead Man's Chest.

Earlier this week Variety ran a good article about the growing use of motion capture in feature films and how actors are increasingly been utilized as virtual puppeteers to create fantastic creatures and effects. Most of the article deals with Bill Nighy's impressive mo-cap performance in last summer's Pirates of the Caribbean: Dead Man's Chest.

Inside the Machinima

2006-12-08T00:00:00.000-05:00

A Machinima film that explores how Machinima gets made. Very meta. From Machinimated Studios.

Don't mind the mess

2006-11-29T02:31:00.000-05:00

I'm in the process of converting Machin-X to a new Blogger Beta template so the site may look messy for the next several days. Also, embedded YouTube videos don't seem to work anymore. Does anyone know what's up with that?

Turn a Mac in to a marionette

2006-11-23T15:34:00.000-05:00

Digital puppetry usually involves using computers to create puppetry, but this may be the first example of actually turning a computer in to a puppet...the newer Mac Powerbooks are equipped with a Sudden Motion Sensor that detects if the computer has been dropped or is flying through the air. The purpose of the sensor is to minimize damage from impacts by locking down the Mac's hard drive if it's suddenly in motion, but since the sensor can be accessed via software with a little programming knowledge you can hack it and explore all kinds of interesting possibilities.

Andreas Nicolas Fischer in Germany used this feature to turn his Powerbook in to a marionette for an art project. When the computer is swung a animated character on screen reacts with realistic physics to the movement. A build summary for the project is available on his web site along with some of the source code used for the project.

Thanks to Mike for the info!