Various enterprises and personal interests, such as Man-Machine Interaction (MMI), gesture studies, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Category: Motion Sensing

Avatar Kinect

Introduction to Avatar Kinect by Microsoft.

Avatar Kinect is a new social entertainment experience on Xbox LIVE bringing your avatar to life! Control your avatar’s movements and expressions with the power of Avatar Kinect. When you smile, frown, nod, and speak, your avatar will do the same.

Ah, new developments on the Kinect front, the premier platform for Vision based human action recognition if we were to judge by frequency of geeky news stories. For a while we have been seeing various gesture recognition ‘hacks’ (such as here). In a way, you could call all interaction people have with their Xbox games using a Kinect gesture recognition. After all, they communicate their intentions to the machine through their actions.

What is new about Avatar Kinect? Well, the technology appears to pay specific attention to facial movements, and possibly to specific facial gestures such as raising your eye brows, smiling, etc. The subsequent display of your facial movements on the face of your avatar is also a new kind of application for Kinect.

The Tech Behind Avatar Kinect

So, to what extent can smiles, frowns, nods and such expressions be recognized by a system like Kinect? Well, judging from the demo movies, the movements appear to have to be quite big, even exaggerated, to be handled correctly. The speakers all use exaggerated expressions, in my opinion. This limitation of the technology would certainly not be surprising because typical facial expressions consist of small (combinations of) movements. With the current state of the art in tracking and learning to recognize gestures making the right distinctions while ignoring unimportant variation is still a big challenge in any kind of gesture recognition. For facial gestures this is probably especially true given the subtlety of the movements.

A playlist with Avatar Kinect videos.

So, what is to be expected of Avatar Kinect. Well, first of all, a lot of exaggerating demonstrators, who make a point of gesturing big and smiling big. Second, the introduction of Second Life styled gesture routines for the avatar, just to spice up your avatars behaviour (compare here and here). That would be logical. I think there is already a few in the demo movies, like the guy waving the giant hand in a cheer and doing a little dance.

Will this be a winning new feature of the Kinect? I am inclined to think it will not be, but perhaps this stuff can be combined with social media features into some new hype. Who knows nowadays?

In any case it is nice to see the Kinect giving a new impulse to gesture and face recognition, simply by showcasing what can already be done and by doing it in a good way.

Air Guitar Toy by Mannak

Ronald Mannak, a former colleague, is now developing toys at his own 1uptoys. His toys at hand are the SilverLit V-Beat AirDrums, AirGuitar and BoomBox. This week our university’s ‘newspaper’ has an interview with him: Luchtgitaar met Geluid. And here he is in a video demonstrating his AirGuitar:

Still a long way to go before he can try for world champion airguitar, I think. But the product is interesting to consider. At first I thought it looked quite nice and cool. But then I wondered: why would anyone want to actually have an AirGuitar? Isn’t the point of playing air guitar that you don’t have to have the damn thing? If I am going to buy something to play guitar I might as well, or even better, buy a real (toy) guitar, right?

Is this going to be cheaper than a real guitar? I would guess that the additional electronics will not be cheaper than the bits of extra wood, metal or plastic needed for a physical guitar. But then again, microelectronics can be cheap if they are sold in large quantities.

So, is this going to provide a better experience? I think that by definition that is impossible. The point of playing air guitar is to imitate the actual playing, to go thorugh the motions and almost ‘feel like’ you are really playing. In other words, it can never be better than the real thing, or can it?

Maybe it can. Maybe it can help people who can not play guitar ‘feel more like’ they are playing guitar. Maybe the AirGuitar can take care of the difficult stuff like putting your fingers in the right position on the strings and remembering the chords and licks, and leave the exciting stuff to you, like strumming wildly, creating vibrato or smashing it.

That would be neat, Ronald if you read this, can you make it so it can be smashed?

Space Invaders with Gesture Recognition

The more I think about it, the more I am convinced that the (near) future of gesture recognition lies in entertainment. Here is yet another gaming application with gestures: A multi-player, wall display remake of Space Invaders. A highly advanced gesture interface seems to allow any kind of movement at a certain spot on the baseline as ‘fire from here’. A camera tracks if a hand blots out one or several of a series of lights that represent the positions on the baseline.

YouTube: Development: Douglas Edric Stanley / This is installation was developed on-site for the Gameworld exhibition at the Laboral Art Center, Gijón, Spain. March 30 – June 30, 2007. For more information visit the responsible art centre in Spain.

Wii Mainstreams Gesture Recognition

Play sports games virtually but with the real movements


The Nintendo Wii controller is starting to hit the big spotlights. Interaction designers, like Matt MacQueen, are noticing the power they can bring to gaming experience. He has written a nice piece reflecting on Wii experiences sofar and projecting trends for the future.

See the huge line for the Nintendo Wii demonstrations at the E3 2006 conference

Update: here is a nice Dutch review of Wii gaming experience.

Tags: , , ,

Exergames blogs about physical gaming

Exergames is a nice-looking blog on exercise and physical gaming. They do not update much though. I guess they are too busy playing with their new Nintendo Wii‘s. Or perhaps they were seriously injured from the intense physical interaction, preventing them from typing? Wanna work out with a game? (source) By the way: Wouter has a nice post on another quirk of Nintendo Wii: The controls of enthusiastic gamers are flying out of their sweaty hands and breaking their expensive TVs or windows. Every rose has its thorn.

Powered by WordPress & Theme by Anders Norén