Fellow PhD student at the TU Delft, Miguel Bruns-Alonso created a nice video of his Music Cube (his graduation project, see paper). And then Jasper van Kuijk (another colleague) blogged it for usability. And here I come wandering wondering: whether moving this Cube in certain ways to control music playing can or should be considered gesturing.


Perhaps this is a highly irrelevant question. I am pretty sure Miguel could barely care less. But that’s me, always worrying about silly gesture stuff.

In a way the question is similar to a previously unanswered question Is Sketching Gesturing?

Like with sketching it is not the movement itself that matters. Rather, it is the effect that the movement causes that is important. Although the case of “shuffling” may be an exception because the “shaking” movement is fairly directly registered. Other commands are given by changing the side of the Cube that is up (playlists), or by pressing buttons (next, turn off), or turning the speaker-that-is-not-a-speaker (volume). These are fairly traditional ‘controlling’ movements, comparable to adjusting the volume or radiofrequency with a turn-knob (as in old radios).

I will leave aside the question whether such tangibility constitutes a more valuable or enjoyable interaction with our machines. Some believe that it does and who am I to disagree. Like it or not, take it or leave it, you choose for yourself.

What concerns me is whether such developments and other gesture recognition developments share certain characteristics. If so, then exchanging ideas between the areas may be a good idea. One of my bits of work is on discriminating fidgeting and gestures.

The question rises whether the Music Cube will allow people to pick it up and fidget with it without immediately launching commands. Can I just handle it without ‘touching the controls’? Like with other gesture recognition applications I want this Cube to allow my fidgeting. In that case rules for human behaviour regarding the difference between behaviour that is intended to communicate (or control) and behaviour that is just fidgeting would be useful. And why don’t we carry the thought experiment of the Music Cube further? If it has motion sensing, it should be able to do the sort of things that the Nintendo Wii can too. Why not trace gestures in the air to conjure up commands of all sorts? How about bowling with the Cube? Or better yet, playing a game of dice?