Various enterprises and personal interests, such as Man-Machine Interaction (MMI), gesture studies, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Tag: Theremin

Imaginary Gestures for Music Therapy?

A while ago I received an inspiring email with an interesting question, see below (and btw, I would encourage anyone reading my blog to start corresponding, which I greatly enjoy). I will try to respond here, sharing my thoughts not only with Paulo Suzuki, but with anyone interested.

Hi Jeroen,

I’m an IT professional and Music Therapist.

I’m working on a research using the Theremin as a music therapeutic tool and
I’d like that you recommend a book (basic or introdutory) regarding
“gestures” so that I can use to justify my issues.

My research is oriented to music therapy patients with little or without
movements – and so I’d like to know if you are familiar with any research on
“imaginary gestures”?

Thanks for the time.


Paulo Suzuki

PS: my 1st resource regarding “gesture” is your blog “Nice Gesture”; thanks
and congratulations.

Before I give you some answers, I would like to point to a prior post about Musical Gesturing with a Theremin and other posts in the Musical Gestures category. Looking back on that post, I would like to add that I believe that the feelings that music arouse in us are as real as we allow them to be, which is ‘very real’ for many people. Undoubtedly, this can be put to many positive uses, such as ‘Musical Therapy’. However, if you are in this business then I think it is important to remember that you are working for a large part with created conventions that need to be learned and accepted by your ‘patients’. Granted, there may be some feelings or ‘gestures’ that certain music has, in and of itself. For example, ‘beats per minute’ may be related to a slower or faster beating of the heart, and therefore to our arousal or activation. However, trusting too much in the emotional values of certain music quickly becomes speculative, highly subjective, and therefore not advisable (unless you either can rely on or have the time to build up a library of shared conventions with your audience, or patients).

So, to answer the first question, I would like to recommend two books:

  • Gesture, Visible Action as Utterance, by Adam Kendon (the basics about gesture and how humans use gestures).
  • Music and Gesture, by Gritten & King (Eds.) (I haven’t read this one myself, but from the contents it looks like it covers the topics you might be interested in).

To answer your second question, regarding research on ‘imaginary gestures’:

  • There is the work on ‘mirror neurons‘ that shows that there is a very close connection in our brains to ‘seeing a gesture’ and ‘producing a gesture’. That might be useful to you? A wild thought: if one wants patients to be able to play music with imaginary gestures perhaps it is possible to record their brains while watching a musician play a certain bit of music, identify the ‘brain activation’, and then watch for that when they ‘play in their minds’ themselves, translating their ‘imaginary gestures’ back to music. I’m taking a couple of shortcuts with this line of reasoning, but well.
  • Then there is the work on helping people who lost a hand or an arm. See for example this story: Robot arm ‘controlled with thoughts’. To some extent the nervous system may still process stimuli that pertain to certain hand actions, or gestures.
  • Another interesting bit of work was done a while ago by Sidney Fels: GloveTalk, a system that translates hand gestures to speech through an adaptive interface. Perhaps the micro gestures Fels uses, can also be translated to music instead of speech. Incidentally, his first ‘user’ was reported to be an experienced piano player…

Hope it helps 🙂

Musical Gesturing with a Theremin

Here is a man called Eliot Fintushel, who neatly exemplifies how people come to talk about the expression of feelings in music. Furthermore, because he is using ‘gestures’ to operate his Theremin he even talks about his feelings being present in his movement and therefore (magically) transferred to the music of the Theremin. So, if you make a ‘nervous’ movement, you get a ‘nervous’ sound.

To my ears, it all sounds like a big circular argument: If you tell people that a certain movement is ‘nervous’ and then that the sound they hear is also ‘nervous’ then what will they see and hear? That’s right, ‘nervousness’. But what if you had not told the audience what to hear or see? They would simply rely on the facial expressions (which are overacted and conventional rather than universally understandable in this case) to know the feelings they should be feeling. Projection will do the rest.

But like I said before, relying on conventions (even if they are shared with only a select few) to achieve some effect is not necessarily a problem. If a concert audience wishes to share a fantasy of going through all sorts of feelings, guided by a performer, then let that fantasy be as real as possible. Some of that feeling may actually be genuine, the rest may serve our emo-thirsty souls just as well.

Powered by WordPress & Theme by Anders Norén