Various enterprises and personal interests, such as Man-Machine Interaction (MMI), gesture studies, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Category: Gesture with Emotion and Feeling

GW2009 Keynote: Antonio Camurri

Keynote: Antonio Camurri (also here)
Toward computational models of empathy and emotional entrainment

Casa PaganiniInfoMusEyesWeb

Camurri has already done a lot of interesting work on movement and gesture, all of it in the ‘expressive corner’, working with dance and with music.

He just talked about a really nice application: He created a system to paint with you body movements, But it does so only if you move without hesitation. So, patients with hesitant movements (Parkinson?) get a stimulus to move better.

Next, about part of Humaine: something about the visibility of emotion in musical movements (not the sounds). There were previous talks in this area:

Florian Grond, Thomas Hermann, Vincent Verfaille & Marcelo Wanderley:
Methods for effective ancillary gesture sonification of clarinetists

Rolf Inge Godøy, Alexander Refsum Jensnius & Kristian Nymoen:
Chunking by coarticulation in music-related gestures

Next work with Gina Castellana (?): influence the way you listen to music through movement and gesture. Nice video.

There is also work on robotic interfaces. A ‘concert from trombone and robot’. Stockhausen, Milano. Robot had a radio, drove around, so spatially and in playing the robot had to be in tune with the trombone player. Collaboration with S. Hashimoto and K. Suzuki (Waseda University), See here for a publication.

He also worked together with Klaus Scherer from Geneva. Gael talked about Scherer’s work on the emotions as being quite good.

Camurri seems to be involved in many European networks and projects.

He is now explaining a project on synchronization. Quite interesting stuff about violin players (as cases of oscillators) try to get synchronized with a manipulated signal or with each other. It is going too fast to write much about it, but it all looks really nice. Violinists synchronizing their movements. And he is making much of a concept called ’emotional entrainment’. There is decent explanantion of the term here, but I’ll quote it:

A Quote by Daniel Goleman on emotional entrainment, influence, charisma, and power
Setting the emotional tone of an interaction is, in a sense, a sign of dominance at a deep and intimate level: it means driving the emotional state of the other person. This power to determine emotion is akin to what is called in biology a zeitgeher (literally, “time grabber”), process (such as the day-night cycle of the monthly phases of the moon) that entrains biological rhythms. For a couple dancing, the music is a bodily zeitgeber. When it comes to personal encounters, the person who has the more forceful expressivity – or the most power – is typically the one whose emotions entrain the other. Dominant partners talk more, while the subordinate partner watches the others face more – a setup for the transmissions effect. By the same token, the forcefulness of a good speaker – a politician or an evangelist, say – works to entrain the emotions of the audience. That is what we mean by, “He had them in the palm of his hand.” Emotional entrainment is the heart of influence.
Daniel Goleman : Harvard PhD, author, behavioral science journalist for The New York Times
Daniel Goleman
Source: Emotional Intelligence: Why It Can Matter More Than IQ, Page: 117

Interesting remark about the violinists who synchronize with an adjusted signal: They did not hear their own sound but rather a manipulation of the pitch of the movement. So what they did did not match what they heard. At some point these players got motion sickness…

Now there is a weird video from the opera, where a man and a woman use a chair to communicate (?). He lost me there for a moment.

Announcement: eNTERFACE 2009, European Workshop on Multimodal Interfaces, 13 July – 7 Aug, Casa Paganini (here)

– About publications: you can download them from

Gesture and Emotion

Let us broaden our horizon.
Let me turn your gesture perspective to new topics (well, revisited actually, see here).
Let us ponder emotion.

Pieter Desmet created the PrEmo method and interface (source)

Several colleagues here are studying ‘design and emotion’. One of the methods developed to evaluate people’s emotional response to product is PrEmo. It shows a little guy in different pictures that represent different emotions. The trick is, however, that they are not pictures but animations with sounds. So, the little guy makes facial expressions, gestures, and some exclamations. Of course, the question immediately comes up how reliably these gestures represent the intended emotions. Does everyone see it the same way? Apparently, Gael showed me some results, people do see it the same way for most of the pictures. Yet, some of them, such as surprise, are not see reliably perceived.

Sadly, the animations, which are in flash, are not available publicly. I understood there is licensing involved, and you have to see them to be able to really evaluate the gestures.

If you want to read more about emotion and how it can be measured: The Design & Emotion Society is quite a useful resource. You can register as a member for free and then they provide a good knowledge base. Another site is the HUMAINE Portal. With them you have to pay a small amount.

Paul Ekman on Recognition of Emotions

In a long interview in Dutch popular science journal Psychologie of today, Paul Ekman ends with the following statement: (source)

“Often people think I am a magician who can read minds. But I really can not do that. Just by paying attention really well, by being very secure, and by putting all the pieces of the puzzle [facial expression, head and eye movements, gestures, context, and story consistency] together I can make a statement about my strong impression about someone’s real motives. That’s all.”

I think that is a very fair and true statement from the founder discoverer of Mankind’s universal facial expression of emotions. There was also a test. I managed to score 7 out of 7 from pictures of a girl enacting basic emotions. And still I do not consider myself a good judge of other people’s emotional state.

As I am typing this, I wonder if my wife is cross with me over something I neglected to do… …She does seem to ignore me… But that might be because she is tired… or maybe she is simply concentrating on her movie… I tried to read her frowning face, But it’s simply too familiar I failed to catch her feelings there, perhaps she’ll fan them to me later?

Powered by WordPress & Theme by Anders Norén