Month: June 2006 Page 1 of 2

Back in 1980, Richard A. Bolt from MIT wrote Put-that-there:voice and gesture at the graphics interface. It was a pioneering multimodal application that combined speech and gesture recognition. I graduated on a pen and speech interface in 1998. At that time I think everyone in the field of multimodality paid tribute to Bolts work.

Source: original paper (pdf)

The MIT Media Lab, which Bolt helped to found, is a place where the memory of Put-That-There still resides. He doesn’t appear to work there anymore, though his site is kept there. Bolt has written down much of his thoughts on HCI and multimodality here At the Interface… It is not very accessible web info, but if you take the time to read it you will be rewarded with plenty of insights.

So, does anyone know what happened to.. “Put-That-There”? Does it still exist? Is the memory kept alive?

America. What can I say of the gestures of the United States of America? Often cross-cultural web info compares culture X to America, like here, here or here on gestures. Or unless stated otherwise the context of given information is the USA like in the nonverbal dictionary. So, are we to believe that Texas is the same as Maine in this respect? Do Latinos gesture like Amish? Do Jews gesture like Italians? According to David Efron, Gesture, Race and Culture, they start to do so if you put them in a meltingpot long enough.

The American Indians used sign languages, which have been documented quite a bit.

Modern Plains Indians Sign?

I found it rather surprising then that if you are going to study in America, you are advised on how to point and smile. I do agree that people exaggerate the importance of the OK sign. But lets wrap it up. America gestures and signs as vibrantly as elsewhere in the world. Immigration played its role, and probably still does. Good night, America.

A great classical text is now online:

Quintilian: Institutio Oratoria

It is a massive work of twelve books used to educate boys to become orator. Part of becoming a good speaker is mastering the delivery of your message, through appropriate use of voice and gesture. This is the topic of section 3, Book XI. (lines 1-2, 14-15, 61-149 especially).

Quintillian came from Spain but taught in Rome in the first century AD. His very influential work builds on that of Cicero, who discussed gesture in oratory as something to be cultivated, and before that Aristotle, who discouraged gesture and other theatrical means as mainly working on the emotions of the public, favoring pure reason instead. Quintillian separates what is appropriate for orators and for actors though. For orators he gives some very specific guidelines on gestures, how to make them, and when to make them.

So, do you want to be seen as a captivating presenter, a convincing lawyer, or a trustworthy politician? You will have plenty to learn from Quintillian. Somehow, I don’t think the instructors of our days such as Lenny Laskowski, these guys, this guy, or any of these for that matter have much better to say.

Of course they fascinate me, those classic cross-cultural gesture mix-up stories, like OK versus Asshole and many, many others.

I will pay you for a good video of a gesture that was misunderstood

In truth, the stories are always about someone doing something that could have been mistaken for something else. Or someone makes a gesture and is then told his gesture or behaviour is rude or even obscene in the culture he is visiting. That in itself says it all. The gesture was seen and recognized as not intentionally rude or obscene and the producer was informed about the mismatch. We guess the intentions of other people from the situation, the location, their signs of communication (be it speech, gesture or otherwise). So why did the gentle culture teacher point out the mismatch? Well, someone else could have taken offense! Of course, we are not stupid but we are afraid other people might by stupid and genuinely insulted.

The only first hand evidence, like here, is given by people with a commercial interest, for example in exploiting the stories in lectures or courses to business travelers. So, that is why I have put out a reward. Just send me a video, or preferably a link in the comments. If it is good (neither an enacted scene nor with commercial connections) I will contact you and pay you.

Update: I discussed this reward with some colleagues (GL, EO and A). They felt I shouldn’t be the judge so they volunteered to be a panel.

Update: The reward has gone up from 50 to 100 to 150 euro.

ps. Here’s the nice generator

Update – An ‘almost’ example: this bit of cross-atlantic communication between Bush and Merkel comes fairly close. Yet I do not think Bush’s gesture (giving a shoulder rub) was misinterpreted. It is a gesture of closeness, saying it is okay to let your guard down. This can be accepted by the receiver, who is then however in a bit of an underdog position. They both seem aware of this. Merkel, however seems surprised and her response is clearly one of not accepting. Instead of undergoing the shoulder rub and all its implications, she puts her hands up and smiles uneasily. To me this indicates she makes a compromise between shrugging him off and letting him save face.

Trust the Brazilians to come up with a great move, be it in football or gesturing. I already read and saw the Brazilian gesture for speed, but you really need to see the action.

Can you do it? Hold your thumb and middle finger together, relax your finger muscles, but hold your wrist. Then shake your hand so your index finger snaps against the middle finger. It sounds a bit like snapping your fingers, which in the U.S. can mean quickly as well, but is mostly a rude way to draw attention from a waiter or a dog, or a nice way to accompany singing.

I’ve been practising for a while now, but failed to achieve any result. My fingers remain silent. The best I got were barely audible pats. But it took me a while to learn to snap my fingers as well when I was a kid like this one. My colleague B. in the video reassured me that I’m not the only hopeless wannabee Brazilian. Perhaps I’d better try my luck with football?

The 2006 FIFA World Cup is dominating the hearts and minds of most western Europeans at the moment. One of these is Rob Oudkerk who wrote a column about the gestures Marco van Basten was making at Netherlands vs Serbia-Montenegro.

The man with a plan?

Oudkerk appears to be a gesture conaisseur. He can tell Marco van Basten is a visionary leader with a mission, as he shows the signs of a champion. He can also see the opposite from Balkenendes cheers (a political adversary). And he doesn’t stop there. The entire inner life of San Marco, his emotions, thoughts, and feelings became clear as Oudkerk watched a video summary of the coach during the match.

The confident, alert body language of a prime minister to be? Nothing escaped his keen eye for politics, or did it?

As an experienced Dutch politician Oudkerk always enjoyed the gestural displays in parliamentary debates. They say so much more than words. Some of his own famous last words as a ambitious centre-left politician were a nasty comment about immigrants. It was accidently recorded. I guess sometimes words do say more than gestures.

I came across a strangely informative site by a couple of students. They collected quite a bit of ‘Nonverbal Communication’ bits and pieces for their web project. For die-hards only, be warned.

Perhaps the most popular form of nonverbal communication among students?

They seem to promote Public Displays of Affection, or PDA. They put together a reasonable cross-cultural comparison of common greetings, courtesies and practices. There’s a fairly indepth piece on Eye Contact and Movement. The bit on Sign Language is ridiculously incomplete. Next are chaotic collections about Facial Expression and putting up a Front. Last but not least there are essays on Posture and Body Positioning and The Handshake.

All in all I would say it is not a bad job and a solid B is in order (the wonderfully inappropriate graphics sadly prevent an A). But can someone please tell them nobody calls it nonverbal communication anymore?

According to Desmond Morris (Gestures, 1979) the old, widespread, mocking gesture known as ‘thumbing your nose (at someone)’ may be ‘a crude pastiche’ of a fan gesture. Though it’s unlikely to be the original source the mild resemblance has probably played a role at the time when fans were very popular. The gesture is also known as Queen Anne’s Fan, Spanish Fan, and Japanese Fan.

Can’t afford a fan to blow someone off? Just cock a snook at him!
Source: PUNCH, OR THE LONDON CHARIVARI. VOL. 1., p10, JULY 17, 1841. (online)

Here are some online manuals or dictionaries: Fan Language which seems to be mirrored here as 32 Victorian Fan Signs. And here is Spanish fan language where it appears still to be taught. I suspect a system of fan gestures was restricted to the court where it was used. These Spanish signs are different from the Victorian ones. More fan language is shown here at Kara’s Flamenco Page.

I love you – Tomorrow – Write me – I Can’t – I’m Alone – Stop – Yes (Bleschunov Museum). Would you get the hint? Is contemporary etiquette so loose that such messages are simply said?

A while ago (April ’05) BBC News already had a topic on project Audioclouds by researchers at the University of Glasgow. The aim is to control gadgets using movement and sound. Motion is sensed using accelerometers, not via cameras. The coverage by the BBC led to a discussion on engadget about the desirability of such gesture-controlled gadgets. Most of the obvious pros and cons are given, as well as more careful arguments. Since then the project moved on, featuring contributions to lots of HCI conferences mostly.

How would you feel about dialing a number by tracing it in the air? (PC World)

Gesture-controlled, or motion-controlled gadgets: DJammer, iPod, Nintendo Wii, Vodaphone’s Sharp V603SH handset, Pantech’s PH-S6500, LG Electronics’ SV360, Samsung Electronics’ SCH-S310, Antar,