Various enterprises and personal interests, such as Man-Machine Interaction (MMI), gesture studies, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Month: July 2006 Page 1 of 3


Two weeks leave means two weeks without blogging.
CU, Jeroen

Help, my phone gestures

There are now several phones with motion sensors allowing you to interact with them using gestures (let’s not worry about definitions for once). Imagine you are a software developer for such a phone. Obviously you would want to trace numbers in the air to dial them (?). What else would yor contribution be? Some examples for your inspiration:

Look! The Dots are Going Up!

CHICAGO (Reuters) – Scientists reported on Friday what they said was the first scientific evidence that people unconsciously gesture with their voices. What a world of assumptions lies hidden beneath this innocent looking pool of semiotic quicksand.

What did Howard Nusbaum, Hadas Shintel, and Arika Okrent do? They let people look at dots and then say whether they went up or down. Separately, people read “It is going up” or “It is going down”. If the dots went up, their pitch rose. It lowered when the dots went down. This also happened when the sentences were read. This is the ‘unconscious gesturing’ mentioned in the title.

Look, the dots are going up (say in a rising pitch)

First of all, the people did not ‘gesture’ with their voice, they modulated the fundamental frequency (F-0) of their speech signal to a higher frequency, or you could say they shortened its wavelength. If we assume the term gesture to form a pair with voice to encompass Man’s communicative behaviour, surely intonation (and all other prosody) falls under voice. Why would it be useful to shove intonation under gesture? And in fact Nusbaum et al. talk about analog acoustic expression at first and only compare these to gestures. Do they mean gestures are analog? And if so, what are their digital counterparts? Sign language signs? I think this is what they have in mind.

(More on this? Read Scott K. Liddell, Grammar, Gesture, and Meaning in American Sign Language. But in general I disagree with this presentation of gesture. I think even co-speech gesturing is a mix of ‘analog’ and ‘digital’ encoding of meaning. People simply have many strategies for semiosis which can be employed in just about any modality.)

Second, talking about a pitch going ‘up’ or ‘down’ is a convention. There is nothing natural about it as such. On a piano it goes from left to right.

Is praying to the Lord above simply a natural, analog manual expression? (Image by OYBB)

Third, why would the intonation be any more or less ‘conscious’ than the words themselves?

Reward Raised to 100 Euro

Since there were no contenders since June 25, the reward for a gesture mix-up video is raised to 100 euro. Any takers?

Is An Octopus’s Signal in his Shade?

Behold the Octopus. The James Bond of the sea.

Ringo Starr once wrote:

I’d like to be
Under the sea
In an Octopus’s garden
In the shade

They pinned a romantic story on him about Octopi being aesthetic collectors. But an Octopus’s garden is no more than the leftovers, the bones, spines and shells, outside a den. Of course, a true romantic could counter that an Octopus picks aesthetic food…

The question I would ask is: does an octopus gesture? First, let us say that the term gesture originated in the distinction between voice and gesture, or oralité et gestualité, as the French would say. The pair together refer to our total of communication means. It functions as a rough division of all the ways in which we create meaning for eachother. Since an octopus does not talk we could say that all his signaling behaviour falls under gesture. But that would be a fairly useless statement, unless we add that gesture is reserved to behaviour that is intended to communicate.

Go ahead, punk, make my day

At fUSION Anomaly, some gladly accept the linguistic status of the Octopus’s signaling. They even feel it is superior to our own language, which as we all know is strictly limited to our tiny mouths. What do they do exactly? They change colour and texture (see explanation of chromatophores), they may use polarization of light. But why? Often the colour and texture changes are camouflage, but chey can also indicate arousal and/or threat. I find it all rather amazing. But then I saw what cuttlefish can do…

Signing to Deaf Dogs

It appears sign language can be used with dogs as well as gorillas. Check this video (nb: ad leader). Two deaf Dalmatians, Hogan and Georgia, were taught some 65 signs by owner Connie Bombaci. Methinks this is but a classic use of some gestures to get a dog to do tricks. So what if the gestures happen to be ASL signs?

What are the main differences between ASL and Dog Sign Language?

Koko, the Talked Down Gorilla

In 1971 I was born. They called me Koko. Penny Patterson took me in. She teaches me stuff. She teaches me sign language. I am as much a person as you. Says Penny. I use 1000 signs. I hear you talk. I make small sentences. I never win a Pulitzer. But I am great blogger. My chat session was a waste. I didn’t get any nipple.

How many swear words would Koko use? (picture by A.L.I.)

Barbet Shroeder and Nestor Almendros filmed the events back in 1977, and made a movie out of it. It was brought out on DVD recently with extra material (hence this post).

Some reviewers focus on the strangeness of the relationship between Penny and Koko (the one scolds and talks down the other as an unfortunate child). Other reviewers stay with the main message about big apes. Funny thing, the movie is not mentioned on Penny’s Koko’s site.

Dolphins 1 – Apes 1

GW 2007 Call for Papers

Upcoming event: GW 2007, or The 7th International Workshop on Gesture in Human-Computer Interaction and Simulation 2007 (23-24-25 May). Edificio ISCTE, Lisbon, Portugal. It’s a biannual thing so take this opportunity or wait to 2009.

The web site is empty but for the Call for Papers. For papers, the deadline is 19 January 2007.

The Dolphins Got the Point and Left

Pointing appears ever so natural to us. Everyone points (even though most of us feel it’s rude to point at someone) and you don’t need to learn it, right?

As a young father of two I wonder: Could finger-pointing be innate?

Well, an interesting point is raised by animal studies. Even though chimpanzees follow our gaze it seems difficult to point something out to them, contrary to dogs and wolves. Now, a study pointed out that dolphins understand pointing as well.

If you have read So Long and Thanks for all the Fish it will not surprise you that dolphins understand us. The real question may well be: Why don’t we understand dolphins? (with the possible exception of this guy.)

Dolphins 1 – Chimps 0

Saint Peter Watches Over G8

In a world full of wonders, it is but a small step from a fish called Saint Peter, to the town of St Petersburg, where the G8 is now being held. And joining the online readers today with a nice analysis of the G8 summit body language is Dr Peter Bull of York University (for BBC News).

Well it’s one for the money, two for the show…

Apparently we need hardly listen anymore to what they say. Dr. Phil Peter shows us how we can read the state of interpersonal affairs from a couple of pictures. And I have to say, it really fits well with what I already felt. It’s uncanny, just like when I had my hand read last year.

Do we not simply project
that which we like and expect?

Page 1 of 3

Powered by WordPress & Theme by Anders Norén