Various enterprises and personal interests, such as Man-Machine Interaction (MMI), gesture studies, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Month: April 2007 Page 1 of 2

Montoya fined $10.000 for a playful finger

I agree with some of the comments on YouTube that the Americans have a strange fear of the finger. They will not even show it unscrambled in this video clip. In general US media do not show pictures of people flipping the bird or making what they feel are insulting or obscene gestures.

I believe that the finger is not really obscene, but in most cases a gesture of defiance. It can also be a playful gesture of outsmarting someone or being smug to the competition, as it is in this case, which is related to defiance.

But there is hardly ever any sexual connotation at all. Nor is there always an insult. Unless you feel insulted by the open defiance. Perhaps some people simply cannot stand other people showing their defiance. Do they have difficulty accepting gestures of defiance? Somehow this is turning into something alltogether too much political and too close to current events.

Not a Nice Gesture by Baros

Lyon striker Milan Baros will face a disciplinary inquiry about his “go away, you stink” gestures to Rennes defender Stephane M’Bia with during a Ligue 1 match.

Was it racist or just obnoxious on a personal level?

My friend Gael said that if you make reference to an existing racist or discriminating stereotype, such as “all grubbers (fictional race or country inhabitants) stink” then this should be considered racism or discrimination. I agree with Gael (don’t tell him though), and find that this gesture is as much a reference to stinking as any words you could use.

Context Changes Gesture Perception

Here is a movie of one very powerful bionic finger:

The question I want to put forward is: How much of the time is this guy actually gesturing? My guess would be that he does not really give anyone or anything the finger anywhere. He holds up his bionic finger for inspection, he talks to it as he watches it, he is using it as a scanner, fighting with in different ways, but he never actually flips the bird. He doesn’t really make the gesture. Therefore, besides being funny, this video neatly demonstrates how context changes the perception of (insulting) gestures.

Space Invaders with Gesture Recognition

The more I think about it, the more I am convinced that the (near) future of gesture recognition lies in entertainment. Here is yet another gaming application with gestures: A multi-player, wall display remake of Space Invaders. A highly advanced gesture interface seems to allow any kind of movement at a certain spot on the baseline as ‘fire from here’. A camera tracks if a hand blots out one or several of a series of lights that represent the positions on the baseline.

YouTube: Development: Douglas Edric Stanley / This is installation was developed on-site for the Gameworld exhibition at the Laboral Art Center, Gijón, Spain. March 30 – June 30, 2007. For more information visit the responsible art centre in Spain.

Signed Children’s Rhymes

I once bought a very nice CD-rom from Effatha-Guyot with 19 signed children’s rhymes. The short songs or poems (not sure what to call them exactly) are by Diny Visch and Arie Terpstra.

Here is an example of the title-rhyme called ‘The Elephant and the Mouse’:

Original title: “Van de Olifant en de Muis… Peuterliedjes in Nederlandse Gebarentaal”

These signed poetic expressions (still struglling) are interesting. They demonstrate several aspects of how one can play with sign language: rhythm, handshape repetition, movement repetition, symmetry (or alternation), etc. And it is a rare example of something that is actually tailored to very small kids (without a biblical background).

Update April 19: I finally found out the videos did not play for anyone but me, so here they are again.

Morrison fined $25,000 for giving the finger

Adam Morrison, a Canadian basketball player, was fined $25.000 for giving the finger to a nagging fan.

Finger in happier times
Well, it was probably another finger but nobody is hosting an image of the incident anywhere… (source)

Is this fine fair? Here are some exactly similar cases to help you decide a fair punishment:
* Zach Randolph (basketball, USA) was fined $133.333
* Mark van Bommel (football, Germany) was fined EUR6,200
* Michael Vick (American football, USA) was fined $10.000
* Ron Artest (basketball, USA) was fined $10,000
* Natasha Zvereva (tennis, Wimbledon) was fined $1,000
* Juan Pablo Montoya (NASCAR, USA) was fined $10,000

More on fines and jailtime for gestures

Perception of Fidgeting during Signing

Do signers fidget during signing? This is a question I often pondered. How do signers handle fidgeting during signing. I have often witnessed that signers were fidgeting (rubbing their nose, or other self-adaptors) during signing. Even for me, a hearing non-signer, it seemed easy to spot such fidgeting movements as not being part of the signing. Now, luckily I came across a nice joke told in ASL at by Kenton Hoxie, a Deaf vlogger (one of a fast growing group of ASL video bloggers cruising on YouTube). Obviously I cannot follow the joke, maybe you can.

But I will propose a game of my own to you: Spot the Fidgeting!: Try to spot the fidgeting in this video, and ask yourself what makes you see it.

Is sign perception hampered or disturbed in any way by fidgeting? I would find it hard to believe, as even I can filter it out, and ignore it while trying to understand the signing.

It is also interesting to check whether fidgeting is like saying ‘ehm’. Surprisingly, fidgeting occurs with one hand while the other is signing. But it does seem to occur at pauses in the signing (between phrases?). It might therefore be related to utterance planning (which I think saying ‘ehrm’ is as well). If anyone has suggestions about this, they are more than welcome.

Iconicity in Gestures and Sign Language

Let us assume for a moment that Mueller was right: Iconicity in Gestures is achieved by hands that act, embody, model and draw. We further agree with her that achieving iconicity presupposes certain knowledge in the observer (and in the producer for that matter). But it is general knowledge about techniques such as modeling. It does not require spcific conventional knowledge of symbols. What we have is then a handful of strategies that people can apply to try to produce (or understand) gestures.

We can then ask the question: To what extent can people rely on the strategies of gestural iconicity to understand and produce gestures or sign language? Two examples came to my mind that I would like to share. The first is about applying the rules of gestural iconicity to reading sign language (jokingly) if you do not know it. The second example is of a applying the rules of gestural iconicity to create a fantasy signed interpretation of the lyrics of Torn (also a well known joke).

My first impression upon revisiting these movies was that the rules of gestural iconicity are at work here. It seems to be a good way of looking at what these people are doing. The joke interpretation of signing is almost entirely built from guesses at meaning through applying iconicity rules (needless to say, not very succesfully). But at the same time it does not seem to be enough. One thing especially appears to be missing. Much of the pantomime in Torn is of displaying emotions with the face and the body. Although you could see that as enacting, in the sense of imitating, it seems to be a slightly different matter.

Nyst Documents Adamorobe Sign Language

Victoria Nyst defended her PhD thesis titled ‘A Descriptive Analysis of Adamorobe Sign Language (Ghana)’ at the University of Amsterdam (UvA) two weeks ago (March 30). The local university website wrote a short piece about this Unique sign language in African village with high hereditary deafness.

One claim is that AdaSL is highly iconic; can you guess this sign? (source)

The village Adamorobe has a high incidence of hereditary deafness so there is a local sign language that both deaf and hearing villagers use. In the summaries there are some speculative claims about the different ways a sign language can develop or has developed in this case. Whether or not there is a large and isolated Deaf community is suggested to be the main factor in the arguments.

* provides a good summary in Dutch.
* The Foundation for Endangered Languages also has its eye on AdaSL.
* Promotors are Prof. Anne Baker (UvA), Prof. Maarten Mous (Leiden University)

Hands that Act, Embody, Model, and Draw

At a 1998 conference called Oralité et gestualite (ORAGE 98). Communication multimodale, interaction. Actes du colloque. Cornelia Mueller presented a paper on Iconicity and Gesture (1998). The ideas already present in this paper about how the hands construct iconicity were important in the lecture she gave last week together with Alan Cienki on Metonymy in Metaphoric Gestures.

Abstract Müller 1998: Iconicity is thought of as a natural property of gesture and is considered to require no further investigation. I will show that iconicity in gesture is achieved and how it is achieved. I will specify four different modes of representation which make clear how iconicity in gesture is constructed. On the basis of ethnographic descriptions and semiotic analyses of gestures in sequences of Spanish and German dyadic conversations, different forms of semiosis are distingished. It is shown that gesturing is based, on the one hand, on a diverse set of everyday practical activities such as opening doors, driving cars, and giving or taking objects. On the other hand, various cultural representation practices such as moulding and drawing are employed. Occasionally, the hand portrays the object itself, for example, a flat hand is used to portray a piece of paper. It is concluded that iconicity in gesture is not restricted to ‘iconic’ gesture, but is one of the most powerful means of constructing gestural meaning in general.

During the lecture slightly different terminology was used for the different modes of representation: * Hand acts (imitates) * Hand models (moulds) * Hand draws (tracing) * Hand embodies (portrays)

Hand that moulds

The hands that mould? (source)

The picture above is a joke at first sight of course, but just imagine how you would explain the procedure to someone else without the equipment or the material. I think there will be a lot of enacting, modeling, and tracing involved. It seems to me to be a very useful way of looking at gestures. I do think that conventions are also used and may lead to more arbitrary non-iconic gestures. I think that it is also possible, or even in some cases desirable to avoid iconicity during sign formation. Consider, for example, more or less secret gesture systems (e.g. the signs made by a baseball coach to the players, which should not be understood by opponents). But in general, the principles can be applied fruitfully is my intuition.

At the talk it was suggested that the above 4 modes of representation are all cases of metonymy. It was argued that only a part of what was represented could be enacted, embodied, modeled or drawn (because the hands are not the thing). It seemed a somewhat trivial statement though, with little to offer for a deeper understanding. But the concept of metonymy might harbour more than meets the metaphoric eye? Going a bit deeper into the matter: How are these specific modes of representation perceived in case of unknown meanings? Are people able to observe and understand gestures whereby the hand enacts, embodies, models or draws without (much) contextual clues? If so, then this could perhaps explain the universal ease of communicating gesturally with foreigners.

Let me set up a little theory:

1) Hands that act can be understood because mirror-neurons (if we know the action) provide us an association with the referent action
2) Hands that embody, model or trace can be understood because the share perceptual similarities (disregarding philosophical objections, like Eco’s for the moment) with their referents
3) These are universal capabilities of humans
4) If we see an unfamiliar gesture we try to apply the strategies above to understand it

Now, if you encounter a foreigner and wish to communicate you may both assume that the other will use the same modes of representation and start gesturing. Chances are, you will work out what the other guy or girl means if your intentions are good.

Norsemen meet native american indians

Odin, please don’t let me be misunderstood. (source)

The argument could be expanded to include movement more generally. If we look at fidgeting for example I would argue that to a certain extent the physical forms of fidgeting are determined biologically and are therefore candidates for universality. So, we know those movements if we see someone else make them and we can understand that there was no communicative intent. If there happens to be a movement that we are unfamiliar with then we can try to understand it as a practical action (by looking at the result of the action and the maker’s response to that result) or as a gesture. If we try to understand an unfamiliar movement as a gesture we may follow the strategies suggested above:

* We can evaluate our embodied cognition (mirror neurons) for any associations with familiar actions (“ah, he’s making a swimming motion”).
* We can try to project a certain object into the shape and path of the hands (“he’s modeling a boat”), into the outline drawn by the hands (“he’s drawing a fish”), or into the hands themselves (“he’s pointing a gun”).

If these attempts fail we may be unable to understand the other. We may assume he refers to unknown actions or objects and try to witness or inspect the actual thing referred to itself.

Or we may just nod and smile. That usually works, too.

Page 1 of 2

Powered by WordPress & Theme by Anders Norén