A Nice Gesture by Jeroen Arendsen

Various personal interests and public info, gesture, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Search results: "metaphoric"

MPI Lecure on Metonymy in Metaphoric Gestures

The Nijmegen Gesture Centre Lecture Series 2007 hosted a talk by Alan Cienki (currently at the VU Amsterdam as lecturer MA in English Language and Culture) and Cornelia Müller (Berlin Gesture Center) last week. I attended the lecture ‘How metonymic are metaphoric gestures?’ together with about 20 people. The talk and discussion afterwards were perhaps a little incoherent, partly because of most people’s unfamiliarity with the concepts of Metonymy and Metaphor (or was that just me) and particularly why they would be useful for gesture studies. It should also be noted that McNeill (1992) introduced a specific use of the words metaphoric gestures:

Hand and Mind (1992), p145: Metaphoric gestures create images of abstractions [as opposed to (iconic) gestures that exhibit images of events and objects in the concrete world]. In such gestures, abstract content is given form in the imagery of objects, space, movement and the like.

It is crystal clear that there is a whole world of thought (and fun) behind the word Metaphor. Big names Lakoff and Turner wrote brilliant books about it that are the stuff cognitive linguistics is made from. And to cap it all off, Mueller and Cienki lifted a tip of the veil about a hot upcoming book on metaphor in gestures too. It will be all I can do to stop from rushing to my online bookstore (as soon as I find time). We once had a serious young undergraduate student called Michelle Hilscher for a couple of months in our faculty. Then she was doing experiments with metaphor in images and I remember having the same incoherent discussions (from my part at least) about metaphor and iconicity with her as I did at the MPI now. Curiously, she is winning awards, and I am here blogging my PhD away. Time going to waste, I better get back to grooming my paper.

ps. If you can guess how many metaphors this post counts you receive an honorary mentioning.

A Case of Co-Speech Gestures

A wonderfull new video on YouTube of two guys (programmers, it says) talking and ‘co-speech-gesturing’ (is that a verb?).

“Real programmers use sign language” (by ekabanov)

I think it is safe to assume that it is for real. Their whole behaviour looks too natural and wacky to be scripted.

I also think this is a great case study to spend some time on while discussing some of the ideas of David McNeill. Because what we have here is what his theories and ideas are concerned with. There is (of course) no sign language nor did I spot any other ’emblematic gesture’ (those vulgar things you get fined or jailed for or the goofy ones that seem to be must-haves for ad campaigns). I also do not see any pantomime. No, this is the stuff they like in Chicago: Co-speech gestures. An episode full of deictics, beats, iconic and metaphoric gestures, right?

From the McNeill lab: A misconception has arisen about the nature of the gesture categories described in Hand and Mind, to wit, that they are mutually exclusive bins into which gestures should be dumped. In fact, pretty much any gesture is going to involve more than one category. Take a classic upward path gesture of the sort that many speakers produce when they describe the event of the cat climbing up the pipe in our cartoon stimulus. This gesture involves an iconic path-for-path mapping, but is also deictic, in that the gesture is made with respect to an origo –that is, it is situated within a deictic field. Even “simple” beats are often made in a particular location which the speaker has given further structure (e.g. by setting up an entity there and repeatedly referring to it in that spatial location). Metaphoric gestures are de facto iconic gestures, given that metaphor entails iconicity. The notion of a type, therefore, should be considered as a continuum –with a given gesture having more or less iconicity, metaphoricity, etc.

Wrong! Apparently the main problems of McNeill’s typology of gestures, that has sent many an engineer on a wild goose hunt for iconic gestures, are now even recognized at the source (McNeill, 1992). It is not mutually exclusive but rather an index of the functioning of a gesture (‘as a beat’ – ‘through spatial reference (deictic)’ – ‘referring thorugh iconicity to something concrete’ – ‘referring via iconicity first to something concrete and second through metaphor to something abstract’). Good. I never liked ‘beats’ for example. I don’t think I ever saw one. But to say that it was a misconception… I vaguely recall an annotation procedure called the ‘beat filter’ that begs to differ.

Anyway, at least this clears up the discussions regarding ‘metaphoric gestures’ considerably [they are de facto also iconic, the metaphor functions on another level]. And it also clears the way for an annotation of this video. Any volunteers? Well, you would have to get a decent file of the movie instead of the YouTube flash stuff anyway, so let’s forget about it.

McNeill wrote a new book recently (2005) which is mostly about growth points. But before you read the summary by McNeill you might want to check Kendon’s brilliant poem called ‘The Growth Point‘, which he delivered at McNeill’s festen. I find it neatly captures my feelings towards growth points (and more that is beyond my grasp). I am at once awed, baffled, and stupefied when I read about growth points and catchments.

And so it goes. Again I tried to get it. Again I failed to learn anything from reading about growth points. One thing only. If David McNeill (or Susan Duncan) is right, then annotating gestures in episodes like this will be eternal hell 🙂 And without the speech it will not work. Thank God. I can go to bed with a clear conscience.

McNeill, D. (Fall 2005) Gesture and Thought. Chicago: University of Chicago Press.
McNeill, D. (2000) (Ed.). Language and Gesture. Cambridge: Cambridge University Press.
McNeill, D. (1992). Hand and Mind. Chicago: University of Chicago Press.

Gesture as a Metaphor for Musical Expression

Talking about gesture and metaphor often gets me into a critical state. I am not sure I agree with McNeill’s idea of opposing iconic and metaphoric gestures (just because they refer to something concrete or abstract respectively). I was also somewhat puzzled by the MPI Lecure on Metonymy in Metaphoric Gestures by Cienki and Mueller. And that was the sort of talking over lunch just now. One of the professors here, Paul Hekkert, is also interested in metaphors. He gave a masterclass at a workshop at the university of Tilburg, as part of their Advanced Studies Initiative on MultiModal Metaphor. We talked a bit about metaphor and gesture, and then… It struck me that in the experience of music people often use the term Musical Gestures, which I think is actually a metaphoric use of the word gesture. As far as I can tell there is no actual gesturing involved, there are just musicians playing their instruments. But somehow, when they play the music in a certain and when the audience picks that up, people start to talk about musical gestures. That would mean that the music played is understood in terms of a gesture. What is a gesture then? Well, let us say now that gesture is the display, through any action, of an intention to communicate. Musical score
A dramatic gesture of “Maternal Love” ? (source)

Going back to the music, we can then say that with a musical ‘gesture’, the musician displays an intention to communicate (his feelings?) through his playing. I guess we should add: ‘beyond that which is displayed through the usual playing of music’. And maybe we should also add: ‘or at least insofar as such an intention is perceived by an audience’. Perhaps it would even be more accurate to add: ‘insofar as an audience chooses to project such intentions to communicate on the performance’. For as far as I can tell nothing is stopping people in the audience from projecting the grandest displays of feelings on the tiniest blowing of a flute. It actually does not matter whether the musical gestures are real (produced and perceived) or imaginary (projected but not actually produced). If I have paid 50 euro to attend a concert I have every right to experience the music in whatever way suits me best.

See also:
The Musical Gestures Project
Journal of Music and Meaning
Meaning in Music Gesture

Hands that Act, Embody, Model, and Draw

At a 1998 conference called Oralité et gestualite (ORAGE 98). Communication multimodale, interaction. Actes du colloque. Cornelia Mueller presented a paper on Iconicity and Gesture (1998). The ideas already present in this paper about how the hands construct iconicity were important in the lecture she gave last week together with Alan Cienki on Metonymy in Metaphoric Gestures.

Abstract Müller 1998: Iconicity is thought of as a natural property of gesture and is considered to require no further investigation. I will show that iconicity in gesture is achieved and how it is achieved. I will specify four different modes of representation which make clear how iconicity in gesture is constructed. On the basis of ethnographic descriptions and semiotic analyses of gestures in sequences of Spanish and German dyadic conversations, different forms of semiosis are distingished. It is shown that gesturing is based, on the one hand, on a diverse set of everyday practical activities such as opening doors, driving cars, and giving or taking objects. On the other hand, various cultural representation practices such as moulding and drawing are employed. Occasionally, the hand portrays the object itself, for example, a flat hand is used to portray a piece of paper. It is concluded that iconicity in gesture is not restricted to ‘iconic’ gesture, but is one of the most powerful means of constructing gestural meaning in general.

During the lecture slightly different terminology was used for the different modes of representation: * Hand acts (imitates) * Hand models (moulds) * Hand draws (tracing) * Hand embodies (portrays)

Hand that moulds

The hands that mould? (source)

The picture above is a joke at first sight of course, but just imagine how you would explain the procedure to someone else without the equipment or the material. I think there will be a lot of enacting, modeling, and tracing involved. It seems to me to be a very useful way of looking at gestures. I do think that conventions are also used and may lead to more arbitrary non-iconic gestures. I think that it is also possible, or even in some cases desirable to avoid iconicity during sign formation. Consider, for example, more or less secret gesture systems (e.g. the signs made by a baseball coach to the players, which should not be understood by opponents). But in general, the principles can be applied fruitfully is my intuition.

At the talk it was suggested that the above 4 modes of representation are all cases of metonymy. It was argued that only a part of what was represented could be enacted, embodied, modeled or drawn (because the hands are not the thing). It seemed a somewhat trivial statement though, with little to offer for a deeper understanding. But the concept of metonymy might harbour more than meets the metaphoric eye? Going a bit deeper into the matter: How are these specific modes of representation perceived in case of unknown meanings? Are people able to observe and understand gestures whereby the hand enacts, embodies, models or draws without (much) contextual clues? If so, then this could perhaps explain the universal ease of communicating gesturally with foreigners.

Let me set up a little theory:

1) Hands that act can be understood because mirror-neurons (if we know the action) provide us an association with the referent action
2) Hands that embody, model or trace can be understood because the share perceptual similarities (disregarding philosophical objections, like Eco’s for the moment) with their referents
3) These are universal capabilities of humans
4) If we see an unfamiliar gesture we try to apply the strategies above to understand it

Now, if you encounter a foreigner and wish to communicate you may both assume that the other will use the same modes of representation and start gesturing. Chances are, you will work out what the other guy or girl means if your intentions are good.

Norsemen meet native american indians

Odin, please don’t let me be misunderstood. (source)

The argument could be expanded to include movement more generally. If we look at fidgeting for example I would argue that to a certain extent the physical forms of fidgeting are determined biologically and are therefore candidates for universality. So, we know those movements if we see someone else make them and we can understand that there was no communicative intent. If there happens to be a movement that we are unfamiliar with then we can try to understand it as a practical action (by looking at the result of the action and the maker’s response to that result) or as a gesture. If we try to understand an unfamiliar movement as a gesture we may follow the strategies suggested above:

* We can evaluate our embodied cognition (mirror neurons) for any associations with familiar actions (“ah, he’s making a swimming motion”).
* We can try to project a certain object into the shape and path of the hands (“he’s modeling a boat”), into the outline drawn by the hands (“he’s drawing a fish”), or into the hands themselves (“he’s pointing a gun”).

If these attempts fail we may be unable to understand the other. We may assume he refers to unknown actions or objects and try to witness or inspect the actual thing referred to itself.

Or we may just nod and smile. That usually works, too.

Powered by WordPress & Theme by Anders Norén