A Nice Gesture by Jeroen Arendsen

Various personal interests and public info, gesture, signs, language, social robotics, healthcare, innovation, music, publications, etc.

Category: Touch Gestures

Christian Stoessel Helps the Elderly

Christian Stoessel, Hartmut Wandke & Lucienne Blessing:
Gestural interfaces for elderly users: Help or hindrance?
Publications here

Christian starts out by pointing towards the changing demographics (Dutch ‘vergrijzing’). There will be many elderly people. Or perhaps  better, many people above 65 years, because these people may be more healthy in body and mind than previous generations of elderly (is that true?).

There is a somewhat optimistic view of the potential of gesture technology, in the sense that he thinks it is possible to identify sets of ‘intuitive’ gestures in a gestural interface. In the end he is measuring accuracy with which people performed gestures, rather than if they were intuitive or not.

Regarding the reality of creating a set of ‘intuitive’ gestures. He expands nicely on ‘intuitiveness’ as being something that is fuzzy. They don’t mean a gesture is intuitive from the start, but perhaps it will be more easy to remember.

Apple Gesture Patents around iPhone

I noticed a flurry of gesture patents that mentioned a ‘portable mutlifunction device’. That’s patentspeak for iPhone. The patents were all from APPLE Inc. Well done Apple. That’s how you manage a patent portfolio. Philips and IBM used to be the masters in this line of completely covering an area with a barrage of patents. It will give Apple something to negotiate with in future business deals with other vendors.

Here they all are as far as I could tell:

  1. PORTABLE MULTIFUNCTION DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR INTERPRETING A FINGER GESTURE ON A TOUCH SCREEN DISPLAY (WO 2008/086302)
  2. PORTABLE ELECTRONIC DEVICE SUPPORTING APPLICATION SWITCHING (WO 2008/086298)
  3. SYSTEM, METHOD, AND GRAPHICAL USER INTERFACE FOR INPUTTING DATE AND TIME INFORMATION ON A PORTABLE MULTIFUNCTION DEVICE (WO 2008/086073)
  4. APPLICATION PROGRAMMING INTERFACES FOR GESTURE OPERATIONS (WO 2008/085848)
  5. MULTI-TOUCH GESTURE DICTIONARY (WO 2008/085784)
  6. GESTURE LEARNING (WO 2008/085783)
  7. PORTABLE MULTIFUNCTION DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR INTERPRETING A FINGER SWIPE GESTURE (WO 2008/085770)
  8. PORTABLE ELECTRONIC DEVICE, METHOD AND GRAPHICAL USER INTERFACE FOR DISPLAYING INLINE MULTIMEDIA CONTENT (WO 2008/085747)
  9. PORTABLE MULTIFUNCTION DEVICE,METHOD, AND GRAPHICAL USER INTERFACE FOR TRANSLATING DISPLAYED CONTENT (WO 2008/085744)
  10. OVERRIDE OF AUTOMATIC PORTRAIT-LANDSCAPE ROTATION FOR A PORTABLE MULTIFUNCTION DEVICE WITH ACCELEROMETERS (WO 2008/085741)
  11. METHOD, SYSTEM, AND GRAPHICAL USER INTERFACE FOR VIEWING MULTIPLE APPLICATION WINDOWS (WO 2008/085739)
  12. METHOD, SYSTEM, AND GRAPHICAL USER INTERFACE FOR PROVIDING WORD RECOMMENDATIONS (WO 2008/085737)
  13. Somewhat earlier this year: DELETION GESTURES ON A PORTABLE MULTIFUNCTION DEVICE (WO 2008/030975)
  14. SOFT KEYBOARD DISPLAY FOR A PORTABLE MULTIFUNCTION DEVICE (WO 2008/030974)
  15. PORTABLE ELECTRONIC DEVICE PERFORMING SIMILAR OPERATIONS FOR DIFFERENT GESTURES (WO 2008/030972)
  16. EMAIL CLIENT FOR A PORTABLE MULTIFUNCTION DEVICE (WO 2008/030970)
  17. PORTABLE ELECTRONIC DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR DISPLAYING STRUCTURED ELECTRONIC DOCUMENTS (WO 2008/030879)
  18. PORTABLE MULTIFUNCTION DEVICE, METHOD, AND GRAPHICAL USER INTERFACE FOR CONFIGURING AND DISPLAYING WIDGETS (WO 2008/030875)
  19. PORTABLE ELECTRONIC DEVICE FOR PHOTO MANAGEMENT (WO 2008/030779)
  20. PORTABLE ELECTRONIC DEVICE FOR INSTANT MESSAGING (WO 2008/030776)
  21. 2007: UNLOCKING A DEVICE BY PERFORMING GESTURES ON AN UNLOCK IMAGE (WO 2007/076210)
  22. 2006: GESTURES FOR TOUCH SENSITIVE INPUT DEVICES (WO 2006/020305)

Who will be able to argue with this patent portfolio? Who will be able to claim that the things Apple has patented were already invented elsewhere? Who will be able to maintain that gestures are not technical inventions but natural human communicative actions? Who will pay the lawyers to fight these fights?


Here it all is in a fashion that is easier to digest than sifting through 22 patents.

I think Apple has won this fight before it could even get started.

Is Cube-Flopping Gesturing?

Fellow PhD student at the TU Delft, Miguel Bruns-Alonso created a nice video of his Music Cube (his graduation project, see paper). And then Jasper van Kuijk (another colleague) blogged it for usability. And here I come wandering wondering: whether moving this Cube in certain ways to control music playing can or should be considered gesturing.

 

Perhaps this is a highly irrelevant question. I am pretty sure Miguel could barely care less. But that’s me, always worrying about silly gesture stuff.

In a way the question is similar to a previously unanswered question Is Sketching Gesturing?

Like with sketching it is not the movement itself that matters. Rather, it is the effect that the movement causes that is important. Although the case of “shuffling” may be an exception because the “shaking” movement is fairly directly registered. Other commands are given by changing the side of the Cube that is up (playlists), or by pressing buttons (next, turn off), or turning the speaker-that-is-not-a-speaker (volume). These are fairly traditional ‘controlling’ movements, comparable to adjusting the volume or radiofrequency with a turn-knob (as in old radios).

I will leave aside the question whether such tangibility constitutes a more valuable or enjoyable interaction with our machines. Some believe that it does and who am I to disagree. Like it or not, take it or leave it, you choose for yourself.

What concerns me is whether such developments and other gesture recognition developments share certain characteristics. If so, then exchanging ideas between the areas may be a good idea. One of my bits of work is on discriminating fidgeting and gestures.

The question rises whether the Music Cube will allow people to pick it up and fidget with it without immediately launching commands. Can I just handle it without ‘touching the controls’? Like with other gesture recognition applications I want this Cube to allow my fidgeting. In that case rules for human behaviour regarding the difference between behaviour that is intended to communicate (or control) and behaviour that is just fidgeting would be useful. And why don’t we carry the thought experiment of the Music Cube further? If it has motion sensing, it should be able to do the sort of things that the Nintendo Wii can too. Why not trace gestures in the air to conjure up commands of all sorts? How about bowling with the Cube? Or better yet, playing a game of dice?

Is sketching gesturing?

My brother in law Coen sent me a link to a YouTube video about a digital drawing board: It raises a nice question: Is sketching gesturing? Two easy answers: Yes, it falls under gesture because it is movement that is intended to communicate (or movement that expresses intention. No, it is different because it is not the movement itself but rather the result, the lines drawn, that convey the message. A few observations about sketching and gesturing: (1) If you would like to sketch but do not have pen and paper, you may very well revert to gesturing and ‘sketch in the air’. (2) If you explain a sketch it will often include gestures that enrich or clarify the message contained in the sketch. (3) There is style of drawing and painting that refers to gesture. It is called gesture drawing or action painting. The main idea is that the resulting art should capture or even show the original movements, for those are deemed an important expressive power. The Pirates (source) I am afraid I will leave the question unanswered for you for now. You are cordially invited to share your opinion. I will point you to an interesting book that, in my opinion is a must read to study the relation between gesturing and sketching: André Leroi-Gourhan (1964-65). Le geste et la parole, Paris, Albin Michel. (amazon.fr) English translation by AnnaBostock Berger, 1999: Gesture and Speech, MIT Press. An excellent review by Copple in Gesture. And to be frank, I did not read beyond Copple’s review yet, so I want to wait before making any sweeping statements on the issue. I do like the overall theory that both gesture and drawing are tied to our capacity for symbolic communication (but that is such a general remark it is almost trivial).

Flat Gesture Recognition

Here we find another example of gesture recognition straight from the heavens above.

At Philips they tinkered a bit with Looking Glass and HandVu, and now they have got it: A gesture controlled home environment. Just the thing we will be needing if our future is anything like minority report. What strikes me most is that your gestures appear to be captured from above. You do not gesture at the camera but rather hold up your hands for inspection. It is a posture and not a gesture if ever the two are to be set apart.

I like the way they provided three different interaction means: gestures, touchscreen and mouse. That should provide people with options to explore their preferences. Speech and gesture recognition need not replace mouse and keyboard. Just add it and create multimodal interaction. (We will deal with those silly little integration issues later).

Powered by WordPress & Theme by Anders Norén