The curiousness of children is infinite. My 18 month baby is into everything, looking, pointing, grabbing, pulling. As any parent will tell you, it is pretty relentless as the world is explored anew. With each discovery comes the instinctual need to communicate with us what they are thinking and feeling. It is phenomenal how much is actually being taken in but of course without language, deciphering the baa-baa-beedahh-dees feels like a task for Bletchley Park.

What has been incredibly enlightening however was how gesturing through sign language can help to bridge that gap. After several months of “Sign and Sign”, which is a baby group to teach basic sign language, I was able to communicate about specific things; abstract and literal. For example, if she sees a star then fanning the fingers out from a fist tells us or if she is hungry or has a pain then a simple gesture lets us know. The recognition of understanding and being able to communicate is self evident, it is what we all continue to try to achieve even after the nappies are off.

The parallels between this and gestural interaction are clear. If you can communicate with your audience using simple gestures they are likely to become engaged more easily as the threshold to participation is lowered. I no longer need to coordinate by hand with the mouse to cause a cursor to move to a specific point to click…baa-baa-beedahh… I just press, or circle or swipe.  It is inherent, immediate, intuitive and why these types of interfaces are becoming woven more tightly into our social fabric.

Of these the Touch addition, buried at the bottom of the list?! represents the biggest new playground for designers. Whilst touch interaction is nothing new, the native support for it into what will become the most installed OS around is huge.

Having Touch not only changes the way we interact with information but changes the way the information should be displayed.

Getting your screen hands dirty

As you’d expect the inventory of flicks, swipes, twofingercombomoves, taps and pinches is all there. Using the simple flick gestures we are able to move around the interface dragging up, down, moving back and forward as well as performing tasks such as copy, paste, undo, and delete, right there in and amongst it. w00t!

Practicalities aside what gets me hot is that this method of interaction allows the user to really get stuck in or as Ben Shneiderman would say directly manipulate the interface. It brings with it the high probability that we will see more minority report UI’s but more importantly it presents opportunities to break down the engagement barrier of having to operate a mouse and keyboard and in doing so opens up new richly immersive environments and online services where involvement is more than cerebral, it’s physical.

In doing so our audience suddenly got a lot broader. Gestural interaction lowers the threshold to participation, just look at the way the Wii has found a home in households. It means generations of older folk and those who do not work with computers everyday can do the things they want to without technological intimidation or hangups.

Look and Feel

A new aesthetic is also required to play in this world, one that takes cues from kiosk applications and the burgeoning world of iPhone and its immitators. Input controls need to be bigger to deal with our low fidelity fingers (sounds odd saying our fingers are low fidelity..) than the pixel perfect pointer.

Controls also need to be spaced further apart. This means throwing menus tight into the edges of your application isn’t the best move for Touch as it makes activating them more difficult. This changes what we’d traditionally expect according to Fitts’ Law which sates “the time required to interact with a target depends upon the size of the target and the distance to it. The smaller a target is, and the further away it is, the harder it is to use. But due to the large surface area of the fingertip, small controls that are too close together can also be difficult to target precisely.” MSDN guidelines

With the advent of Touch we are also seeing a renewed interest in haptic feedback. The lack of this has been one of my gripes with the iPhone and I’ll often turn on sounds to use the phone keypad as a way to give the buttons some weight. With haptic we can do things like vibrate the device when a button is pressed or use it to signal an event. This facet provides another way to make our Touch experience authentic and I’m interested to feel the nuances in design as interfaces and brands differentiate themselves through vibrations.

*Waves goodbye*

As Touch enabled computers permeate the market it means designing interfaces just got a lot more exciting. It also means things have got a lot more complicated as now we need to consider interaction across a new input device and all that this tactile one brings with it. In some ways it reminds me of creating a Flash site and an HTML site, both to essentially convey the same information albeit via a different experience. How do you design for both concurrently without compromise? Are users likely to stay in a mode or is it constantly in flux dependent upon task? How does it change what we already know about usability? What new services are going to appear that enrich this form of interaction? How will current ones adapt?…

How we choose to design for this will take time as we become au fait with the experience. In turn expectations will change and with it new patterns will undoubtedly arise. I’m currently working with the talented folk at Splendid who have developed some great applications and concepts for MS Surface and are inventing some of this future. More about this another time.

If you have worked on or seen some really great examples of multi-touch interfaces let me know about it in the comments, would love to check it out..

Related reading

Dan Saffer wrote a book about gestural interfaces
Microsoft’s Touch guidelines

I attended an inspiring interview with Nolan Bushnell at BAFTA last night. A wildly successful charismatic character, in an unassuming kind of way, Bushnell of Atari and Pong fame took us on an informal excursion into his innovations in gaming c1970 through to his current uWink business.

Avoid missing ball for high score

The common thread which ran through his anecdotes was one of unfailing optimism and self belief coupled with a fascination for play. He built the Atari console after being unable to convince an established game publisher to take on Pong - ball missed; Chuck E.Cheese because he understood that kids do not want to sit still and talk quietly at diner but want to run around and cause havoc. Chuck E. Cheese now turns over $1bn a year.

It was refreshing to hear about these adventures. Like many designers intent on developing interesting ways to engage with people, his ‘let’s just do it’ attitude is what we seek out in others and often precisely what is needed. The experimentation of investigation that leads to new understandings any connections.

Engineering in fun

His work is the precursor to so much of modern gaming, many view him as the founding father of computer gaming. The underlying sense of philosophy to games and motivation can be seen directly in today’s Nintendo Wii. When Bushnell talks about social gaming he is interested especially in the embodied physical social experience rather than the distributed network experience of online gaming, predicting the continued growth and development of gestural interaction to a point where participation is simply a matter of moving ones body to effect change, negating the need for an additional input controller.

Other stories included his decision to incorporate “reflexing” into gameplay. Everyone is aware what it is like to play a game against a more skilled opponent. It’s great to improve your own game but also a little demoralizing. Bushnell introduced a simple handicap system whereby the difficulty of play altered in response to the players skill. So in multi-player Pong this meant that the more a player hit (“hogged”) the ball, the shorter their paddle became until it eventually disappeared. This meant that the player had to depend on their team mates.

When asked about his preference for modern game pads compared to the joystick he found that players could excerpt a massive amount of energy when they are getting beaten and there was no way to make an affordable joystick robust enough to withstand the reflected punishment or as Bushnell observed “Physics killed the joystick.”

Related links

Google tech talks: Nolan Bushnell

The event was being recorded so I’ll post a link here when it is online.

 —  —  —  —  —  — 
here’s the video link http://bafta.org/learning/webcasts/nolan-bushnell,727,BA.html
 —  —  —  —  —  —