Beyond Multi-Touch: Touchless Device Controls

To go beyond touch and create touchless 3D device control interface, Nokia decided to use Ultrasonic Transducers (USTs), arrayed around the perimeter of the display.

Ultrasonic transducers emit ultrasonic sound waves that propagate through air. When finger gets into the path of these waves, the signal is reflected back and detected by the transducer.


Three or more transducers (14 A-B-C in picture above) arrayed around a perimeter of the mobile device create a 10-20 cm working volume of space above the display, where user finger ( 20A and B) locations and their movements in real time can be detected using triangulation techniques. These movements can then be interpreted as various three dimensional gestures.

To ease the operation of the device, visual finger pointers (12A, B) are displayed on the screen of mobile device.

3D gestures are translated into various object oriented, or gesture/browsing oriented commands. The patent application gives several examples.

Object oriented commands:
  • Select: – Picking up gesture – Finger 1 at a display corner or some reserved area, Finger 2 moves slowly under a displayed object to be selected
  • Copy: – when selected, click by single finger on the object
  • Paste: – fast double click by a single finger
  • Move: – move slowly two fingers located on moving objects
  • Delete: – double (fast) click by two fingers on previously selected object
  • Switch: – switching (on/off) is based on change in directions of fingers movement or, alternatively, on a change in finger acceleration
Gesture/browsing oriented commands:
  • Select object attributed to the pointer position: Open/closed hand
  • Forward/Backward Browsing: anticlockwise/clockwise cyclic rotation by a single finger
  • Zoom in/Out: expand/close two fingers
  • Run/Execute pre-selected icon/command: make a circle with your thumb and pointing finger (an OK sign)


In addition to operation of the device, 3D tracking can be used for text entry. In this case virtual keyboard is presented on the screen of the device, with visual and audio effects providing the necessary feedback to the user, e.g. symbols change color when a finger moves above them, become bigger and a click sound is heard when a key pressed and accepted.

Different finger combinations can be used also as a shortcuts to various applications. Make a letter C with you fingers and your calendar application launches, use a “V” sign to launch an address book. And if the operation of the device grinds to a halt and starts to really piss you off, just give him a “finger” and it resets to start anew ;)

Via Unwired View

This entry was posted in Multitouch, Navigation, Nokia, UI and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s