Microsoft Investigates New Touchscreen Surfaces such as Your Body
posted Sunday Oct 23, 2011 by Jon Wurm
The future is looking bright for people looking for an excuse to touch themselves and other ad hoc surfaces with purpose, literally. Chris Harrison of the Human-Computer Interaction Institute at Carnegie Mellon University and Microsoft researchers Hrvoje Benko and Andrew Wilson have been focusing their efforts on creating OmniTouch technology that expands touchscreen surfaces beyond the normal array of touchscreen devices like the phones and tablets that we have all become so accustomed to. They realized that the average human hand has more surface area than most touchscreen phones and that there were a lot of usable surfaces already existing in the real world, like your hand, coffee tables and notepads, that could be put to better use. According to Benko,
We wanted to capitalize on the tremendous surface area the real world provides. The surface area of one hand alone exceeds that of typical smart phones. Tables are an order of magnitude larger than a tablet computer.
Find out how this is possible after the break.
The hardware involved at this stage of research simply consists of a depth sensing camera (Kinect) and a pico projector. The software is a lot more complicated since the system is constantly recognizing the surface, interface and calculating the relation of your hand with regards to the surface within a depth map. In a trial with 12 people 6,048 clicks were successfully perceived at 96.5% accuracy, which is comparable to the accuracy rate of other common touchscreens.
In this case, we're detecting proximity at a very fine level. The system decides the finger is touching the surface if it's close enough to constitute making contact. This was fairly tricky, and we used a depth map to determine proximity. In practice, a finger is seen as 'clicked' when its hover distance drops to one centimeter or less above a surface, and we even manage to maintain the clicked state for dragging operations.
As of right now, the application is limited to 2D surfaces but 3D experimentation is not out of the question. As far as the Johnny 5 robot head sitting on their shoulders, Microsoft doesn't expect shrinking the hardware down to a matchbox size is out of the question but Microsoft hasn't expressed interest in commercializing this technology yet either. I seriously hope they will.
Check the project website for more information; it is awesome.
