Multi-touch mice from Microsoft Research

There is article and video in the Seattle Times about some research into multi-touch mice done by Microsoft Research. There is a video that include four concept mice, each using a different concept and technology. This video is worth watching to see what kind of concepts they came up with.



Before watching the video, I was skeptical about the utility of a multi-touch mouse. Mostly I think that point of view was colored by the fact that I think multi-touch trackpads as being inferior to multi-touch displays. I assumed that a MT mouse would be pretty much the same, but after watching the video, I can see there is some advantage.

My problem with the MT trackpad is that it does not allow the NUI concept of direct manipulation. Sure, it is multi-touch and can recognize gestures, but those gestures are only a small incremental improvement to the scroll-sensitive side area of trackpads. With the trackpad is used to move the cursor, the finger movement is relative to the cursor. When the trackpad is used with a multi-touch gesture, it is also relative, but multi-touch gestures are much better using absolute positioning and direct manipulation. True Natural User Interfaces are not possible with MT trackpads. (Would the iPhone be as cool with a non-touch display and a separate MT trackpad and cursor?)

In the video, they showed that each concept multi-touch mouse has one (or more) regular mouse position sensor on the bottom but also a method to detect (many!) touches somewhere on the top. This is different from MT trackpads in that the cursor positioning is done with relative movement of the entire mouse, but multi-touch gestures are done using the fingers on the MT-sensitive surface. Some of the concept mice even responded to whole palm gestures and positioning.

The separation of the relative cursor positioning from the multi-touch gestures may be a solution to the issues I had with MT trackpads. In addition, it allows additional degrees of freedom so that you can rotate and stretch a picture using gestures while and panning by moving the mouse all at once, for example, whereas with the trackpad you would likely be limited to one type of manipulation at a time.

I can envision an ecosystem where multi-touch mice being accepted as a low-cost enabling technology that introduces people to Natural User Interfaces without a large investment. Those would be used in addition to or to complement multi-touch displays and larger Surface devices. It would benefit NUI adoption by allowing existing computers to use NUI software. While it would add to the complexity of planning interfaces across a larger variety of hardware capabilities, the potential for mass adoption may be more important.

Update: Picture of the mice courtesy of CrunchGear: