Monday, February 9, 2009

GUI + Touch != NUI

Since the Windows 7 pre-beta and beta has been available, I've been reading some articles and seeing videos where people review the touch/multi-touch aspects of the new OS. Opinions have been mixed:

Engadget: Windows 7 Multitouch: It's a gimmick (for now)

OSNews: Windows 7 Beta's Touch Capabilities Underwhelm

Lifehacker: Windows 7 Multitouch: So Far, So Good

Gizmodo: Windows 7 Touch Control Makes Media Center More Awesome

GottaBeMobile: Will the multi-touch ecosystem be ready for Windows 7 launch?

A common theme can be seen across all of these articles -- applications designed for touch (such as Media Center) work well, but regular Windows and applications do not benefit from touch or multitouch. At best, the multitouch features in the Windows 7 shell are forced.

Media Center (originally designed for use on living room tvs) is probably the best built-in experience. There are only a few MT applications out there -- the most common is the air hockey game. Depending upon the hardware and beta drivers, that game works mediocre to okay, and some of the above articles discuss.

Note this video shows pre-beta drivers on a pre-beta OS build, but watch how the user accidentally closes the app while trying to maximize it.

What this boils down to is the fact that you can't just throw a touch or multitouch screen on top of a GUI application (written with WIMP in mind) and expect it to be a new or compelling experience. NUI is not just a GUI with touch. You really have to design a Natural User Interface application with touch (among other things) in mind for the best experience.

To Microsoft's credit, they have made modifications to e.g. the task bar to make it more touch friendly, and have updated the UX guidelines to account for touch. They defined three categories for touch support:

  • Good - Free touch support provided by Windows (i.e. regular GUI apps)

  • Better - Gesture support, touch-friendly UX (i.e. GUI + basic touch aware apps)

  • Best - Touch-optimized experience (i.e. GUI + touch integrated apps and NUI apps)

(My comments in parenthesis.)

Windows 7 shell (GUI), a "Better" experience but not very fulfilling. (Note at 1:39 it switches to a Mac with multitouch touchpad, which is even worse.)

Media Center, a "Best" experience which looks and feels natural

I think that touch devices are going to have a hard time gaining market share until the are plenty of "Best" applications. This seems very similar to the rollout of the Tablet PC. Stylus input was touted as the next big thing, but there were not many stylus aware applications. Most applications were just GUI + Stylus.

If we want to build demand for NUI applications and touch and multitouch devices, we are going to have to think beyond GUI + Touch and make developers, decision makers, and the general public aware of the Natural User Interface Revolution.

What do you think we should do to make NUI applications a success in the market?


  1. Hey Josh,

    Interesting article on UX indeed. I'm not sure if the two modes of GUI and NUI need to be active all the time. Maybe it would work better if users were able to decide when they used each. It seems great for media control but horrible in most other instances at the moment. I guess it would go well on the Windows Table - see

  2. Hi Will,

    Thanks for the comment.

    I suppose an application could provide both a GUI and NUI mode, though if an application works well in NUI at all then there would be little reason to make a redundant GUI version. (Similarly, unless an application has specific reason for being command line only and not GUI, then it wouldn't make sense to make a CLI and GUI version.)

    Media control apps do well in NUI, but there are a lot of others that do well as well. The only ones that may not are the ones that require significant precise data entry (word processor, photoshop). Check out my previous posts for some examples if you haven't already. Also check out

  3. Great post, Josh. I'll be back with a few comments!