Deconstructing: Considerations for Designing Touch UI

Here is a cool video by Punchcut summarizing some considerations when designing touch interfaces. Video via Richard Monson-Haefel.


Punchcut Perspective: Considerations for Designing Touch UI from Punchcut on Vimeo.

The content itself makes sense, but I was more interested in the unique way the content was presented. The same content could have fit easily into a handful of PowerPoint bullets, but instead it is presented as if using a multitouch interface to control the content. You can see the shadows of hands in some parts and the auras of the fingers leave traces to make it clear what gestures are being used to manipulate the content.

It is a little bit like the HP commercials where the famous person moves their hands around mid-air and photos or skateboard designs or documents move and animate as well. Of course the HP commercials are not meant to suggest that you could actually manipulate content like this with their current products. Well, maybe with the sixth sense device some of those gestures and visuals could be reality, but that's not what HP was thinking about.



Unlike the HP commercials, the Punchcut video and the finger auras are actually close to a feasible design. It made me think what would have to change in order for that to be a real application. Let's deconstruct this NUI!

Each of the sections in the video illustrated a different way to transition or manipulate content. This could be thought of similar to how PowerPoint allows different slide transitions or animations, but the manipulation mode is much more fundamental here and integrated with the content. Let's enumerate the sections, my name for them, and the gestures used within.

1. (0:00) "Hello." - Flip card. Swipe to display text, then swipe to flip text around. Two-touch zoom out gesture to grow text.


2. (0:05) "Five considerations" - Pivoting text. First finger establishes pivot point. Second finger swipes to control the pivoting transition from one content item to another. Finally swipe through text ("touch interfaces") to erase it.


3. (0:12) "1" - Direct drawing. The finger draws any shape, in this case numbers, and the aura trail lingers about five times longer than in the other modes.


4. (0:13) "Design for" - Pick a word. Swipe to fade in foreground text and background words. Tap each background word in order to add it to the foreground text. Kind of like typing except with entire words instead of letters. Note that the words are preformatted and punctuated. "immediate access" shows up in the title font size, while the rest of the words show up in the subtitle font size. Swipe completed text to push it off-screen.


5. (0:24) "2" - Direct drawing.


6. (0:26) "Tap is the new click" - Scatter text Swipe to drag text in from off-screen, then more swipes to push it around. Double tap to fade in additional text. Finally swipe the text off-screen.


7. (0:32) "3" - Direct drawing.


8. (0:35) "Clear mental models" - Rolodex Swipe up to rotate text in to view in 3-D. Previous and next text phrases are shown out of focus in the appropriate place in 3-D. Swipe up again to rotate the last phrase away.


9. (0:41) "4" - Direct drawing.


10. (0:43) "Design for hand sizes" - Zoom text & Keyboard Zoom in gesture to grow initial text from zero size. Touch and drag part of the phrase open to make room. Minimal keyboard appears, type to insert text. Press the larger button (enter?) to show additional lines of text. Note the keyboard doesn't display any labels to us, just outlines of buttons. Presumably the presenter either knows what the buttons do, or can see button labels on a different screen, or the specific buttons don't matter and will insert the correct text regardless. Finally, drag the text and keyboard off-screen with separate fingers.


11. (0:53) "5" - Direct drawing.


12. (0:56) "Touch feedback is key" - Tap feedback Drag text in from off-screen. Tap each line of text for a different highlight effect. Drag text off-screen.


13. (1:03) "Touch is merely one dimension" - Directional transition by line Touch two fingers to establish an unseen line. A third finger swipes toward the line and text is animated on or off in a direction perpendicular to the line. (Sounds complicated but it makes sense when you see it.) One final swipe fades in some text.

Now that I have actually written all this down, there isn't anything on this list that would not be a reasonable way to manipulate a presentation. There are some considerations necessary to make an application like this usable, though:

• If replicating this presentation exactly, all the text and transitions would have to be prebuilt in an authoring mode. It would basically be glorified PowerPoint, and after the shininess factor wears off, it would not be that compelling. Some of the ideas below show how this application could be used in a much more dynamic and collaborative way.


• In addition, there are sometimes no visual cues to the presenter which gesture he or she needs to use to continue the presentation. Worse, the same gesture (swipe left) could do totally different things in different modes. In a presentation of any length with a variety of these transitions it would be very easy to forget the order and embarrass yourself by making gestures aimlessly trying to make content appear.


• Instead of having an authoring and presentation mode, merge the two and allow the presenter to choose the content and which manipulations to use during the presentation. This would require some type of manipulation menu and a content library. The content could still be authored in advance, but it could also be sucked up from external sources, for example a network drive or Flickr account.


• Allow the presenter to organize the content library however they want, but provide some tools to help. They might do it completely free form with piles of content, or might use a tool to draw lines between content or groups of content representing different relationships. The relationship could be specific to a presentation order or relate to the content itself. Let the user decide what works best for them. This by itself would be an enormous improvement over the enforced PowerPoint outline.


• Provide a resizing presentation area that overlays the rest of the interface. When the user is ready to present, he or she can drag content in and out of the presentation area as needed. Only the presentation area is shown on an external monitor or projector. If the presentation is really just a few people around one computer, the presentation area isn't necessary and the meeting should become more fluid and collaborative.


• Keep all the non-presented controls like the manipulation menu and unused content out of the presentation area. The presenter will be able to select from the manipulation menu or navigate the content library relationships off-screen, then bring items into the presentation area when ready. In this way the presenter can react to audience feedback and questions to bring up appropriate content, or a small group working together could just explore content and discuss it in a much more social way than linear PowerPoint presentations allow.

I better stop here, otherwise I'll give away too many ideas for free! There is a lot of potential with NUI and touch interfaces for rethinking how we did things in the GUI world. The Punchcut video demonstrates this kind of NUI thinking perfectly.