Create the emotional connection

Any basic communications course will no doubt teach you that only a tiny part of person-to-person communication is verbal. Albert Mehrabian did research on this and found face-to-face communication consists of:

• Words (7%)


• Tone of voice (38%)


• Body language (55%)

Mehrabian found that the non-verbal elements were important for communicating feeling and attitude. For example, the feeling expressed through body language overrides the feeling expressed through words. Read the full article for more on this.

Let's apply this to NUI. Words can clearly be translated into the raw content that the NUI is presenting, such as text, images, video, and sound.

Tone of voice is used in speech to modify the words directly and apply emotional context and help the listener determine which words are important. This could translate to the presentation of the content, such as font, font size, and color for text, volume for sound, and the relative positioning and size of visual content elements. I'll just call this presentation.

Body language is communicated separately from speech and adds contextual cues about the meaning of the content. This would be equivalent to the perceived affordance of UI elements (e.g. how the UI looks like it can be manipulated), animations, and other manipulation behavior properties. I'll call this animation (since manipulating something implies it is also being animated.)

• Content


• Presentation


• Animation

In the original study, the ratios of 7%/38%/55% were weights of how much each communication type contributed to the listener liking the speaker. For our NUI analogy, the relative contribution of Content, Presentation, and Animation to the user having an excellent user experience (or liking the application) may not be exactly 7%/38%/55% but I can imagine a similar ratio.

How does this help? When designing the user experience, think about what the presentation of the content says about its importance and context. Think about what the animations and manipulations tell the user regarding how they can interact with the content.

Keep all this in mind when designing the overall user experience because it will determine the user's emotional response to the application. NUI applications focus on interacting with the content directly, but the content itself is only a small part of the overall experience. Similar to tone of voice and body language affecting whether a person likes another person, the presentation and animation of the content are critical factors in whether the user enjoys their experience.