Metaphors and OCGM

It really seems like OCGM has sparked a lot of discussion! Ron first described it two days ago, and already there have been several different bloggers posting their own ideas and responses to those blogs. Lynn has a good summary of the conversation so far, and the thread on nuigroup.com also has a summary of links. I think overall the reception has been positive. There is a lot of enthusiasm for NUIs, and OCGM is just one way for people to understand NUIs better.

There have been some comments about Objects, Containers, Gestures, and Manipulations that seem to be the result of not fully understanding OCGM yet. It is a new concept, and not much has been publicly discussed yet. I want to help people understand better by explaining a bit about metaphors and OCGM.

What is a Metaphor?

A metaphor is the understanding of a complex idea in terms of another, well-understood idea. Here is an example:

Most people (myself included) do not understand the complexities of the financial markets. When the economic meltdown really got going in 2008, the media struggled with understanding what was going on and how to explain it to the layman. One term that was heard in almost every story about banks was the Toxic Asset.

The phrase "Toxic Asset" refers to financial assets with some fault that means they cannot be sold for a reasonable price. The word toxic is used to describe specific attributes of these complex financial tools. There is nothing literally toxic about them -- they're contracts and legal documents, and touching them won't poison you. Toxic waste and these financial assets have certain similar properties, and the metaphor of a "toxic" helped people understand the complex idea (financial assets) in terms of a well understood idea (toxic waste.) If you had it, you want to get rid of it, but no one else wants to take it from you.

Metaphors are used literally all the time in every day language and activities and are core to the human experience. The word "core" in the previous sentence is a metaphor to help you understand the relationship between metaphors and the human experience. Metaphors are not actually inside of the human experience, in the way the core of a planet is inside the planet. But both the planet core and metaphors are important parts of the things they are metaphorically the core of. For more on this, I really recommend reading about Conceptual Metaphors and Conceptual Blending on Wikipedia.

What about metaphors and interfaces?

Computers are extremely complex things. The software running on the computer is also very complex. In order for non-computer geniuses to be productive at all, the human-computer interface has to help the user understand the complex parts of the software in terms of something the user understands.

Even though some people claim that NUIs will eliminate metaphors, that is not completely accurate. NUIs will still have metaphors (unless you want to interact directly with binary electrical pulses), but the NUI metaphors are based around a different core set of values. The NUI metaphors will be less complex than GUI metaphors. (There we go with Occam's Razor again!) This will result in easier to use applications and more enjoyable user experiences.

Even the command line interface had a few metaphors. It had the keyboard input, which to early users was similar to the typewriter. It had folders and files, which are familiar to office workers. It had the command prompt, commands, and responses, which are similar to the interaction with service industry workers: "What can I do for you today?"


A command line interface waiting at the prompt for your input.

We all know and have discussed ad nauseum that GUI's basic metaphors are Windows, Icons, Menus, and Pointers, so I will skip discussion of those.

Is OCGM a metaphor?

You can probably guess by now where I'm going with this. Objects, Containers, Gestures, and Manipulations are each metaphors, and they will help us understand and interact with software in a different way than before.

Aside from the interface, an application that transitions from GUI/WIMP to NUI/OCGM could be exactly the same. It can still have the same databases and same business logic and same web services and everything else. The new NUI interface will be designed using OCGM as the foundational metaphors. There will be other specific metaphors based upon the application content and interactions, but those metaphors build upon Objects, Containers, Gestures, and Manipulations.

What do Objects, Containers, Gestures, and Manipulations have to do with each other?

Ron described in his latest post, Welcome to the OCGM Generation! Part 2 that OCGM has two categories (items and actions) and two subcategories (base and complex.)

I agree with the items and actions categories but want to propose a different alternate category. I would also prefer to visualize it in this way.


OCGM Category Matrix

As you can see, Objects are literal items, and Containers are symbolic items. Manipulations are literal actions, and Gestures are symbolic actions. Items are elements of the interface, and Actions are ways to ways to interact with the interface. Literal means it is real, direct, actually what you think it is. Symbolic means it is representing something else, such as the relationships between objects or a short-cut for several manipulations.

There is a beautiful symmetry around these concepts.

Fine, but what is OCGM a metaphor for?

An Object is the basic unit of content or data. If your application has a database, what are all the basic entities in the database? Most of those entities are good candidates for being an object. The metaphor is that each piece of unique content is represented in the interface by an individual, self-contained object. An Object has a one-to-one relationship with the content or data. Don't represent multiple copies of content with a single object. That is what Containers are for.

Containers are all about the relationship between objects (not necessarily just hierarchical.) Think about how the application's data is naturally organized. There could be several different ways to organize it. Perhaps certain content is naturally alphabetical (like contacts) or other content is only grouped by categories. The metaphor is that containers should present the Objects with the relationship type in mind. Note that an Object could have a Container within it, thus allowing the nesting of Objects.

Gestures represent an indirect, discrete, symbolic action. With Gestures, the important part is the actual movement of the Gesture. Here is one way to think about this: Gestures are the keyboard shortcuts of NUIs. A keyboard shortcut, such as CTRL-C, represents an action that occurs only after the shortcut is complete. The buttons CTRL and C are not directly related to the copy function, except through the shortcut. All of this is the same idea with Gestures.

Manipulations are direct, continuous actions with immediate feedback while the Manipulation is occurring. With Manipulations, the movement itself is not important (unlike Gestures) since you could take many paths to get to the same end result. To contrast with CTRL-C in the Gesture example, think about how in GUIs you could also find the menu selection or toolbar button for copy and click that. It may involve several individual clicks, mouse movements, or drags to perform. All the while you can explore to find what you want. In NUI, the Manipulation is similar. Manipulations should let you explore without unexpected consequences.

To make the metaphors a little more apparent, I made this table that shows what the complex ideas and the familiar ideas are.

OCGM Metaphors

Keep in mind that OCGM is just the foundation. Application designers will need to build upon OCGM and create specific metaphors based upon the nature of the application's content, or content relationships.

What about [XYZ] concept?

Several people have suggested other concepts that might be added to OCGM. Richard Monson-Haefel is concerned that Affordance and Feedback are not getting enough attention. One of my colleagues at work, Josh Wall, suggested Transformations and Interactions are important. Ben Reierson has suggested Exist Sense Intend Act Result (ESIAR, like easier) as an alternative.

I agree with them that these are all good things that need to be considered during interface design; however, none of these are metaphors! It would not make sense to include them in a foundational metaphor acronym that is meant to replace the phrase WIMP. Concepts like Affordance and Feedback and Transformation are important, but are important a different part of the design process than the metaphor selection. For example, Affordance would come in to play when picking a specific manipulation for a specific type of content. The visual would need to be designed to help the user realize they can manipulate the content in the intended way.