MIX11 Open Call Surface and NUI Sessions

MIX is a great conference that focuses on development, design, and upcoming technology trends. Last year, MIX started an “Open Call for Content” where the general public was invited to submit sessions and the general public voted to decide which sessions would be given actual spots during the conference. The Open Call sessions can be about any topic the community desires to hear and they augment the Microsoft sessions that make up the meat of the conference.

This year Open Call is back and there are 228% increase in proposals! Out of 384 entries, they posted 207 sessions. I read through every single one yesterday and expected to see some great sessions and some mediocre sessions, but to my surprise almost every single session sounds fantastic. MIX11 is going to have some great content.

Last year, I was fortunate enough to be one of 12 MIX10 Open Call sessions. I presented “Developing Natural User Interfaces with Microsoft Silverlight and WPF 4 Touch [and Windows Phone 7]”. (Read my MIX10 recap.) My session was one of three that focused on NUI, multi-touch, or Surface topics.

 

This year there are 8 9 entries about Surface or NUI and I’m hoping several will be picked. I’m organizing a panel session featuring the Microsoft Surface MVPs, some interesting discussion, and some cool demos. You can vote for up to 10 sessions, so please consider including the Surface MVP panel and any of the other NUI sessions that sound interesting to you in your voting.

The Microsoft Surface MVPs present: Natural User Interfaces, Today and Tomorrow; an interactive discussion and demonstrationJoshua Blake; Neil Roodyn; Dennis Vroegop; Rick Barraza; Bart Roozendaal; Josh Santangelo; Nicolas Calvi

The Natural User Interface (NUI) is a hot topic that generates a lot of excitement, but there are only a handful of companies doing real innovation with NUIs and most of the practical experience in the NUI style of design and development is limited to a small number of experts. The Microsoft Surface MVPs are a subset of these experts that have extensive real-world experience with Microsoft Surface and other NUI devices.This session is a panel featuring the Microsoft Surface MVPs and an unfiltered discussion with each other and the audience about the state of the art in NUI design and development. We will share our experiences and ideas, discuss what we think NUI will look like in the near future, and back up our statements with cutting-edge demonstrations prepared by the panelists involving combinations of Microsoft Surface 2.0, Kinect, and Windows Phone 7.

[Vote for this session here!]

I also want to highlight my InfoStrat colleague Josh Wall’s entry, which should be very interesting.

Microsoft Surface at the Smithsonian – 30 seconds to MagicJosh Wall

How do you design a Microsoft Surface application that creates a magical experience in less than thirty seconds, uses real-world physical objects, and is both fun and educational for kids? That is quite a task, but oh yeah – you need to build seven of these magical applications that are distinct yet connected by a common theme. This was the challenge presented to InfoStrat, a Surface Strategic Partner in Washington DC, by the Smithsonian Institution. This session will explore the design and development process that went into building the innovative physical object interactions for a suite of Microsoft Surface applications used in the “Wonder of Light: Touch and Learn!” exhibit at the Smithsonian Institution in Washington, DC. Josh Wall, director of the InfoStrat Advanced Technology Group, will demonstrate and analyze five different types of physical object interactions that take advantage of the unique vision system in Microsoft Surface. He will also discuss lessons learned from the experience, including designing Surface applications for kids, when to use a linear or non-linear task path, and how to apply Natural User Interface (NUI) concepts to interactive museum exhibits.

[Vote for this session here!]

Here are the other NUI sessions:

Microsoft Surface v2 – designing for the new form factor
Josh Santangelo

Building Really Social Software
Neil Roodyn

Wave, Touch, Pen, Speech, Mouse and Keyboard
Neil Roodyn

How to build a great Microsoft Surface application
Neil Roodyn

How NUI can help to perform complex tasks, like flying a helicopter
Bart Roozendaal

Understanding the Metro design language – from Zune to mobile to Microsoft Surface and beyond
Nathan Moody

[Update 1/27 9:40am: I overlooked the following session! Sorry Davide!]

Silverlight, Windows Phone 7, Multi-touch and Natural User Interfaces
Davide Zordan

There are also many other interesting topics, but I hope you’ll help vote in the NUI and Surface sessions. The MIX content team takes into account the popularity of various topics. (For example, last year they accepted 12 Open Call sessions when they were originally planning only 10.)

Voting is open until midnight PST on February 4th, so lock in your votes now!

P.S. Here is a really nifty pivot viewer for visualizing and filtering the Open Call entries: http://www.xpert360.net/MIX11/

Natural User Interfaces in .NET book update

Today Manning pushed out a new update to my book’s MEAP (early access Ebook). This is the first update in quite a while, and that’s because I’ve been secretly updating chapters and writing new chapters that include NDA content.

Note: please read through to the end of this post for a 50% off coupon for my book good until February 20, 2011.

The secret NDA content is about the future of the Surface platform and Surface SDK. Now that Surface 2.0 has been announced, I can start talking about it and pushing out the non-NDA content. You’ll notice in this update an updated table of contents and a new title, Natural User Interfaces in .NET, to reflect the reorganized content and focus. Here is the new cover:

Based upon reader and reviewer feedback and the announcement of Surface 2.0 and Kinect, we decided to focus the book on the WPF 4 platform with Surface SDK 2.0. The combination of WPF 4 and Surface SDK 2.0 makes the most advanced NUI platform available that targets a wide variety of devices, form factors, and NUI input technologies. Surface SDK 2.0 is the successor to both Surface Toolkit and Surface SDK 1.0 and will run on any Windows 7 machine, including desktops, tablets, slates, and Surface 2.0 units. 

I’ve also decided to include Kinect content because I have determined that we can use the same WPF 4 and Surface SDK 2.0 platform to target depth sensing camera interfaces as well. In early experiments, I have injected hand tracking and hand pose information into the WPF 4 touch stack and can control any WPF 4 touch interface with my hands. Using this unified NUI platform, you can write a single interface that users can use with all of the Windows 7 multi-touch devices, Surface 2.0, as well as Kinect and similar depth sensors.

Of course, you are going to want to design the interface differently based upon which input device your application is using. For example, Surface lets you detect tagged objects and finger orientation and hand tracking depth sensors can give you extra information on hand poses and gestures. That’s why I’ve organized the book into the three parts:

Part 1: Introducing NUI Concepts
What developers need to know about NUI design

Part 2: Learning WPF Touch and Surface SDK
A unified NUI platform

Part 3: Building great experiences
How to take advantage of the unique capabilities of your input devices

I still have a few more chapters to complete, but the content complete milestone is in sight. I’ve been getting great feedback from my MEAP subscribers and continue to make improvements based upon feedback. (Anyone who pre-orders the eBook or print book is also subscribed to MEAP updates and can read the completed chapters as they become ready.)

If you haven’t pre-ordered my book yet, you can buy the eBook or print book (and get the MEAP for free) for 50% off until February 20, 2011. All you need to do is enter in coupon code multinet50 during checkout. If you buy it directly after clicking this link, Manning will give me a couple extra bucks and I would appreciate it. You can also download a free sample chapter and the sample code from that link.

OpenKinect: Creating next-generation interfaces, Jan 20, Arlington, VA

This Thursday, January 20 at 7pm I'll be presenting at the CapArea.NET Silverlight SIG in Arlington, VA. If you're in the area, please come by.

Navy League Building
First Floor Conference Room
2300 Wilson Blvd
Arlington, VA
[Bing Maps | Google Maps] and is Metro accessible (the Court House station).

Free street parking after 6pm.

Presentation Title:
OpenKinect: Creating next-generation interfaces

Presentation Description:
Kinect is awesome. Kinect on your PC is even more awesome. An international community of developers creating free, open source drivers and software for using Kinect with your PC is the most awesomest of all. That is what the OpenKinect Community is about. In this session, Joshua Blake, founder of the OpenKinect Community, will tell the story of how the community worked together to open up Kinect, demonstrate Kinect working on a PC and some videos of what other people have created so far, and show you how you can start programming with Kinect and create next-generation natural user interfaces.

Kinect hand tracking with WPF 4 and Bing Maps 3-D

I’ve been playing around with Kinect and PrimeSensor in WPF for a bit and have achieved a small technical success that I wanted to share in a new video from InfoStrat. Using the same techniques that allow us to use WPF 4 on Surface 1.0, we can now use depth camera hand tracking to control multi-touch applications.

Here is a very rough proof-of-concept where I’m controlling the InfoStrat.VE WPF 4 multi-touch control using a depth camera.

Bing Maps controlled with Kinect 3D-sensing technology

This is just a multi-touch application and I have added my one line of code to enable hand tracking to feed the WPF 4 touch stack.  I also display outlines of the tracked hands to provide better feedback about what is going on. In this video I also used OpenNI and NITE from PrimeSense.

The tracked hands can participate in all of the multi-touch manipulations and gestures that you’ve already written for your touch application. You can even interact using hand tracking and touch at the same time in the same window. The code that enables this is part of our internal InfoStrat.MotionFx project and will eventually be open sourced, but it needs a bit more work to be practical.

One enhancement we’re planning is adding hand pose extraction, including palm orientation and finger positions. This would allow you can use hand poses and hand gestures to control whether a hand is “touching” the screen or not instead of the current technique. Currently, it determines that a hand is “touching” if it is over a certain distance from the shoulder. Knowing the hand pose would also enable new types of interactions, just like finger orientation, blob size, and tagged objects on Surface enables new interactions.

In the end, we will want to design interfaces that use motion tracking to take advantage of the unique capabilities of that modality to create truly natural interactions. (Pinch-to-zoom with large arm movements is not the most natural interaction.) What I’ve shown above, though, is that it is feasible to use the WPF 4 Touch stack and Surface SDK as the unified platform for both multi-touch and motion tracking modalities.

Microsoft Surface 2.0 Data Visualization Controls by InfoStrat

On Wednesday at CES2011, Microsoft finally announced the next version of Microsoft Surface. It takes advantage of a new technology called PixelSense where IR sensing pixels are interwoven with display pixels right in the LCD display itself. The computer specifications were updated and form factor went through a major overhaul, making Surface 2.0 one sexy table computer:

My employer, InfoStrat, is a member of the Surface Technology Adoption Program (TAP), so we’ve had access to early SDK builds and have been giving feedback to the Surface team since May 2010. (Serious feedback… at the last TAP event I won a backpack full of swag for finding and submitting the most bugs during the TAP. Don’t worry, the bugs were all addressed.)

InfoStrat and other Surface TAP members have also been working on applications using Surface SDK 2.0. If you were watching the Microsoft CES2011 keynote (skip to 44:15), then you saw one of those applications. Unfortunately there wasn’t time to feature all of the partner applications during the keynote, but that doesn’t mean that we can’t show them to you anyway!

Today InfoStrat announced that we will be open sourcing several of our WPF 4 data visualization controls that we designed for Surface SDK 2.0. This includes the DeepZoom and Pivot controls I’ve been hinting about on twitter as well as a few others. Unfortunately, we cannot publish the controls or code quite yet. We have to wait until Surface SDK 2.0 becomes available to the public.

We’ve been working on these controls for several months and so I’m very excited to share with you a video of these controls used in our Surface 2.0 application, InfoStrat Show for Microsoft Surface. This application is designed to take advantage of the unique capabilities of Microsoft Surface to enable natural and engaging presentations and small group collaboration. This video is just a sneak peak at the controls. We’ll be sharing more as we get closer to the code release. Below the video is InfoStrat’s press release about the controls.

Microsoft Surface 2.0 Data Visualization Controls by InfoStrat. Also shown are the updated visual styles of the Surface SDK 2.0 controls.

For Immediate Release

9 a.m. PST

January 6, 2010

InfoStrat Releases Next-Generation Data Visualization Controls for Microsoft Surface 2.0

Washington DC – January 6, 2010 – InfoStrat today announced plans to support Microsoft Surface 2.0 by releasing a control suite that accelerates the development of next-generation multi-touch data visualizations. The controls will be made available as open source software at no charge on CodePlex.com in the first half of 2011.

This data visualization control suite provides multi-touch versions of the following controls:

§ Deep Zoom multi-resolution image control that allows high performance display of very high-resolution imagery

§ PowerPoint Viewer which enables slide decks to be arranged and presented using multi-touch

§ Pivot Viewer chart control that allows dynamic sorting and categorization of data

§ Physics Canvas which provides an infinite, dynamic canvas for viewing and organizing content

Other features of the controls:

§ Works on both Microsoft Surface and Microsoft Windows 7 with touch

§ A single application built with the data visualization framework can support multiple hardware form factors including: horizontal multi-touch tables, tablets, and large format vertical touch screens

§ Innovative object recognition to enable rapid data manipulations (only on Microsoft Surface)

Watch a sneak preview of the control suite on YouTube: http://www.youtube.com/watch?v=lEVtjHlrf4I

InfoStrat is a member of Microsoft’s Technology Adoption Program (TAP) for Microsoft Surface. As a Microsoft Surface 2.0 TAP member, InfoStrat receives early access to hardware and software, allowing InfoStrat to gain expertise and influence the development of the product before it was released to the public.

In 2008, InfoStrat solved the problem of using Bing Maps 3D on Microsoft Surface in a way that performed well and was WPF-friendly. InfoStrat open-sourced the solution as a reusable control for the WPF and Surface community. Since then, the control has received over 120,000 page views and has over 8200 downloads, and has also been featured in many of our own applications. This control, known as InfoStrat.VE, has become one of the most popular controls for building mapping applications on Microsoft Surface: http://bingmapswpf.codeplex.com

“We are proud to be part of the Microsoft Surface development community,” according to Jim Townsend, president of InfoStrat, “and excited about the possibilities of Microsoft’s new version of Surface.”

Microsoft Surface provides a new way to experience and use information and digital content, engaging the senses, improving collaboration and empowering people to interact. Microsoft Surface is at the forefront of developing software and hardware that uses vision-based technology to fundamentally change the way people use computing devices. More information can be found at http://www.surface.com.

Information Strategies ("InfoStrat") is an award-winning Microsoft Gold Certified Partner and a Microsoft Surface Strategic Partner and member of the Technology Adopter Program.

For more information, press only:

Josh Wall, InfoStrat, (202) 364-8822 ext. 202, joshw@infostrat.com