How to write WPF 4 Surface applications with Blake.NUI

This post is part of a series about Blake.NUI, my open-source library for multi-touch natural user interface development with WPF 4 and Surface. In this post I discuss the origin of SurfaceTouchDevice and MouseTouchDevice and how to use them in your application. See my Blake.NUI announcement post for details as well as a discount code for my book Multitouch on Windows, good until July 23.

Between InfoStrat projects and writing my book, I've been neck-deep developing WPF 4 Touch and Surface applications for months. If you have used them both then you'll know that the WPF 4 Touch API is a little bit easier to develop for than the Surface SDK it is based upon. This is because WPF 4 touch and manipulation features are integrated directly into the framework. There are also a few other subtle differences, like not needing a separate ManipulationProcessor and InertiaProcessor.

On my current project, I needed to create a Surface and a Win7 version of the same application. I decided that I wanted to use .NET 4 on Surface. This would let me take advantage of the new animation easing functions in .NET 4 and it would let me reuse more code between both versions of the application. First problem: can I install Visual Studio 2010 and .NET 4 on Surface? I tweeted @Surface and Eric Havir replied. Figure 1 shows this twitter conversation.

Figure 1. A twitter conversation between me and Eric Havir from @Surface where he confirms .NET 4 would probably install but could not be used for Surface apps.

That is pretty much what I expected. It's been known since PDC09 that the roadmap for the future of Surface includes the WPF 4 Touch API on Windows 7, as shown in figure 2. This sounds like a great unified platform, but it doesn't help me now. The currently available Surface SDK was designed for .NET 3.5 SP1 and uses a custom Surface Input mechanism that doesn't feed into the WPF 4 touch system.

Figure 2. A slide from the PDC09 Multi-touch on Microsoft Surface and Windows 7 session.

I went ahead and installed Visual Studio 2010 on my Surface anyway because I wanted to use the easing functions. It turns out that Surface applications compile just fine under .NET 4 and even the SDK works, both in the emulator and on a Surface device. I figured I would just create some common controls and branch most of my project to target either the WPF 4 Touch with Surface Toolkit or the Surface SDK.

But wait! WPF 4 uses TouchDevices. TouchDevices are extensible. Anson Tsao, a Principal Program Manager who was responsible for the WPF 4 Touch API, wrote about creating custom touch devices in January.

So I'm thinking maybe if I create a custom touch device that listens to the Surface Contact events, I could process those events into WPF 4 TouchDevices and not even write against the Surface SDK.

Well once I had the idea in my head, I had to do it. I created a class, SurfaceTouchDevice, that derives from TouchDevice and promotes Surface SDK Contact events to WPF 4 TouchDevice events. This means that if you install .NET 4 on your Surface and write against the WPF 4 Touch API, It Just Works™.

Here's the best part. All you need to do to enable this dark magic is add this line of code to the constructor of your SurfaceWindow:

SurfaceTouchDevice.RegisterEvents(this);

That's it. Now the same code that works with WPF 4 Touch will also work on Surface. This also means you can develop with Visual Studio 2010. The SurfaceTouchDevice class will not work on a Win7 machine, though, since the Contacts class is not present in the Surface Toolkit.

In most cases you won't need to do anything else. Surface SDK controls, such as TagVisualizer, will react to and handle Contact events. Unhandled events will be promoted to WPF 4 Touch events by the SurfaceTouchDevice class, and then your code will react to the WPF 4 Touch events.

If you do want to check an actual Surface property, such as whether the TouchDevice represents a tag or a blob, you can get access to the original Contact by casting the TouchDevice to a SurfaceTouchDevice:

   1:  private void TouchDown(object sender, TouchEventArgs e)

   2:  {

   3:      bool isFinger = true;

   4:   

   5:      SurfaceTouchDevice device = e.TouchDevice as SurfaceTouchDevice;

   6:      if (device != null)

   7:      {

   8:          isFinger = device.Contact.IsFingerRecognized;

   9:      }

  10:      //Do something 

  11:  }

You can also filter out touches from your manipulations by iterating through e.Manipulations, casting each IManipulator to SurfaceTouchDevice, and calling Manipulation.RemoveManipulator for undesired IManipulators. I will probably add this feature to SurfaceTouchDevice in an update to Blake.NUI.

I also created a MouseTouchDevice class that operates similarly except promotes mouse input to touch input. This is helpful so I do not need to create MouseDown/Move/Up events when I'm creating a custom touch control with TouchDown/Move/Up.  It also means the mouse can participate in the manipulation processor.

Both SurfaceTouchDevice and MouseTouchDevice are available, free and open-source, in Blake.NUI, my library of helpful multi-touch NUI code for WPF 4 and Surface.

Now that SurfaceTouchDevice is available, I cannot think of a reason why you would not want to install Visual Studio 2010 on your Surface and write all applications with WPF 4. In my next post, I'm going to talk about how to set up your application architecture so you can share 99% of your code with no changes or branches for Microsoft Surface and Win7.