Sharing binaries between Surface and Windows 7

This post is part of a series about Blake.NUI, my open-source library for multi-touch natural user interface development with WPF 4 and Surface. In this post I discuss how to set up your projects to share 99% of your binaries between Surface and Windows 7. See my Blake.NUI announcement post for an overview of the project as well as a discount code for my book Multitouch on Windows, good until July 23.

In my last post, I showed how to use the SurfaceTouchDevice class to let WPF 4 Touch applications work on Microsoft Surface. In this post, I will discuss how to set up your project to allow you to reuse almost your entire WPF 4 code-base between a Surface SDK application and Win7 Surface Toolkit application. This approach will make your code binary compatible between the two platforms.

Note that as I discuss the technique, I am assuming that you are setting up your projects on a Windows 7 multi-touch machine with Visual Studio 2010 and both Surface SDK SP1 Workstation Edition and Surface Toolkit beta installed. Once the projects are setup, you can compile the solution on a Surface device as long as you create a solution configuration that does not build the projects that reference Surface Toolkit.

1. Project Structure

If you want to target both Win7 and Surface with the same code-base, you'll need to keep the Surface SDK-specific code separate from your reusable code. There are several possible approaches, but here is the one I have been using. Figure 1 shows the architecture of the solution.

Figure 1 Most of the controls and code is located in a common project that is consumed by a Surface project and a Win7 project.

I recommend creating a shell project for Surface and a shell project for Win7. Each of these projects has only a window. In the Surface project, add SurfaceTouchDevice.RegisterEvents(this) to the SurfaceWindow constructor. (I also add MouseTouchDevice.RegisterEvents(this) to both Surface and Win7 projects.) Then in a separate common library I put all of my MVVM code and added the same view user control to the window in each of the shell projects.

Note that the Surface project references the Surface SDK (v1.0) and the Win7 project references Surface Toolkit (v1.5). You will need to check the properties on the assembly references to ensure they are the correct version for each application. The Surface project probably is referencing v1.5 of Microsoft.Surface.Presentation.dll and Microsoft.Surface.Presentation.Generic.dll. Simply remove those reference and in the Add Reference dialog browse and add references to the 1.0 version of the assemblies.

Surface Toolkit contains a subset of controls from the full Surface SDK. This means that you can reference either version 1.0 or 1.5 of Microsoft.Surface.Presentation.dll in your common project and use any of the common controls that exist in both assemblies, such as SurfaceButton and ScatterView. As long as you stick to the subset of common controls, the same binary will work on both Surface and Win7. I would recommend using v1.0 in the common project so you can still compile the project on your Surface device without Surface Toolkit.

2. Loading the correct assemblies

Assuming that you do reference Microsoft.Surface.Presentation.dll v1.0 in the common project and Microsoft.Surface.Presentation.dll v1.5 in the Win7 project, you need to do one more thing to ensure that the application will load correctly. If you do not, then depending upon the order in which components load, you may get a runtime exception like the one shown in figure 2.

Figure 2. If you mix the assembly references, you may get an IOException: Cannot locate resource 'controls/touchvisualizations/touchvisualizationsresources/trailtop.png'. This exception occured on the first touch after the Win7 application loaded.

The solution to this is to add a versioning policy to redirect the v1.0 assembly to v1.5. This blog has a good overview of versioning policies. In the Win7 project, add this to the app.config within the configuration section:

<runtime>

  <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

    <dependentAssembly>

      <assemblyIdentity name="Microsoft.Surface.Presentation"  

          publicKeyToken="31BF3856AD364E35" culture="neutral"/>

      <bindingRedirect oldVersion="0.0.0.0-1.5.0.0"  

          newVersion="1.5.0.0"/>

    </dependentAssembly>

  </assemblyBinding>

</runtime>

Just to be safe, you may want to also add the same thing to the Surface project's app.config, except change the newVersion to 1.0.0.0. This ensures that each project will always load the correct version of the assembly, provided it is installed.

I have committed a few updates to the Blake.NUI project, including TestApplication.Common, TestApplication.Surface, and TestApplication.Win7. These three projects illustrate the ideas discussed in this blog. Download the latest code from the Source Code tab on the CodePlex site.

3. Summary

We have learned how to structure a WPF 4 solution to target both Win7 and Surface while reusing almost all of our code in a common library. We also saw how to configure the projects to ensure they load the correct version of the Surface assemblies at runtime.

So far, we know that we can only reference the Surface controls that exist in both Surface SDK and Surface Toolkit. In the next post, I will discuss some techniques that can enable you to use the Surface SDK-specific controls such as TagVisualizer and ElementMenu while maximizing the code reuse.

How to write WPF 4 Surface applications with Blake.NUI

This post is part of a series about Blake.NUI, my open-source library for multi-touch natural user interface development with WPF 4 and Surface. In this post I discuss the origin of SurfaceTouchDevice and MouseTouchDevice and how to use them in your application. See my Blake.NUI announcement post for details as well as a discount code for my book Multitouch on Windows, good until July 23.

Between InfoStrat projects and writing my book, I've been neck-deep developing WPF 4 Touch and Surface applications for months. If you have used them both then you'll know that the WPF 4 Touch API is a little bit easier to develop for than the Surface SDK it is based upon. This is because WPF 4 touch and manipulation features are integrated directly into the framework. There are also a few other subtle differences, like not needing a separate ManipulationProcessor and InertiaProcessor.

On my current project, I needed to create a Surface and a Win7 version of the same application. I decided that I wanted to use .NET 4 on Surface. This would let me take advantage of the new animation easing functions in .NET 4 and it would let me reuse more code between both versions of the application. First problem: can I install Visual Studio 2010 and .NET 4 on Surface? I tweeted @Surface and Eric Havir replied. Figure 1 shows this twitter conversation.

Figure 1. A twitter conversation between me and Eric Havir from @Surface where he confirms .NET 4 would probably install but could not be used for Surface apps.

That is pretty much what I expected. It's been known since PDC09 that the roadmap for the future of Surface includes the WPF 4 Touch API on Windows 7, as shown in figure 2. This sounds like a great unified platform, but it doesn't help me now. The currently available Surface SDK was designed for .NET 3.5 SP1 and uses a custom Surface Input mechanism that doesn't feed into the WPF 4 touch system.

Figure 2. A slide from the PDC09 Multi-touch on Microsoft Surface and Windows 7 session.

I went ahead and installed Visual Studio 2010 on my Surface anyway because I wanted to use the easing functions. It turns out that Surface applications compile just fine under .NET 4 and even the SDK works, both in the emulator and on a Surface device. I figured I would just create some common controls and branch most of my project to target either the WPF 4 Touch with Surface Toolkit or the Surface SDK.

But wait! WPF 4 uses TouchDevices. TouchDevices are extensible. Anson Tsao, a Principal Program Manager who was responsible for the WPF 4 Touch API, wrote about creating custom touch devices in January.

So I'm thinking maybe if I create a custom touch device that listens to the Surface Contact events, I could process those events into WPF 4 TouchDevices and not even write against the Surface SDK.

Well once I had the idea in my head, I had to do it. I created a class, SurfaceTouchDevice, that derives from TouchDevice and promotes Surface SDK Contact events to WPF 4 TouchDevice events. This means that if you install .NET 4 on your Surface and write against the WPF 4 Touch API, It Just Works™.

Here's the best part. All you need to do to enable this dark magic is add this line of code to the constructor of your SurfaceWindow:

SurfaceTouchDevice.RegisterEvents(this);

That's it. Now the same code that works with WPF 4 Touch will also work on Surface. This also means you can develop with Visual Studio 2010. The SurfaceTouchDevice class will not work on a Win7 machine, though, since the Contacts class is not present in the Surface Toolkit.

In most cases you won't need to do anything else. Surface SDK controls, such as TagVisualizer, will react to and handle Contact events. Unhandled events will be promoted to WPF 4 Touch events by the SurfaceTouchDevice class, and then your code will react to the WPF 4 Touch events.

If you do want to check an actual Surface property, such as whether the TouchDevice represents a tag or a blob, you can get access to the original Contact by casting the TouchDevice to a SurfaceTouchDevice:

   1:  private void TouchDown(object sender, TouchEventArgs e)

   2:  {

   3:      bool isFinger = true;

   4:   

   5:      SurfaceTouchDevice device = e.TouchDevice as SurfaceTouchDevice;

   6:      if (device != null)

   7:      {

   8:          isFinger = device.Contact.IsFingerRecognized;

   9:      }

  10:      //Do something 

  11:  }

You can also filter out touches from your manipulations by iterating through e.Manipulations, casting each IManipulator to SurfaceTouchDevice, and calling Manipulation.RemoveManipulator for undesired IManipulators. I will probably add this feature to SurfaceTouchDevice in an update to Blake.NUI.

I also created a MouseTouchDevice class that operates similarly except promotes mouse input to touch input. This is helpful so I do not need to create MouseDown/Move/Up events when I'm creating a custom touch control with TouchDown/Move/Up.  It also means the mouse can participate in the manipulation processor.

Both SurfaceTouchDevice and MouseTouchDevice are available, free and open-source, in Blake.NUI, my library of helpful multi-touch NUI code for WPF 4 and Surface.

Now that SurfaceTouchDevice is available, I cannot think of a reason why you would not want to install Visual Studio 2010 on your Surface and write all applications with WPF 4. In my next post, I'm going to talk about how to set up your application architecture so you can share 99% of your code with no changes or branches for Microsoft Surface and Win7.

Announcing Blake.NUI - a multi-touch NUI library

Almost all my free time and creative efforts recently have been focused on my book, but I've been able to put together something really cool that I think will be useful for anyone doing multi-touch natural user interface work with WPF. Actually, it's a couple really cool things packaged into one.

I'm happy to announce Blake.NUI, an open-source library for multi-touch NUI development. Blake.NUI is available under the MS-PL license at CodePlex:

http://blakenui.codeplex.com

It is a collection of the most useful reusable code that I have created for my own projects and for my book's code samples. It currently includes code for WPF 4 and Surface, although in the future I may add Silverlight code as well.

The library is still in undergoing changes. I may add or rearrange things, but I will document changes from version to version. Here are some of the things currently included in Blake.NUI:

• Controls
• ZoomCanvas - an infinite canvas control that is the heart of NaturalShow
• CircleText - layout text along the circumference of a circle
• Gestures
• TapGestureTrigger - a configurable Blend Trigger that fires when the user completes a tap gesture
• DoubleTapGestureTrigger - a configurable Blend Trigger that fires when the user completes a double tap gesture
• TouchDevices
• MouseTouchDevice - promotes mouse input to WPF 4 touch events
• SurfaceTouchDevice - promotes Microsoft Surface vision system input to WPF 4 touch events
• Common classes and utilities
• DisplayMatrix
• AnimateUtility and SurfaceAnimateUtility
• ScatterMatrixHelper
• MathUtility
• VisualUtility
• NaturalShow
• Common models, views, factories, and other classes

The full NaturalShow application is not yet included, but many of the core classes are. I'm updating NaturalShow to use Blake.NUI rather than its older, internal versions of the same code.

I'm particularly excited about SurfaceTouchDevice, since with one line of code it enables you to write multi-touch applications against the WPF 4 Touch API, including the integrated touch events and manipulations, and run it on your existing Microsoft Surface devices without modification. This also means you can, with a simple project structure, target both Win7 Touch and Surface v1 with 99% of the same code base! You do have to install .NET 4 and/or Visual Studio 2010 on your Surface.

I will be celebrating the launch of Blake.NUI all this wee by posting a series of blog posts about how to use the library. In addition, you can buy the ebook edition of Multitouch on Windows until July 23rd for 35% off through this link using coupon code blake35bl. (Disclosure: if you buy after clicking this affiliate link it gives me a few extra dollars from the sale.)

I am asking as a courtesy that if you find Blake.NUI useful, particularly for commercial projects, that you buy a copy of my book, Multitouch on Windows. The coupon code provided should help. 

Please let me know what you think about Blake.NUI and whether you think you'll use it in your applications.