Thursday, September 15, 2011

Windows 8 on Microsoft Surface 1.0 (video)

I've been neglecting this blog a bit since I'm trying to focus on finishing my book, but I had to break my self-imposed radio silence to break this news:

I installed Windows 8 on Microsoft Surface 1.0 hardware, and it works! By "it works", I mean the Microsoft Surface 1.0 shell and applications run as expected on Windows 8, and immersive-mode applications and regular desktop applications can be controlled using the Surface vision system touch input.

Windows 8 Start screen working on Microsoft Surface 1.0
I tweeted the above photo while video taping proof of this technical hackery. The video is embedded below:

Yes, this is the Windows 8 that was released publicly only a little over a day ago. I did not have early access, so this is another 24 hour hack for me like Kinductor from the Kinect Code Camp, though it didn't take me a full 24 hours of straight effort.

How did I do it?
Since Surface 1.0 is intended to run on Vista, the fact that any of this works is a testament to the backwards compatibility story of Windows 8. In short, I shrunk the Surface's Vista partition then installed Win 8 from USB key to the new partition (make it 40 GB minimum if you want to develop on it). The installer automatically set up dual booting, so I can boot back to Vista for the "correct" Surface 1.0 environment if I like.

The tricky part was getting the Surface SDK (full version that comes on Surface, not the workstation edition) to install (edit the msi launch conditions using Orca) and then getting the inputs from the Surface vision inputs to drive the Windows 8 UIs. To do that I used MultiTouchVista, which has a component that can connect to SurfaceInput.exe as a client as well as push touch data to an HID touch input driver. Windows 8 doesn't know it's not a capacitive touchscreen at all. (This technique also works for combining Windows 7 with Surface.) Win8 UI also won't be aware of tags or blobs but if you want to write a Surface application that uses those, just use the Surface SDK! Some other time I'll post a deep technical walkthrough, but this thread should be sufficient to get you started.

All of the Surface hardware drivers and Surface applications run great. The immersive mode touch responsiveness is a bit flaky as you can see in the video, but this could have been any number of things from the MultiTouchVista plug-ins to the Win8 touch stack to the Start screen. The Win8 touch visualizations (trails and auras) were always working fine, so I'm leaning towards the problem being in the Win8 UI layer. This is a pre-beta build though, so entirely understandable.

I do need to say that this configuration is completely unsupported and would probably violate the warranty if it wasn't already expired for this particular machine. It also has nothing to do with Surface 2.0. You won't ever see Microsoft create a configuration like this for general use since running three shells (Surface, Immersive, and Desktop) side-by-side would be entirely confusing for real users. In the video I was basically in the equivalent of Surface administrator mode, manually running SurfaceShell.exe, and then switching ad-hoc into immersive mode, which is like a full screen start menu. There are too many competing metaphors and modes for this configuration to be used by the general public.

Does Immersive mode make the Surface shell obsolete?
No way. It makes the Surface shell even more important. My initial impressions with actually using the Win8 immersive mode on Surface is that it really doesn't work for a large, horizontal, multi-user form factor. For one, everything is oriented in one direction, and two, I need to reach too far to get to certain interface elements across the table. That's the opposite of the ergonomic intent of the immersive mode, which was design for smaller screens and slates. It might do better on a vertical wall-mounted form factor. There is definitely a place here for both Immersive mode as well as the Surface shell since they are each designed to do different things in different scenarios. This is especially true if we account for the improved Surface shell that is coming soon with Surface 2.0 hardware.

I think we'll see that there will is a large difference between the design of an immersive mode application to run on a Slate or PC and used by one person, and the design of Microsoft Surface applications to be used by multiple people in a 360 degree interface.

What Surface and Win8 can learn from each other
Even though the designs for each of these modalities need to be different, there is a lot that they can learn from each other. I really like swiping in from the edge to open the charms and app bars, and could see that being incorporated into the version of Surface after 2.0. Windows 8 isn't perfect though, since as soon as there is an extra finger somewhere in immersive mode, the whole interaction breaks down.

It may just be the developer bits, but many or most of the Win8 sample games were almost unusable as a Surface application. My 3 year old tried to play a few with me but we kept getting in each others way since the games were designed for a single orientation. Worse than that, my touches would block her from interacting with the game, and vice versa. Some of the interns that developed those samples apparently did not account for multiple types of input (mouse and touch) or even multi-touch at all, indicating they were still thinking about coding interactions from a single-point mouse point-of-view.

I know that on slates and PCs with a single user they may not need to plan for a full 360 degree interface, but Windows 8 developers need to learn the lessons that Microsoft Surface developers have learned years ago about allowing multiple inputs, multiple touches, and letting the user recover from mental mistakes and accidental touches.

Watching the Windows 8 unveiling during the Build Day 1 keynote was somewhat amazing for me because all of a sudden NUI design and development skills (my focus since 2008) has gone from a small niche to a core requirement for the Windows platform. Anyone interested in building truly great Windows 8 applications needs take an active interest in learning from Surface developers. I'm expecting that some of my colleagues in the Surface community will end up coming up with the most amazing Windows 8 applications in no time at all. (Feel free to send those apps to me for testing on my FrankenSurface Window 8 machine!)

Natural User Interfaces in .NET
While at first I was nervous that the Windows 8 announcements would make my almost-complete book a waste, now that I think about it things are still pretty good. If you want to use the premier technology for creating Natural User Intefaces, then .NET and WPF is still the way to go. You can't target Surface or Kinect in immersive mode, and the immersive mode interface isn't well suited for multiple people. You also have to consider that Windows 8 is still six months to a year away, but you can develop NUIs for Windows 7 multitouch, Surface, and Kinect today with WPF. You can even give them a metro look-and-feel.

If this interests you and you haven't already, take a look at the free sample chapter of my book, and if you like it pre-order it and get the early access PDF today with all of the completed chapters.