Samsung SUR40 for Microsoft Surface Receives Best of Innovations Award at CES

The Samsung SUR40 for Microsoft Surface (the Surface 2.0), which was announced at last year’s CES, has been awarded a “Best of Innovations 2012” award (http://blogs.msdn.com/b/surface/archive/2012/01/13/ces-award.aspx).

Compared to the Surface 1 unit we have in T4G’s Moncton office, this is a great step forward. The Surface is a computer vision based system for doing multitouch systems. In Surface 1.0 this meant that the solution relied on a DLP projector, and a set of cameras to detect touches by fingers or objects. This made the Surface 1 large, heavy (around 200 lbs), and limited deployment options (for example, the Surface had to be horizontal.

Surface 2 is still a computer vision based system, but uses a new technology called “PixelSense”, in which there is an IR sensor attached to every pixel. This allows the device to be much thinner than the original (about 4 inches), and weigh less than half as much. It also allows it to be deployed horizontally, vertically, or anywhere in between.

Other specifications have also been greatly improved. The Surface 2 is a FullHD 40 inch LCD, compared to the original unit’s 32 inch, 1024×768 DLP projection. The new Surface also has considerably more processing, video and memory capacity than the original (as it should – the original’s specs were from 2007!)

It is also made of Gorilla glass, making it say to deploy in “uncontrolled” environments. When it was announce last January, it was the largest piece of Gorilla Glass ever produced, but at CES 2012, Perceptive Pixel demonstrated an 82 inch touch display made of Gorilla Glass).

Also improved is the development model. While the original was programmed in .NET using either WPF or XNA, it extended those frameworks in a way very specific to the Surface. In the Surface 2.0 SDK, it builds upon the touch support designed into .NET 4.0, and allows applications to be built to run on either the Surface or Windows 7 touch devices with minimal code changes.

Bing Maps WPF Control Beta with Surface 1.0

Microsoft surface table
Image via Wikipedia

As announced on the Microsoft Surface Blog (and elsewhere, I am sure), a Bing Maps WPF Control Beta has been released. This control targets WPF4 with functionality similar to the Bing Maps Silverlight control.

As a WPF4 control, with support for touch, it will also work in the Surface 2.0 environment. For those of us developing on the Surface 1.0 hardware, which of course runs over .NET 3.5, I wanted to see if I could make it work there.

Some time ago, Josh Blake wrote a post on how to write WPF4 applications for Surface 1.0 using his Blake.NUI library.

Using that same approach, I was able to build a simple application using the new Bing Maps Control under WPF4 on my Surface 1.0 hardware. On the downside, the app seems a little jumpy when panning, but I may be able to clean that up with some more work This is a start, though.

Microsoft: Get Your Shit Together on Touch Development

Testing Samples from Microsoft Surface Toolkit...

Image by John Bristowe via Flickr

I have been playing with multi-touch development for a while, both on Windows 7 with my 2740p and on the Microsoft Surface table (version 1, not the new one).

I have consistently had challenges using WPF4 multi-touch events on the 2740p, or using the Microsoft Surface Toolkit for Windows Touch Beta, and now with the newly released Surface 2 SDK.

With any of these tools, I have challenges getting the software to recognize touch events, and even more trouble getting the software to recognize 2 simultaneous touch points (the 2740p supports 2 touch points). A second touch point always cancels the first touch point.

What is funny (to me, anyway) is that the samples included with the machine in the Microsoft Touch Pack for Windows 7 work just fine on the 2740p. This indicates that it is something in the managed drivers used with WPF that is not working.

I am fairly certain that this is a driver issue. I had it working at one point after a lot of hacking and installing drivers different from the default updates. I guess an update somewhere (Windows Update, or HP Tools) has overwritten the driver I had setup to make it work.

Can I fix the drivers to make this work again? Absolutely! But that is not the point here.

This should just work!

How am I as a software developer, or as an ISV, supposed to recommend this platform to my customers, or build applications for it, when even getting it to run on different machines requires significant hacking of drivers?

If Microsoft wants to stop being seen as a joke in the multi-touch and/or tablet market, they really better find a way to get their shit together on this.

Bill Buxton: “NUI – What’s in a name?”

Recently (early October) Bill Buxton gave another talk nominally about Natural User Interfaces. For those who don’t know, Bill is Principal Researcher at Microsoft Research and has a 30 year involvement in research, design and commentary around human aspects of technology, and digital tools for creative endeavour, including music, film and industrial design (and a lot of other things, but I am not going to copy and paste his whole bio!).

The presentation, given at Microsoft Development Center Copenhagen, covers a lot more than just current ideas around NUIs. It looks back at the history of efforts to develop natural and touch user interfaces going back to the early 70s, as well as looking at what exactly we are trying to accomplish with these UI paradigms, what natural really means in a UI, and what makes good design in general.

While I highly recommend taking the time to watch the entire video, here are a few points I found really interesting:

  • The “Long Nose”: the concept of the “Long Tail” turned around, indicating that technologies (even successful ones) have a very long lifetime before they get on anyone’s radar, and in fact are usually in existence for about 20 years before they become major industries. This interesting implication of this, is that if you are looking for technologies that will be game-changers (can’t believe I used that term – I hate it) 10 years from now, you need to be looking that technologies that have been around for 10 years already.
  • Ask what your idea is worst at: Every idea is best at something and worst at something. It is just as important to be able to identify what your idea is least suited for as what it is best at.
  • You do not succeed in spite of your failures; you succeed because of your failures.
  • There is nothing all that new or revolutionary in the iPhone, iPad, Surface, or any other tablet-like devices. Most of the technology they rely upon has existed for 20-30 years or more).
  • Many people are stunned by how far technology has come (smart phones, touch interfaces, etc.), when really it is surprising how little progress has been made, given where things were in the 70s and 80s.
  • Most of us still carry around paper notebooks of some sort in order to scribble notes, sketch ideas, etc. We were getting to the point of replacing them with Tablet PCs. Unfortunately that is going away now with the current  generation of smartphones and slates, since they have done away with the stylus because marketing people have told us (so it must be true) that we do not want to take notes or make sketches.
  • The next generation of natural user interfaces need to be context aware. Not software context aware, but real context – where am I, what is the environment, what are the constraints.
  • Why the buttons on women’s clothes are all wrong!

Those are just the things I found interesting. The video is about 90 minutes long (60 minutes of presentation, 30 of Q & A), but it is well worth the time it takes to watch.

 http://channel9.msdn.com/posts/TechTalk-NUI-Whats-in-a-Name

Displaying an XPS Document on the Microsoft Surface

As a part of an application I am prototyping, I ran into the need recently to display a document inside a ScatterViewItem on the the Microsoft Surface. Since there is (intentionally) no built-in way to display HTML content on the Surface, and displaying a Word document did not seem feasible, I settled on using an XML document, and using the WPF DocumentViewer control.

This seems pretty easy, right? Just put a DocumentViewer inside your ScatterViewItem, load the document into the DocumentViewer, and you’re done. Couldn’t be easier.

Well, displaying the XML document was indeed that easy. Unfortunately, as I expected, the DocumentViewer would not respond to touch at all. I posted a question to the MSDN Surface forums about this, and did not receive any response for several weeks.   The response I finally did receive was that I would have to develop a User Control, and handle the touch events myself.

Not being one to take advice, I decided to try another approach – I decided to see if I could hack the ControlTemplate for the DocumentViewer in such a way as to have it support touch (keep in mind that I am a relative noob when it comes to all of this WPF/Styles/ControlTemplate stuff).

So I created a simple Surface project in VS2008, and modified the SurfaceWindow1.xaml file to have a ScatterView control, containing a single DocumentViewer control, as shown below:

<s:SurfaceWindow x:Class="SurfaceDocumentViewer.SurfaceWindow1"
    xmlns=http://schemas.microsoft.com/winfx/2006/xaml/presentation
    xmlns:x=http://schemas.microsoft.com/winfx/2006/xaml
    xmlns:s=http://schemas.microsoft.com/surface/2008
    Title="SurfaceDocumentViewer" Loaded="SurfaceWindow_Loaded">
  <s:SurfaceWindow.Resources>
    <ImageBrush x:Key="WindowBackground" Stretch="None" Opacity="0.6" ImageSource="pack://application:,,,/Resources/WindowBackground.jpg"/>
  </s:SurfaceWindow.Resources>
    <s:ScatterView Background="{StaticResource WindowBackground}" >
        <DocumentViewer Margin=”15” Height="Auto" Name="docViewer" Width="Auto" />
    </s:ScatterView>
</s:SurfaceWindow>

Note that I added a margin around the DocumentViewer so that there would be something to grab on to in order to resize the ScatterViewItem. 

I then opened the project in Expression Blend so that I could get a copy of the “default” style/template for the DocumentViewer control. Opening the project in Blend, right click on the DocumentViewer, and select Edit Template | Edit a Copy…, as shown

2

Name the style (I used SurfaceDocumentViewerStyle), and define a location for it (I left it in the SurfaceWindow1.xaml file), as shown below.

3

Click OK, then save your changes and close Expression Blend.

Returning to Visual Studio, you will get a message that the project has been changed outside of Visual Studio. Click Reload to load the changes into Visual Studio. Opening SurfaceWindow1.xaml in design view, you should now see a <Style> element under <s:SurfaceWindow.Resources>, and within that, a ControlTemplate for the DocumentViewer. There are several parts to the ControlTemplate:

  • A ContentControl for the toolbar, that loads from an external assembly.
  • A ScrollViewer named PART_ContentHost that is the container for the actual document display
  • A DockPanel that provides background for PART_ContentHost
  • Another ContentControl names PART_FindToolBarHost where the search box is hosted

The only part important to me was the ScrollViewer. I also wanted to get rig of the toolbar and the search box in order to keep things as clean as possible. So I deleted the other parts.

Here then is the key step to making the DocumentViewer touch aware: I replaced the ScrollViewer with s:SurfaceScrollViewer. My new ControlTemplate now looks as shown below:

<Style x:Key="SurfaceDocumentViewerStyle" BasedOn="{x:Null}" TargetType="{x:Type DocumentViewer}">
    <Setter Property="Foreground" Value="{DynamicResource {x:Static SystemColors.WindowTextBrushKey}}"/>
    <Setter Property="Background" Value="{DynamicResource {x:Static SystemColors.ControlBrushKey}}"/>
    <Setter Property="FocusVisualStyle" Value="{x:Null}"/>
    <Setter Property="ContextMenu" Value="{DynamicResource {ComponentResourceKey ResourceId=PUIDocumentViewerContextMenu, TypeInTargetAssembly={x:Type System_Windows_Documents:PresentationUIStyleResources}}}"/>
    <Setter Property="Template">
        <Setter.Value>
            <ControlTemplate TargetType="{x:Type DocumentViewer}">
                <Border Focusable="False" BorderBrush="{TemplateBinding BorderBrush}" BorderThickness="{TemplateBinding BorderThickness}" CornerRadius="5" >
                    <Grid Background="{TemplateBinding Background}" KeyboardNavigation.TabNavigation="Local">
                        <Grid.ColumnDefinitions>
                            <ColumnDefinition Width="*"/>
                        </Grid.ColumnDefinitions>
                        <Grid.RowDefinitions>
                            <RowDefinition Height="Auto"/>
                            <RowDefinition Height="*"/>
                            <RowDefinition Height="Auto"/>
                        </Grid.RowDefinitions>
                        <s:SurfaceScrollViewer HorizontalScrollBarVisibility="Hidden" VerticalScrollBarVisibility="Auto" x:Name="PART_ContentHost" IsTabStop="true" TabIndex="1" Focusable="{TemplateBinding Focusable}" Grid.Column="0" Grid.Row="1" CanContentScroll="true" />
                    </Grid>
                </Border>
            </ControlTemplate>
        </Setter.Value>
    </Setter>
</Style>

Now, to test this, add an XPS document to your project (mine is name test.xps). Add an assembly reference to ReachFramework (XPS package support), and add using statements to SurfaceWindow1.xaml.cs as shown:


using System.IO;
using System.Windows.Xps.Packaging;

Add the following code to the end of the SurfaceWindow1 constructor to load and display the XPS document:


XpsDocument doc = new XpsDocument("test.xps", FileAccess.Read);
docViewer.Document = doc.GetFixedDocumentSequence();
docViewer.FitToWidth();
doc.Close();

Finally, to make the XPS document resize when you resize the ScatterViewItem, Add an event handler to your DocumentViewer to handle the SizeChanged event, as shown below:


private void docViewer_SizeChanged(object sender, SizeChangedEventArgs e)
{
    docViewer.FitToWidth();
}

If everything went according to plan, you should now be able to run your code and you should get a ScatterViewItem displaying your XPS file which is resizable, and which supports touch to navigate around the document.

(I think this should also work with the Surface Toolkit for Windows Touch, but I haven’t tried it yet)