First Thoughts on Microsoft Surface Development

A brand new Microsoft Surface development unit arrived this week in the Moncton T4G office. As I start to develop some prototypes, I will be doing some related posts, but I wanted to start by talking about the platform a little, and the development environment.

For anyone who has no idea what the surface is, it is a multi-user, multi-touch platform released by Microsoft a couple of years ago. Have a look at this video to see what it can do.

Other the last few weeks, before the unit arrived, I have learned quite a bit about the Surface. The first interesting thing I learned was the the surface is not a touch screen in the sense that your iPhone or multi-touch laptop are. The surface of the Surface is just glass – it is not a capacitative or pressure sensitive material at all. All of the touch behaviours and interactions are based instead on a computer vision system. Inside the box there is a fairly standard PC running Windows Vista, with an DLP projector pushing the image up to the table top. There are also 5 cameras inside the box which perform the actual "vision". These feed into a custom DSP board which analyses the camera feeds into something a little more manageable for the PC. The fact that it is a vision-based system leads to some interesting capabilities, as well as some idiosyncrasies.

When the Surface is running in user mode, the Windows Vista UI is completely suppressed. There are no menus, no windows, and no UAC dialogs – nothing that would indicate it is even running Windows. There is also an Administrator mode which shows a standard Vista UI for administrative functions or for development.   

As far as development goes, the good news is that it is all pretty standard stuff. There are two approaches to programming for the Surface. The first is to use the Microsoft XNA Studio platform, the other is to use Windows Presentation Foundation (WPF). Using XNA gives you a little bit more power, as well as access to more of the "lower level" information like raw images from the video feed. Using WPF is a higher-level programming model, and comes with a set of controls specific to the Surface UI model. The nice thing is that all you know about .NET and WPF programming applies to the surface. And from a larger architectural perspective, Surface can tie into any infrastructure accessible to any other .NET-based model. It is just a different .NET UI layer.

The bigger challenge in developing for the Surface is changing the way we think about the UI, and selecting the right solutions. First and foremost, Surface applications are not just a port of a standard Windows UI. Stop thinking about Windows, Icons, Menus and Pointers (WIMP). The surface calls for a completely different models, one that I am just learning. One of the interesting statement I have read describing the Surface model is "the content is the application."
The Surface is more than just a multi-touch platform. Sure, you could implement a multi-touch solution on the Surface exactly the same as a Windows 7 multi-touch solution, but that is only using a subset of the Surface capabilities. The key characteristics of Surface interaction are:

  • multi-user, multi-touch (up to 52 simultaneous touch points)

  • social interaction – multiple simultaneous users, collaborating or working independently

  • 360 degree user interface – users on all sides of Surface at the same time, with UI oriented to support all of them

  • Natural and immersive – like the physical world, only better

  • Support for physical objects integrated into the experience (tokens, cards, game pieces, merchandise)

When it comes to selecting a solution to deploy on the Surface, the two most important keywords are "social" and "immersive". Social, because the best Surface applications are those in which the computer is not replacing human interaction, it is enhancing it. Immersive, because you want the user(s) to forget that they are using a computer, and to only be thinking about what they want to accomplish. The how should be transparent.

Over the coming days and weeks, I will post more about the Surface and what we are doing with it. Hopefully next week I will be able to post a short video. If you have any thoughts or suggestions, I would love to hear them.

What Should a Slate Tablet Look Like Today? Revisited

A while back I wrote a post What Should a Slate Tablet Look Like Today?. I thought it would be interesting to revisit the post in light of Apple’s iPad launch this week.

Looking back at that post, here is what I listed as my primary requirements:

  1. A form factor similar to the Kindle DX (maybe a larger screen, but similar thickness);
  2. A real back-lit screen – maybe with the ability to turn off back light to conserve power. Without backlight, should look as good as Kindle;
  3. Ink input – and maybe touch;
  4. Running a full OS – a light version of Windows or a Linux distribution (I do not want just an eBook reader);
  5. Software support: Reader for PDF and Office, etc., eMail, Browser, OneNote-like app, Media support;
  6. No extras: no modem, no optical drive, minimal ports, no video out, no web cam;
  7. No HDD – maybe an SDD, maybe the OS in ROM or something;
  8. No keyboard except maybe by USB or Bluetooth;
  9. Maybe assume that most applications I need will be in Browser (cannot believe I said that lol);
  10. Price point around $500

Lets compare those requirements to the new iPad:

  1. The iPad does pretty well on this – though it appears to be quite a bit thicker than a Kindle;
  2. Meets the primary requirement here, but does not satisfy my wish to be able to turn it off and have it look like a Kindle screen (not sure this is even possible);
  3. Ok – fails on this one completely
  4. I am not sure if I consider what is on there a “full OS” – seems pretty limited to me, and is incompatible with everything but iPhone apps;
  5. Fails on this, and even fails on many browser-based apps, given the lack of Flash support (and I am pretty sure it does not support Silverlight;
  6. Seems to be on same page as me here, though I would probably have included a USB port (Apple’s love of proprietary connections strikes again);
  7. Right on here – though the $500 model’s drive is too small to be useful;
  8. Got it right again – though I think Bluetooth would have been a good idea;
  9. As I said above, support for web apps may be limited since Apple does not seem willing to support plugins like Flash or Silverlight;
  10. Bottom end model meets this, but as I said, its 16gd SSD is too small to be useful.

So the iPad meets a lot of my requirements, but without ink support, and with its lack of support for applications I need, it is definitely not what I need as an all around slate tablet. It is a cute toy, but it is just that – a toy.

Some ‘”what if” thoughts on the Apple Tablet

Just what the world needs, right? Yet another post on the rumoured, almost legendary, Apple Tablet.

What I thought I would do here is, putting my personal feelings about Apple and its products aside, talk about what the Apple Tablet would have to be in order for it to be considered (for me) a successful tablet. I have already written about my view on what a modern tablet should look like. I have also written a couple of times on why tablets have failed to sell, and on the adoption of tablets by young people.

I could also write about the new moves by Microsoft and its partners on the tablet front. Unfortunately, there is not much to talk about there. I am not going to go down the path others have and declare the new tablets shown in the keynote to be crap – they just did not show enough to really judge one way or the other, or to be very excited.

So, lets talk about Apple.

My first question about the Apple tablet is this: What is the target market? Is it primarily a consumer product? Or is it targeted at business users? Apple has a good track record, especially in the last decade, of building products for the consumer market. There is no denying that. On the other hand, Apple’s success in the business world (outside of very specific areas like graphic design, etc.) has been limited, probably because it has not been a focus for them. In either case, the question also arises as to what is the target user group in the market? In the consumer market, is the target young people or all age groups? it the target techies or “normals”?

This is an important question, as this will define the features that are important in the product. I must admit that most of my views are driven by business use scenarios. I use my tablet for work first, and for personal use second. I am also in an older age group (in Apple’s market) – I am 47. My preferred mode of interaction is with a stylus – emulating “pen and paper”. I like to scribble notes, I like to make drawings – all part of brainstorming. As i said in a previous post, this is not necessarily how younger people prefer to interact. For the most part, my kids (all in college/university) are not all that comfortable with handwriting, much preferring a keyboard of some kind.

So what are the implications of this question for the Apple tablet? Well, if you are going to produce a device to work in the business work, the fact is (for good or bad) it pretty much has to coexist and play well with the infrastructure in place. Right now, that means it has to handle MS Office documents in a meaningful way. It has to work well with Exchange Server. It has to deal gracefully with authentication in an MS world. Like Microsoft or not, that is the world as it is now, and if you want to be a business device, you have to play nicely in that world.

If it is targeted as a consumer device, this obviously becomes much less important. The consumer market is dominated by browsing, media handling, and social interaction.

A consumer oriented tablet may also be able to succeed without an ink interface. I do not think a business device can. This has hardware implications, as well. If it is not handled properly, ink and touch do not work well together (especially for us lefties). You try to write, and your hand is touching the screen, which is interpreted as a touch and nothing works well. HP has gotten around this by have two digitizers – one for touch and one for the stylus. When the stylus gets close to the screen, the touch digitizer deactivates. THis is not the only way to handle it, but it must be handled if you are going to have both touch and ink. 

The other fundamental question about the Apple tablet (for me) is the OS. What Apple’s plans here? I see four possibilities here:

  1. A variation of the iPhone OS
  2. OSX as is (not likely)
  3. OSX tailored for the tablet
  4. A completely new OS (also not likely)

While I have very successfully and happily used windows (XP, Vista, and 7) on tablets, I have no doubt that a new approach could make things much better – face it, computer interaction has not evolved in a fundamental way since the mid 90s (maybe the mid 80s).

I am hoping that the answer is 3. I really do not want to see the Apple Tablet end up as an iPhone on steroids. I have said before that the tablet could benefit from a completely new interaction paradigm, and it will be interesting to see what Apple can do in that area. Unlike many others, I do not see the iPhone as the ultimate achievement in UI design. It is good as far as it goes, but it certainly does not scale to larger application design.

As much as I dislike Apple, I am definitely eager to see what they can do in this area.

So what is the key to success for the Apple Tablet? Obviously, anything Apple releases will have a certain amount of success within the apple fan base. But is that enough to sustain a tablet product line?

Looking back on the iPhone, when it was launched it enjoyed tremendous success because it was new, cool, and different. That would not have been enough for sustained success, however, without the massive number of applications that have been created for it, with a clean, simple distribution model and low costs. I would argue that the long term success of the iPhone is more due to the App Store than to the device itself.

Similarly, look at the Windows vs Mac battle. If ever there was an opportunity for the Mac to steal market share from Windows than during the Vista fiasco. And while Apple did gain market during this time, it really should have gained more. Why didn’t it? The biggest barrier to most Windows users migrating to Mac (especially corporate users and IT departments) is the lack of application support on the Mac. No matter how much I might like OSX and the Mac hardware, it is no good to me if the applications I need personally or or professionally are not supported.

So the key to long term success of the Apple tablet is application support. It needs to support enough of the applications and environments people use everyday, and it needs an active, dynamic developer community driving excitement like the iPhone has. This has been a fundamental point of failure for the Windows-based tablets – almost no support at all from application developers (even the ones inside Microsoft itself!).

Lets hope Apple does it right – because without some injection of new thinking, I believe the tablet device will die off.

Excellent discussion on Apple (and other) Tablets…

Check out Microsoft’s Slate: Exactly Unlike Apple’s Upcoming Tablet – though the post is self is really kind of biased (but hey, it is on a blog called theAppleBlog after all), there is some really good discussion going on in some of the coments (after the first few Apple fanboy entries). And the discussion is really what all of this Web 2.0 stuff really is all about, right?

The Wonder Of Apple’s Tablet – washingtonpost.com

The Wonder Of Apple’s Tablet – washingtonpost.com

Well, well, well….yet another “hype” article for the rumoured (though probably real in some form) Apple Tablet. I must admit, that I am of two minds on the the Apple Tablet (what ever it is will be called). On the one hand, I am very interested in seeing what Apple does with the idea. Will it be a real tablet, or will it just be a big iPhone? Will it run the iPhone OS or a real operating system?

I am mostly concerned simply because it comes from Apple. I personally find Apple to be one of the most troubling companies on the planet. Their closed systems and closed attitude towards the rest of the computing world bother me. Even worse are Apple fans. I dread to see the Apple Tablet merely on the grounds that 6 months later all of Apple fandom will be declaring loudly “how brilliant Steve Jobs is – he invented the Tablet!”.

Back to the article in the Washington Post. The author rightfully asks the question “Why would anyone want a tablet computer?” I personally love them. I have been using them for years (remember this for next Christmas kids – APPLE DID/WILL NOT INVENT THE TABLET PC). I have written several other posts about why I like them, and where I would like them to go in the future. Right now I have two Tablets – one is a slate model which I love. The other is the convertible Tablet given out to attendees at Microsoft PDC . This one has a great multi-touch interface running Windows 7. Its only weakness is pour handwriting support due to interference between touch capabilities and handwriting. In the house we also have two HP Touchsmart convertible tablets. These both support multi-touch and handwriting extremely well, and are well priced at just under $1000 (in Canada).

(Note here that MS already has a multitouch interface that supports gestures, handwriting, and runs a real OS, so is useful beyond just being another gadget.)

Now for the stupidest statement in the Washington Post article (possibly the stupidest tech statement made this year):

“The truth is that most of us don’t understand the allure of a tablet computer because they’ve all sucked up until now.”

Ok, the author just revealed himself to either be a moron, woefully uninformed, or just completely lacking in objectivity (perhaps stemming from the Crunchpad association). There are a number of very good tablets out there (and have been for a number of years). Any of the tablets from Motion Computing are great, though they are not consumer oriented (I have been using an LE1600 personally for 4+ years). The HP tablets have been consistently good. I have also heard great things about Toshiba, Fujitsu, and Dell tablets. The one complaint I have about all of them (except maybe the HP Touchsmart) is that the prices are way too high, but that is improving.

I will say I really want more out of a Tablet, as I said in a previous post. But that does not mean that all of the existing devices suck. Such a broad generalization, is well, just stupid.

Here is another statement from the article:

We’ll be living in a future with Minority Report, Star Trek, and Avatar interactive technology

it is interesting to note that the user interface in Minority Report was actually inspired by another non-Apple device – the Microsoft Surface.

The last quote I will take from this article is

Part of it is that Apple has a sterling record with consumer-oriented products.

Well, seems to me that Apple has failed a few more times than the author mentions. Seems the Mac Book Air didn’t do so well. Going back much further, anyone remember Steve Job’s Newton? Going back even further, Apple could be the dominant desktop OS right now if not for Job’s immeasurable ego back in the 80s (has that changed at all?).

My big concern here is how much of the consumer community reads and believes unsubstantiated drivel like this, and so dismisses anything non-Apple without even looking at it.   

A big part of the blame for this has to go to Microsoft, as well, and their atrocious marketing department. Tablet PCs have been around since 2002, and yet I still get stopped everywhere I travel by folks asking what my tablet is. How is that for getting the word out on one of your coolest technologies? It does not help that the press does not like to write about anything Microsoft because it is not “cool” to support MS.

So please folks, remember this – multi-touch, gesture-based computing is real and available today, and it is not from Apple. In addition, it runs an OS that lets you use everything you have been used to using, and does not lock you in to buying everything you ever want through Apple. And, you can even replace your own battery, unlike most Apple devices 🙂

PS – More hype for the “Apple saves the tablet” community is here. Also there is an older article Why Have Tablets Flopped? Here Are Five Reasons referenced. Of the five reasons quoted, only one is valid – price. Note also that the only pictures they use are of the Newton – the only real failure of the bunch. It is really sad that all of the media writing about tablets seems to have drunk the Apple Koolaid.

Biking across Africa – lengthwise.

Well, I find this damned impressive!

A colleague of mine at T4G, David Saevitzon, is leaving in January to undertake a little bike ride from Cairo, Egypt to Cape Town, South. TO give you some idea of the scope of this, David will be travelling some 11,800 km over a period of 120 days. David is also making this a charitable event, taking sponsorships to try to raise $10,000 for Nelson Mandela Children’s Fund (Canada).

Check out the web site for David’s adventure, follow his progress, and maybe even sponsor him.

Off to PDC09

Really looking forward to the Microsoft Professional Developers’ Conference in LA this week – except for the getting up at 4 AM to get to the airport tomorrow!

There are a number of areas I am exciting about for the conference. On Monday I am in an all day workshop on Software in the Energy Economy which should be very interesting given a number of projects I am involved in lately regarding energy management systems and systainability support.

At the conference proper, other than the keynotes, there are a fair number of session I have put in my schedule related to Azure – still very much interested in cloud computing, even though I have had little time to look at it.

I am also planning to attend a number of sessions on Workflow Foundation 4.0, even though it will not be available in SharePoint 2010 (at least initially).

Then there are a number of sessions I want to go to just out of personal interest – those involving Silverlight, touch and multitouch applications, and many others. As always, far more things I want to learn than I could possibly have time for!

While there is some great content around SharePoint 2010, I will probably not focus on that since I got a lot of good SharePoint information at SPC09 a few weeks back.

I am hoping do more blogging from PDC than I did at SPC – not hard since I did not blog at all from SPC! Damn twittier took all of my time!

Using Twitter at a conference

I am finding an interesting use for Twitter at a conference. Normally I would be taking notes during the sessions. Instead, I am just tweeting the interesting points. Then, back in my room, I can pull up all my tweets for the day, and POOF – there are my notes for the day.

I think I may write a OneNote add-in to import my tweets to save them as my notes 🙂

%d bloggers like this: