Kinect for Windows v2 First Look

WP_20131123_001

I’ve had a little less than a week to play with the new Kinect for Windows v2 so far, thanks to the developer preview program and the Kinect MVP program.  The original unboxing video is on Vimeo.  So far it is everything Kinect developers and designers have been hoping for – full HD through the color camera and a much improved depth camera as well as USB 3.0 data throughput. 

Additionally, much of the processing is now occurring on the GPU rather than the onboard chip or your computer’s CPU.  While amazing things were possible with the first Kinect for Windows sensor, most developers found themselves pushing the performance envelope at times and wishing they could get just a little more resolution or just a little more data speed.  Now they will have both.

20131126_110049

At this point the programming model has changed a bit between Kinect for Windows v1 and Kinect for Windows v2.  While knowing the original SDK will definitely give you a leg up, a bit of work will still need to be done to port Kinect v1 apps to the new Kinect v2 SDK when it is eventually released.

What I find actually confusing is the naming.  With the first round of devices that came out in 2010-11, we had the Kinect for XBox and Kinect for Windows.  It makes sense that the follow up to Kinect for XBox is the “Kinect for XBox One”.  But the follow up to Kinect for Windows is “Kinect for Windows v2” so we end up with the Kinect for XBox One as the correlate to K4W2. Furthermore,  by “Windows” we mean Windows 8 (now 8.1) so to be truly accurate, we really should be calling the newest Windows sensor K4W8.1v2.  For convenience, I’ll just be calling it the “new Kinect” for a while.

WP_20131123_004

What’s different between the new Kinect for XBox One and the Kinect for Windows v2?  It turns out not a lot.  The Kinect for XBox has a special USB 3.0 adapter that draws both lots of power as well as data from the XBox One.  Because it is a non-standard connector, it can’t be plugged straight into a PC (unlike with the original Kinect which had a standard USB 2.0 plug).

To make the new Kinect work with a PC, then, requires a special breakout board.  This board serves as an adapter with three ports – one for the Kinect, one for a power source, finally one for a standard USB 3.0 cable. 

We can also probably expect the firmware on the two versions of the new Kinect sensor to also diverge over time as occurred with the original Kinect.

kinec2_skel

Skeleton detection is greatly improved with the new Kinect.  Not only are more joints now detected, but many of the jitters developers became used to working around are now gone.  The new SDK recognizes up to 6 skeletons rather than just two.  Finally, because of the improved Time-of-Flight depth camera, which replaces the Primesense technology used in the previous hardware, the accuracy of the skeleton detection is much better and includes excellent hand detection.  Grip recognition as well as Lasso recognition (two fingers used to draw) are now available out of the box – even in this early alpha version of the SDK.

WP_20131123_005

I won’t hesitate to say – even this early in the game – that the new hardware is amazing and is leaps and bounds better than the original sensor.  The big question, though, is whether it will take off the way the original hardware did.

If you recall, when Microsoft released the first Kinect sensor they didn’t have immediate plans to use it for anything other than a game controller – no SDK, no motor controller, not a single luxury.  Instead, creative developers, artists, researchers and hackers figured out ways to read the raw USB data and started manipulating it to create amazingly original applications that took advantage of the depth sensor – and they posted them to the Internet.

Will this happen the second time around?  Microsoft is endeavoring to do better this time by getting an SDK out much earlier.  As I mentioned above, the alpha SDK for Kinect v2 is already available to people in the developer preview program.  The trick will be in attracting the types of creative people that were drawn to the Kinect three years ago – the kind of creative technologists Microsoft has always had trouble attracting toward other products like Windows Phone and Windows tablets.

My colleagues and I at Razorfish Emerging Experiences are currently working on combining the new Kinect with other technologies such as Oculus Rift, Google Glass, Unity 3D, Cinder, Leap Motion and 4K video.  Like a modern day scrying device (or simply a mad scientist’s experiment) we hope that by simply mixing all these gadgets together we’ll get a glimpse at what the future looks like and, perhaps, even help to create that future.

I Just Inceptioned Visual Studio 2013

inception_sim

Building Windows Store apps in Visual Studio 2013 has gotten a lot more fun with the Simulator.  At first, this seemed to be the same thing as the Emulator for Windows Phone development, but there are some interesting differences.

First, the Simulator actually seems to be closer to the simulator used for Pixel Sense (nee Microsoft Surface 2) development since it allows us to use a mouse to simulate finger touches as well as two finger gestures.  In general, we should all be using touch screens for development – but in the field or on unusual environments like Parallels running on a Mac, this isn’t always doable.  Being able to use the simulator gives us an out.  Additionally, because it allows us to simulate alternative aspect ratios and resolutions, it can be handy even when a touch display is readily available.

The really cool thing about the Simulator, though, is that when it fires up, it seems to create a VM of my current system.  I start a new project, set the debug target to “Simulator” and punch F5. 

My desktop background image shows up inside the Simulator and all my apps show up in the Tiles screen. 

I can even search for Visual Studio 2013 with the Search charm and find VS13. 

Then I can fire it up. 

Then I can look at the bottom of the file menu and, under recent project, find the project I am currently running inside the Simulator! 

The next step is obvious, right?  I set the target of the instance of visual studio running inside my Simulator, set that to “Simulator” and hit F5 to get a neat message:

“Unable to start the Simulator.  Another user on this computer is running Simulator, can not start Simulator.”

This is not standard English, so it’s especially fascinating. As everyone knows, the worthwhile Microsoft error messages are the ones that have never been spellchecked.

Does anyone know if I can log into the simulator as a different user at this point?  This is a rabbit hole I really want to go down.