What’s In Kinect for Windows SDK 1.5?

You shouldn't have come back, Flynn.

Microsoft has just published the next release of the Kinect SDK: http://www.microsoft.com/en-us/kinectforwindows/develop/developer-downloads.aspx  Be sure to install both the SDK and the Toolkit.

This release is backwards compatible with the 1.0 release of the SDK.  This is important, because it means that you will not have to recompile applications you have already written with the Kinect SDK 1.0.  They will continue to work as is.  Even better, you can install 1.5 right over 1.0 – the install we take care of everything and you don’t have to go through the messy process of tracking down and removing all the components of the previous install.

I do recommend upgrading your applications to 1.5 if you are able, however.  There are improvements to tracking as well as the depth and color data.

Additionally, several things developers asked for following the initial release have been added.  Near-mode, which allows the sensor to work as close as 40cm, now also supports skeleton tracking (previously it did not). 

Partial Skeleton Tracking is now also supported.  While full body tracking made sense for XBox games, it made less sense when people were sitting in front of their computer or even simply in a crowded room.  With the 1.5 SDK, applications can be configured to ignore everything below the waist and just track the top ten skeleton joints.  This is also known as seated skeleton tracking.

Kinect Studio has been added to the toolkit.  If you have been working with the Kinect on a regular basis, you have probably developed several workplace traumas never dreamed of by OSHA as you tested your applications by gesticulating wildly in the middle of your co-workers.  Kinect Studio allows you to record color, depth and skeleton data from an application and save it off.  Later, after making necessary tweaks to your app, you can simply play it back.  Best of all, the channel between your app and Kinect Studio is transparent.  You do not have to implement any special code in your application to get record and play-back to work.  They just do!  Currently Kinect Studio does not record voice – but we’ll see what happens in the future.

Besides partial skeleton tracking, skeleton tracking now also provides rotation information.  A big complaint with the initial SDK release was that there was no way to find out if a player/user is turning his head.  Now you can – along with lots of other tossing and turning: think Kinect Twister.

Those are things developers asked for.  In the SDK 1.5 release, however, we also get several things no one was expecting.  The Face Tracking Library (part of the toolkit) allows devs to track 87 distinct points on the face.  Additional data is provided indication the location of the eyes, the vertices of a square around a player’s face (I used to jump through hoops with OpenCV to do this), as well as face gesture scalars that tell you things like whether the lower lip is curved upwards or downwards (and consequently whether a player is smiling or frowning).  Unlike libraries such as OpenCV (in case you were wondering), the face tracking library is using rgb as well as depth and skeleton data to perform its analysis.

I fight for the Users!

The other cool thing we get this go-around is a sample application called Avateering that demonstrates how to use the Kinect SDK 1.5 to animate a 3D Model generated by tools like Maya or Blender.  The obvious way to use this, though, would be in common motion capture scenarios.  Jasper Brekelmans has taken this pretty far already with OpenNI and there have been several cool samples published on the web using the K4W SDK (you’ll notice that everyone reuses the same model and basic XNA code).  The 1.5 Toolkit sample takes this even further by, first, having smoother tracking and, second, by adding joint rotation to the mocap animation.  The code is complex and depends a lot on the way the model is generated.  It’s a great starting point, though, and is just crying out for someone to modify it in order to re-implement the Shape Game from v1.0 of the SDK.

The Kinect4Windows team has shown that it can be fast and furious as it continues to build on the momentum of the initial release.

There are some things I am still waiting for the community (rather than K4W) to build, however.  One is a common way to work with point clouds.  KinectFusion has already demonstrated the amazing things that can be done with point clouds and the Kinect.  It’s the sort of technical biz-wang that all our tomorrows will be constructed from.  Currently PCL has done some integration with certain versions of OpenNI (the versioning issues just kill me).  Here’s hoping PC will do something with the SDK soon.

The second major stumbling block is a good gesture library – ideally one built on computer learning.  GesturePak is a good start though I have my doubts about using a pose approach to gesture recognition as a general purpose solution.  It’s still worth checking out while we wait for a better solution, however. 

In my ideal world, a common gesture idiom for the Kinect and other devices would be the responsibility of some of our best UX designers in the agency world.  Maybe we could even call them a consortium!  Once the gestures are hammered out, they would be passed on to engineers who would use computer learning to create decision trees for recognizing these gestures much as the original skeleton tracking for Kinect was done.  Then we would put devices out in the world and they would stream data to people’s Google glasses and … but I’m getting ahead of myself.  Maybe all that will be ready when the Kinect 2.5 SDK is released.  In the meantime, I still have lots to chew on with this release.

Famous Youtubers: from our far-flung correspondent

Still not recovered from book writing, I have asked my eleven year old son to provide an overview of what’s going on in YouTube land.  My son spends a lot of time working on his own videos – mostly guides to Minecraft and short Lego stop-motion films – and looks up to the sort of people who have managed to eek out a living doing this.  Here are some of the movers-and-shakers in his world:

Hello audience, I am Paul Ashley; son of James Ashley… I am writing this article because of my epic writing skills I gained at school! Oh also because my dad said to… My Youtube account is PaulVAshley so remember to subscribe to me! Or don’t… Let’s begin our Top 5 Most subscribed Youtubers!

#5: Freddiew (Freddie Wong) 3,022,460(as of now) Subscribers.

Freddie and Brandon are two good friends who enjoy making videos with sweet VFX. I’ve always liked their videos, and I still do. I was first introduced to the channel by my friend ANONYMOUS. Umm… okay… anyway, he wanted to show me a tutorial Freddie and Brandon made on First Person Shooter Videos. I began watching all of his short movies starting with “Mr. Toots.” I have become one of his biggest fans. I also wonder what he has in store for us in “Video Game High School.” He is a great director and he is my role model!

#4: Machinima 4,356,027(as of now) Subscribers.

Machinima is an actual company that employs people to play games all day and occasionally make a “machinima” (A video with voices filmed from a game) from time to time. I think this channel is slightly unfair because they have hundreds of people making their videos. I enjoy certain songs that they make, but most videos I think to be just plain stupid. This is only my opinion though… Overall, I really like them only they sometimes have a video that is “bad.”

#3: Smosh (Ian Hecox and Anthony Padilla) 4,464,823(as of now) Subscribers.

Smosh is definitely my personal favorite Youtube channel. They upload new videos every week. Ian has a separate channel for making shows called “Ian is bored,” and “Lunchtime with Smosh.” I was first introduced by a few friends, one of them being Sam. Anyways, we would watch the “Theme song” series (Mortal Kombat, Pokémon, Teenage Mutant Ninja Turtles…). That was back in ’07 or ’08. Nowadays, they upload sketch videos. Overall, I love all of their videos except a few crappy ones.

#2: Nigahiga (Ryan Higa) 5,256,220(as of now) Subscribers.

Nigahiga… The most popular, classic Youtuber of all time! He is definitely the most famous among Youtubers. I was first introduced by my friends Shirish, Sam, and ANONYMOUS. We enjoyed videos like “How to be Ninja, Gangster, and Nerd.” My favorite video is “THE BEST CREW: The Audition.” I almost died in laughter. Overall, I like ALL of his videos.

#1: RayWilliamJohnson 5,408,244(as of now) Subscribers.

FINALLY! I’ve been enslaved to write this article for HOURS! So… where were we… Ah yes, lucky number 1. Ray is a Youtuber that makes a web show called =3. I was first introduced by my friend Shirish. Mr. Johnson (hehe) used to entertain me when I was 10, but I’ve grown ever-so bored of his predictable jokes. He also has a channel called “BreakingNYC.” Ahem, now this boy-man is funny to the creepy weirdoes of Youtube. Overall, I hate to sound sketchy but I dislike all of his videos.

YAY! ENDING PARAGRAPH! I like all of the channels I reviewed except RWJ. Okay, bye guys that’s all you get.

-From the insane mind of Paul Vladimir Ashley.