“K” is for Hacking

K

At the Atlanta .NET User Group presentation on hacking the Kinect last night, I promised to provide instructions on installing the necessary components for hacking the Kinect with the OpenNI stack.  I already feel some trepidation about doing this as figuring out these minimal first steps has always been a bit of an initiation rite for the various members of the OpenNI forums.  Strangely enough, there are few straight-forward walkthroughs on installing basic components.

The few that can be found on the web typically leave out key steps or provide the steps out of order — almost as if there is a general conspiracy to keep the details of these essential steps secret.

You are going to install three pieces: 1. SensorKinect which is the drivers that allow communication between the Kinect device and your PC 2. OpenNI which is an API that sits on top of the drivers and 3. NITE which is a higher level API that sits on top of OpenNI.  4. Additionally, you are going to copy some necessary config files onto your machine.

1. First, download the OpenNI component.  It can be found here.  You can choose to download from either the stable or the unstable branch.  The latter will provide more functionality while the former will be … well … more stable.

It may seem strange that you will be installing the higher level API before you install the drivers they sit on, but that’s the way it has to be done.  To do this in any other order risks opening a rift in space-time and ending the world as we know it.

Install OPENNI-Win32-1.0.0.25.exe (or whatever version happens to be current at the time of this reading).

2. Next you want SensorKinect.  You will need to create a github account to download it from here

Install SensorKinect-Win32-5.0.0.exe (or whatever version happens to be current at the time of this reading).

3. Download the NITE middleware which, like OpenNI, comes in both stable and unstable flavors. 

Install NITE-Bin-Win32-v1.3.0.18.exe (or whatever version happens to be current at the time of this reading).  Use the following key: 0KOIk2JeIBYClPWVnMoRKn5cdY4= when prompted.

4. Download and extract the KinectXMLs.zip file: http://www.imaginativeuniversal.com/codesamples/kinectxmls.zip

Copy KinectXMLs\OpenNI\SamplesConfig.xml to C:\Program Files (x86)\OpenNI\Data\

Copy NITE\Sample-Scene.xml, NITE\Sample-Tracking.xml and NITE\Sample-User.xml to C:\Program Files (x86)\Prime Sense\NITE\Data\

These instructions were passed on to me by my colleague Alex Nichols who keeps them in a safe deposit box.  He, in turn, copied these instructions by candlelight from Steven Dawson, a shadowy and rarely seen uber-developer known for writing kinect hacks like this: http://www.engadget.com/2010/12/05/razorfish-ports-davinci-interface-to-kinect-makes-physics-cool/

You know you have installed everything correctly if a portal to another dimension does not open up above your Kinect.  Just to make sure, though, browse to C:\Program Files (x86)\OpenNI\Samples\Bin\Release (“Program Files” if you are on a 32-bit OS).  Plug your Kinect into a USB port and run NiViewer.exe.  If you can see yourself in the screen that pops up, then the OpenNI layer is working correctly.

Next, browse to C:\Program Files (x86)\Prime Sense\NITE\Samples\Bin\Release.  Try running Sample-PointViewer.exe.  You should quickly know if your NITE middleware is installed correctly.

In my experience, depending on which code branch you install, various samples may not work.  That’s okay.  As long as some of them work everything is fine.

Now you’ll want to try programming something.  Kinect hacker Vangos Pterneas provides several excellent examples you can peruse:

http://www.studentguru.gr/blogs/vangos/archive/2011/01/28/kinect-and-wpf-getting-the-raw-and-depth-image-using-openni.aspx

http://www.studentguru.gr/blogs/vangos/archive/2011/02/12/kinect-and-wpf-painting-with-kinect-using-openni.aspx

http://www.studentguru.gr/blogs/vangos/archive/2011/03/15/kinect-and-wpf-complete-body-tracking.aspx

Again, sometimes these work and sometimes they don’t.  If they aren’t working for you don’t worry too much about it.  Just move on.

Next, you’ll want to play with this Visual Studio 2010 C++ project on github: https://github.com/OpenNI/SampleAppSinbad  It is a 3D model of a piratical ogre that will track your movements.  You have to do some configuration if you want to run it from VS2010.  You can, however, just run the executable in the bin directory.  Reach behind your head with both hands to play with the swords.

I also showed several videos last night providing a visual history of gestural interfaces from laboratories, to the movies, to the mass-market, and finally to the hacking community.  If you are a fan of fantasy-UX, then all of these should already be familiar to you:

Star Trek Voyage Home (1986): http://www.youtube.com/watch?v=v9kTVZiJ3Uc&feature=related

Put That There (1979): http://www.youtube.com/watch?v=RyBEUyEtxQo

DreamSpace (1998): http://www.research.ibm.com/natural/dreamspace/#example

Minority Report (2002): http://www.youtube.com/watch?v=NwVBzx0LMNQ

Iron Man (2008): http://www.youtube.com/watch?v=PYjlMflhysc

Project Natal: http://www.youtube.com/watch?v=Mf44bWQr3jc

12 Best Kinect Hacks: http://www.youtube.com/watch?v=ho8KVOe_y08

Again, to everyone who came out for last night’s presentation — you were extremely patient and I can’t thank you enough for your time and your enthusiasm.

Windows Phone 7 at a Crossroads

dragons

This week sees the Windows Phone Marketplace hitting a new milestone: 11,000 apps published.  While this is an important moment for developers of Windows Phone apps, it is also perhaps a good time to ask a question that affects the future roadmap of the marketplace and, in turn, its prospects for driving revenue for independent developers over the next few years: what sort of WP7 apps are being written?

The picture above is a section of the 16th century Lenox Globe, famous among other things for the lexical note hic sunt drancones, “here there be dragons.”  Windows Phone, being neither fish nor fowl, has the potential to be either a device for high-end apps with, ultimately, limited reach like the iPhone, or alternatively a wild-west platform loaded with free but ultimately ungainly apps, like the Droid platform.

Moreover, the quality of the current applications in the WP marketplace will affect which way WP7 goes.  The direction WP7 heads in, in turn, will affect whether the user-base for WP7 grows, attracting more developers to the WP7 platform, growing the user-base, etc. … and bringing in more revenue for the early adopters of WP7 as a development platform.

Ideally, we would all like Windows Phone to have the reach of the Droid platform but the quality of the iPhone app store. 

As things currently stand, however, WP7 developers may be headed in another more dangerous direction – toward having poor quality apps as well as limited reach.  There are two major hurdles that must be overcome in order to avoid such a sad outcome, and the initiative has to come from developers rather than from Microsoft.  First, WP7 must become a platform for designers as well as developers.  Second, WP7 developers must be willing to defer immediate gratification and avoid ad-funded apps.

The Problem with Metro

Metro is in many ways a beautiful design language.  It is also, sadly, a horribly misunderstood design language.  From the original inspiration in train station iconography and sci-fi movies like The Minority Report , the Metro language has become embodied and calcified in the default controls and styles: black and white and blocky.  This is obviously a boon to developers who do not have to think too hard about what their apps are going to look like, and fulfills Microsoft’s promise that WP7 will be the most developer friendly phone platform.

Apps using the default style look nice enough.  The problem here is that black, white and blocky apps don’t look like they are worth paying 99 cents for – and people in fact aren’t paying the 99 cents or even more that developers are trying to charge for these apps. 

What kind of apps do make money?  First, games built on XNA (and which consequently don’t use the Metro style at all) make money.  The biggest excitement in the marketplace at the moment is the news that Angry Birds and Plants vs Zombies are coming to the platform.

What other kind of apps do well?  One of the most popular talks at the upcoming MIX conference is by Andras Velvart and his designer Balint Orosz on how they created Cocktail Flow ($2.99) which, in their words, is “one of the most beautiful apps out there.”  You don’t have to see the session to know what they did right, though.  It’s the simple fact that there was actually a designer involved.  Most apps in the marketplace are written without any sort of designer input and it shows.

It is time to come to terms with the fact that Metro is more than black, white, blocky and Segoe WP.  The UX for the Zune is also Metro – and in fact the original Metro in many ways.  Bing, in its own way, is also Metro.  The Picture Gallery, Office, and other built-in WP7 apps are also Metro.  Those are the things developers need to start taking inspiration from. 

Once that clicks, then hopefully developers will begin to realize that they need to be working with designers – and if they aren’t they shouldn’t expect to have successful apps.

In my opinion, getting UX is going to have to be the responsibility of us, the developers.  This entails understanding our own limitations as well as the limitations of Microsoft. 

Microsoft as a software company will never get UX.  Whenever they look at an app, they just want to add more features.  This is an engineering frame of mind – more features is always better for an engineer.   The heart of “getting UX” – an overused phrase, sadly – is that less features presented beautifully is the true key to success.

This is not to say Microsoft doesn’t have good UX people.  It’s just to say that those people don’t steer the ship.  If you have any doubt about this, compare a web search of Brandon Watson – director over the WP7 developer platform – and one of Albert Shum — the lead designer for WP7 and of the Metro style.  Who has more influence?  Who would you like to hear more from?

[One of the current community leaders in defining what the Metro style is, in practice, is Scott Barnes.  I highly recommend this article in particular if you want to learn more: 5 things you ought to know about Metro.]

The Problem with Ads

Not everyone sees things this way, of course.  When developers notice that no one is buying their apps, they go for the easiest solution.  Rather than improve their apps, developers are instead trying to give them away for free with ads.

Microsoft’s original pitch to developers was “Your apps are worth more than 99 cents”.  As things turn out, however, we are quickly racing to the bottom with ad-funded apps.  Is this the way to go?

Ads are currently the way most successful Droid developers are making money.  Paid apps, on the other hand, are the norm for successful iPhone apps. 

For the long term health of the WP7 marketplace, we ought to follow the iPhone model.  The problem is that the current reach of the Windows Phone isn’t big enough for WP7 developers to make a lot of money off of paid apps.  The only people doing so are those who are being funded by Microsoft to port their apps to WP7 (i.e., like, I assume, the above mentioned Angry Birds and Plants vs Zombies).

So what are we to do while we wait for the marketplace to grow?  Many people are proposing ads as the right way to get ads into the marketplace.  Current rough estimates are that an ad-based app (one in which the developer is paid for impressions rather than clicks) can bring in about a thousand dollars a month for the first few months.  There are caveats, however.  This will only be true for the first few months, as a free app will quickly reach its saturation point.  Additionally, this will also only be true if you are one of the top 100 or so apps.

This is good enough that some WP7 developers can actually quit their day jobs.  Assuming they can get ten apps into the marketplace and that they can get a new app out every two months or so, an enterprising developer can make a good, if not a great, living building WP7 apps fulltime.  See Elbert Perez’s story: http://www.occasionalgamer.com/

A million dollars isn’t cool. You know what’s cool?  A billion dollars.

For the sake of argument, let’s say the top 100 apps in the WP7 marketplace make an average of a thousand dollars a month (these numbers are purely speculative, of course, and shouldn’t be taken for anything more than a thought experiment). That’s roughly a hundred thousand dollars a month or, approximately, a million dollars a year.  Is that the sort of marketplace we ultimately want?

Elbert Perez is without a doubt living the dream – he’s got a plan to make a modestly good living doing something he loves.

Most WP7 developers, however, are simply moonlighting and hoping to bring in, say, $5000 extra a year.  Their best shot at doing this is to push out ad-based apps.  Like domain squatting, it’s a strategy that will definitely make them some modest scratch.  In the process, however, we end up with the equivalent of a lot of misleading sites on the web that, on occasion, can be rather annoying.

An alternative strategy is proposed by Alan Mendelevich, another prominent Windows Phone developer.  For Alan (or at least the way I understand his post) this is not the time to start cashing in on Windows Phone development.  This is the time to build good apps, give them away for free (free as in beer, not free as in ad-funded) in order to build a solid brand and a good reputation. 

The gain in this, for all of us, is that the marketplace will grow in a healthy way.  Those who have been building WP7 apps for a long time and have strong reputations for good apps will be in a position to take advantage of a strong marketplace.  Those who have made a quick profit on tricking you into clicking on ads accidentally will (in a perfect world) suffer from poor reputations.

If the majority of developers follow this strategy, we can avoid the danger of having a crappy marketplace in a year’s time.  Instead, we can spend the time between, say, now and the Mango release attracting designers into the Windows Phone ecosystem and, in the end, all be much better off than if we all cashed in now.