The Kinect’s Past


Outlook Shortcuts, 2005

“Humans communicate using speech, gesture, and body motion, yet today’s computers do not use this valuable information.  Instead, computers force users to sit at a typewriter keyboard, stare at a TV-like display, and learn an endless set of arcane commands, all of which often leads to frustration, inefficiencies and disuse.”

Mark Lucente, Aug 1999

Mark Lucente wrote this for a panel organized in 1999 in Los Angeles that included Bill Buxton (then with Wavefront, Inc.) and others.  At the time, Mark was working on a project called Dreamspace, pictured below. 


Does Dreamspace look familiar?  It uses a combination of gestures and voice commands, just like this fantasy UX from 2002 you may have seen:


Instead of inspiring us to find new ways to do astronomy or police work, of course, in 2010 these sparks of inspiration gave rise to this instead:


Which is alright.  First we hook the kids, then we conquer the future.

I found Mark’s synopsis of the UX issues of 1999 in the archives of the MIT Media Lab Tangible Media Group.  I’m currently working my way through these papers and am overwhelmed by how smart the authors are as well as how far behind I am in trying to understand UX and what can be accomplished with a bit of vision.

Some Talk Abstracts

Every year requests for speakers at the regional conferences start earlier and earlier.  Today is already the final day for submitting talks to CodestockDevlink has also already started accepting abstracts, though their deadline isn’t until sometime in May.

2011 is a wonderful year for talks about the intersection of development and design.  The MUx counter-movement at MIX11 is all about restoring balance to the force developer-designer story and I certainly hope the message that Josh Blake, Rick Barraza and Sean Gerety have been promulgating gets out. 

At the same time, surely it’s possible to simply have more designer talks in the “closed” sessions – or simply invite those open talks that got turned down directly into the MIX11 schedule.  The basement of the Excalibur is a crowded and smelly place to be holding a counter-conference.

Here are some of the talks I’m planning to shop around this year.  I’m particularly stoked about talking on Kinect Hacks, but will have to see if anyone is actually interested in hearing what I have to say on it.  The 100% slides talk about The Minority Report and Fantasy UX is also very close to my heart:

Windows Phone 7: What’s working and what’s not

It’s been a year since developers began to play with the Windows Phone platform. The marketplace has opened, apps are being sold, and we are getting our first glimpse of what is working and what is not. What did we get wrong at the beginning as WP7 developers? Is the Metro language better observed or ignored? Native apps or web apps? Silverlight or XNA? Paid apps or free apps with advertising? Where should we be concentrating your time when developing a WP7 application – design or architecture? This session will address our lessons-learned from the past year as well as the future of WP7 and rumors about the Mango release.


Kinect Hacks 101

The Kinect has been the surprise hit of the year for Microsoft. The moment developers found out you can plug the Kinect into a PC, everyone started trying to replicate scenes from The Minority Report. At Razorfish, the first thing we built was a mash-up of a WPF physics engine and the Kinect in order to port our DaVinci surface app into a gesture-based experience. Since those first efforts we’ve continued to build app after app for the Kinect as the APIs have evolved. In this session, I will show you what we have learned and how you can get started building your own Minority Report experience in your living room.


The Mouse is Dead

The mouse died and no one even noticed. Coupled with the keyboard it has been the primary means of interacting with personal computers for the past 25 years. 2010 and 2011 saw the end of this dominance with the arrival of many new touch devices running various forms of the Windows OS: WP7, Win7 tablets, MS Surface 2 and the Kinect. Learning to program for touch is more complex than simply replacing every MouseLeftButtonDown event handler with a TouchDown handler. Touch involves learning a proliferation of interaction idioms new to both developers and consumers. Additionally, there are variations in how APIs capture these interactions on different platforms: WP7, Surface, Win 7. This session will cover the ins and outs of working with NUI interfaces on Microsoft platforms so you can make NUI work for you.


Welcome to The Minority Report

Sometimes movies that try to predict future technology end up creating it.  "The Minority Report" is such a movie.  Released in 2002, TMR has inspired technology companies to push their hardware and create new interfaces.  In 2011, we may finally be seeing the fruits of that decade long endeavor with NUI-based smart phones and tablets, the new MS Surface 2 and, most surprising of all, the gesture-based Kinect.  In this time, we have also seen vast improvements in speech recognition technology and even natural language analysis.  This session will provide an overview of the past influences of science fiction works such as William Gibson’s Neuromancer and Star Trek, the present influence of The Minority Report, and new works by writers like Charles Stross that are shaping our future.

Silverlight 5, HTML5 and WPF

Last Thursday, Microsoft Evangelists Glen Gordon and Joe Healy held an all-day Silverlight firestarter in Atlanta.  It was a great event and will be travelling to Tampa, FL on 2/22 and Miami, FL on 2/24.

The real fun for me was what occurred afterwards.  Glen organized an Ask the Silverlight Experts panel in the backroom of a nearby sports bar.  After the ‘Bob Muglia Imbroglio’ – I wonder if that will ever catch on -  it was refreshing to hear people whose careers are deeply tied to the future of Silverlight actually speak candidly about it.  MVPs are typically cautious creatures, anxious not to speak out of turn, contrary to Microsoft strategy, or in violation of their NDAs with MS.  Following the Bob Muglia story, everyone is additionally anxious to not be the next person to torpedo Silverlight.

The panel was made up of Sergey Barskiy (Data MVP), Shawn Wildermuth (Data MVP, Silverlight Trainer), Jeremy Likness (Silverlight MVP, author of Sterling), Joel Ivory Johnson (WP7 Dev MVP) and Rob Schiefer (co-author of an upcoming WP7 book).  I was there representing the local Silverlight User Group which I run with Corey Schuman.   Jim Wooley (VB MVP) and Steve Porter (CAD MVP) were in the audience.

The first question asked, and the one that dominated the rest of the night, was to the effect of “What’s up with Silverlight and HTML5?”

I had originally planned to give a recap of all the arguments and theories but realized after attempting to do this for about an hour that I mostly just remember my own arguments and have, in my memory, distorted everyone else’s.

Then I came across this beautiful photo by Philip-Lorca diCorcia — who currently has an exhibit at the David Swirner gallery in New York City if you get a chance to visit – which summarizes everything that was said that night much better than I can.


In case it isn’t clear, the two dudes high-fiving each other on the right are Silverlight 5 and HTML5.

a footnote to the retreat of the mind


The latest Atlantic contains an article by Brian Christian on the annual Turing Test held in Brighton, England.  In order to pass the Turing Test (also known as the Loebner Prize) a computer program must be able to fool 30 percent of the people it interacts with that it is human.  In 2008, one program missed this goal by only one vote.

In the article, Christian quotes Douglas Hofstadter, the author of Godel, Escher, Bach, on the problem of ‘The Sentence.’   The Sentence is the perennial attempt to frame the all-important definition “The human being is the only animal that …”  We once thought this sentence could be completed with uses language, uses tools, does mathematics, or plays chess, only to be confounded each time by further discoveries about the natural and mechanical world.

‘Sometimes it seems,’ says Douglas Hofstadter […] ‘as though each new step towards AI, rather than producing something which everyone agrees is real intelligence, merely reveals what real intelligence is not.’  While at first this seems a consoling position – one that keeps our unique claim to thought intact – it does bear the uncomfortable appearance of a gradual retreat, like a medieval army withdrawing from the castle to the keep.  But the retreat can’t continue indefinitely.  Consider: if everything that we thought hinged on thinking turns out to not involve it, then … what is thinking?  It would seem to reduce to either an epiphenomenon – a kind of exhaust thrown off by the brain – or, worse, an illusion.  Where is the keep of our selfhood? [emphasis mine]

I have always been a fan of footnotes.  In complex academic works, it is usually the footnotes that contain the most fascinating insights.  They are, in a sense, the epiphenomena of the academic world. 

Stephen H. Voss has a fine translation of Descartes’s The Passions of the Soul, a work Descartes wrote for Princess Elizabeth of Bohemia years after separating the mind and the body in his Meditations on First Philosophy.  What comes out in this later work – and to which attention is drawn in Voss’s footnotes — is that the line between mind and body is not a geographical division like that between countries, but rather a kinesthetic separation between the inside and the outside.  In The Passions, Descartes even begins talking about the inner soul and the interior of the soul, further subdividing the line between self and world.

Concerning this, Voss writes in footnote 78:

Since the soul has no parts […], it is hard to see how to distinguish theoretically the interieur, let alone le plus interieur, of the soul from the rest of it.  As we intimated in note 27* in Part I, it is perhaps more reasonable to see such passages as signs of Descartes’s genuinely neo-Stoic attitude toward the world.  We have seen his focus successively narrow in this work: the body, the pineal gland, the soul, and now its ‘interior.’  A similar itinerary can be traced in the First Meditation: objects that are very small or far away, familiar nearby objects, the body and its senses, the soul and its reason.  And so can one more: examining ‘the great book of the world’ on military travels through several countries; Amsterdam, Leyden, and the isolated village of Egmond; and finally the palace in Stockholm.  What walled fastness can ever provide security? [emphasis mine]

I’ve always wondered if this kinesthetic problem of interiors and exteriors is related to the solution of using metalanguages to avoid problems of self-referentiality in logic.  In particular, I’m thinking of Douglas Hofstadter’s chapter in Godel, Escher, Bach describing Russell and Whitehead’s Principia Matematica,  called “Banishing Strange Loops”:

Russell and Whitehead did subscribe to this view [that self-reference is the root of all evil in logic], and accordingly, Principia Mathematica was a mammoth exercise in exorcising Strange Loops from logic, set theory, and number theory.  The idea of their system was basically this.  A set of the lowest ‘type’ could contain only ‘objects’ as members – not sets.  A set of the next type up could only contain objects, or sets of the lowest type.  In general, a set of a given type could only contain sets of lower type, or objects.  Every set would belong to a specific type.  Clearly, no set could contain itself because it would have to belong to a type higher than its own type […]  To all appearances, then, this theory of types, which we might also call the ‘theory of the abolition of Strange Loops’, successfully rids set theory of its paradoxes, but only at the cost of introducing an artificial-seeming hierarchy, and of disallowing the formation of certain kinds of sets…

This connection I am (less-than-tentatively) proposing, of course, only works if interior and exterior can be mapped to the notions of higher and lower level languages.  This is, however, how we typically think of the emergent self in evolutionary biology.  The highest part of the mind — the most selfish bit – is also the last to have developed in time, while the lizard brain, which the higher functions always seek to constrain, is also considered the part that is least ourselves – it is a mechanical, biological process, and when that lizard brain is in control, we are out of control.

*footnote 27: A pervasive Cartesian conviction is that what is far away can deceive, while what is close at hand can give security.  That is true not only of epistemic security (in addition to the present passage, see Meditations 1 and 3: AT VII, 18 and 37: CSM II, 12-13 and 27; and a. 1 above), but also of emotional security (see Discourse, Part 3: AT VI, 25-27: CSM I, 123-124; and aa 147-148 below).

Phrase of the day: Redundant Appetizer

From a New Yorker portrait of horror maestro Guillermo Del Toro:

We drove east to Burbank. Del Toro is devoted to the Valley—he calls it “that blessed no man’s land that posh people avoid in L.A.” We pulled into Ribs U.S.A., a frayed establishment on Olive Avenue. Del Toro ordered ribs and a lemonade, along with a redundant appetizer of “riblets.”

Read more

How to Hotwire your WP7 Phone Battery


My project manager was out last night and forgot to recharge her WP7 phone.  With the particular model she has, once the battery completely loses its charge, you can no longer recharge it simply by plugging in the phone charger. 

The problem seems to be that the phone always needs to boot up just a little when it is plugged in.  If there is no charge left, then that minimal necessary change is simply missing.

Since we have the same model, the first thing we did was to verify that the problem was with the battery by switching out her battery for mine.  It was definitely the battery.

The next thing we did was scour the Internet to find out if others were having this issue.  Indeed they were.  And not only people with Windows Phone but also some people with Android phones.  In the process, we also found a seemingly crazy solution that involves stripping up a USB cable and recharging the battery directly.  Of course, we couldn’t resist trying to hotwire a phone and it worked perfectly.  Here’s what you do.


First, find an old USB cable you don’t need anymore.  It doesn’t matter what’s on the other end of the wire as long as one end is USB.  Cut The Wire!


Next, strip the cut end to remove the plastic from the wire.  You’ll want to strip about 3/4 of an inch off.  This will expose a thin foil wrapper around the internal wires and filament.  Remove the foil.


Bend back the filament as well as the extra interior wires to expose just the red and the black wires.  These are the wires that actually carry the charge in a USB cable.

The interior wires are too thin to actually strip.  What you can do, however, is cut the red and the black wires at an angle.  This will expose enough of the copper to work with.


Pull your battery out of your Windows Phone device.  Your phone battery will indicate where the positive and the negative touch points are.  Plug your USB connector to your laptop.  Take the other cut end and touch the red wire to the positive and the black wire to the negative.

RED => Positive

BLACK => Negative

Hold the wire to the battery for about two minutes.  People will ask what you are doing if, like me, you do this in the office.  Try to be creative with your response.

After two minutes, put the battery back in your phone and plug your phone in to recharge it.  The battery is still at 0 per cent, but now has just enough charge so the phone is able to start up and begin charging normally.

It’s a bit MacGuyveresque, but makes for a great story.