What is a HoloCoder?

holodeck

Over the past few years we’ve seen the rapid release of innovative consumer technologies that are all loosely related by their ability to scan 3D spaces, interact with 3D spaces or synthesize 3D spaces. These include the Kinect sensor, Leap Motion, Intel Perceptual Computing, Oculus Rift, Google Glass, Magic Leap and HoloLens. Additional related general technologies include projection mapping and 3D printing. Additional related tools include Unity 3D and the Unreal Engine.

Despite a clear family resemblance between all of these technologies, it has been difficult to clearly define what that relationship is. There has been a tendency to categorize all of them as simply being “bleeding edge”, “emerging” or “future”. The problem with these descriptors is that they are ultimately relative to the time at which a technology is released and are not particularly helpful in defining what holds these technologies together in a common gravitational pull.

definitions

I basically want to address this problem by engaging in a bit of word magic. Word magic is a sub-category of magical thinking and is based on a form of psychological manipulation. If you have ever gone out to Martin Fowler’s Bliki then you’ve seen the practice at work. One of the great difficulties of software development is anticipating the unknown: the unknown involved in requirements, the unknown related to timelines, and the unknown concerned with the correct tactics to accomplish tasks. In a field with a limited history and a tendency not to learn from other related fields, the fear of the unknown can utterly cripple projects.

Martin Fowler’s endless enumeration of “patterns” on his bliki takes this on directly by giving names to the unknown. If one reads his blog carefully, however, it quickly becomes clear that most, though not all, of these patterns are illusory: they are written at such an abstract level that they fail to provide any prescriptive advice on how to solve the problems they are intended to address. What they do provide, however, is a sense of relief that there is a “name” that can be used to plug up the hole opened up in time by the fear of the unknown. Solutions architects can return to their teams (or their managers) and pronounce proudly that they have found a pattern to solve the outstanding problem that is hanging over everyone – all that remains is to determine what each “name” actually means.

In this sense, the whole world of software architecture – which Glassdoor ranked as the 11th best job of 2015 — is a modern priesthood devoted to prophetic interpretations of “design patterns”.

I similarly want to use word magic to define the sort of person that works with the sorts of technology I listed at the top of this article. I think I can even do it quite simply with familar imagery.

A holocoder is someone who works with technologies that are inspired by and/or anticipate the Star Trek Holodeck.

 

 

interpretations

 

holodeck

The part of the definition that states “inspired by and/or anticipate” may seem strange but it is actually quite essential. It is based on a specific temporal-cybernetic theory concerning the dissemination of ideas which I will attempt to describe but which is purely optional with respect to the definition.

But first: how can a theory be both essential and optional? This is an issue that Niels Bohr, one of the fathers of quantum mechanics, tackled frequently. In the early 30’s Bohr was travelling through eastern Europe on a lecture tour. During part of the tour, a former student met him at his inn and noticed him nailing a horse shoe over the door of his room. “Professor Bohr”, he asked, “what are you doing?” Niels Bohr replied, “The Inn Keeper informed me that a horse shoe over the door will bring me luck.” The student was scandalized by this. “But Herr Professor,” the student objected, “surely as a physicist and intellectual such as yourself does not believe in these silly superstitions.” “Of course not,” Bohr answered. “But the Inn Keeper reassured me that the horse shoe will bring me luck whether I believe in it or not.”

Here is the optional theory of the Holodeck. Certain technologies, it seems to me, can have such an influence that they shape the way we think about the world. We have seen many examples of this in our past such as the printing press, the automobile, the personal computer and the cell phone. Furthermore we anticipate the advent of similar major technologies in our future. These technologies have what is called a “psychic resonance” and change the very metaphors we use to describe our world. To give a simple example, whereas we originally used mental metaphors to explain computers in terms of “memory”, “processing” and even “computing”, today we use computer metaphors to help explain how the brain works. The arrival of the personal computer caused a shift and a reversal in what semioticians call the relationship between the explanans and the explanandum.

wesley in the holodeck

Psychic impact is transmitted over carriers called “memes”. Memes are basically theoretical constructs that are phenomenally identical to what we call “ideas” but behave like viruses. Memes travel through air as speech and along light waves as images in order to spread themselves from host to host. Traditionally the psychic impact of a meme is measured by the meme’s density over a given space. Besides density, the psychic impact can also be measured based on the total volume of space it is able to infect. Finally, the effectiveness of a meme can also be measured based on its ability to spread into the future. For instance, works of literature and cultural artifacts such as religions and even famous sayings are examples of memes that have effectively infected the future despite a distance of thousands of years between the point of origin of the infection and the temporal location of the target.

While the natural habitat of bacteria like e coli is in the gastrointestinal tract, the natural habitat of memes is in the brain and this leads to a fascinating third form of mimetic transmission. At the level of microtubules in the brain where memes happen to live, we enter the Planck scale in which classical physics do not apply in the way that they do at the macro level. At this scale, effects like quantum entanglement create spooky behaviors such as quantum communication. While theoretically people still cannot communicate with each other in time since that level of semiotics is still governed by classical physics, there is an opening for mimetic viruses to actually be transmitted backwards in time as if they were entering a transporter in one brain and rematerialized in another brain in the past. This allows for a third manner of mimetic spread: in space, forward in time, and finally backwards in time.

Riker in the Holodeck

As an aside, and as I said above, this is an _optional_ theory of psychic impact through time. A common and totally valid criticism is that it appeals to quantum mystery which tends to be misused to justify anything from ghosts to religious cults. The problem with appeals to “quantum mystery” is that this simply provides a name for a problem rather than prescribing actual ways to make predictions or anticipate behavior. In other words, like Martin Fowler’s bliki, it is word magic that provides interpretations of things but not actual solutions. Against such criticisms, however, it should be pointed out that I am explicitly engaged in an exercise in word magic, in which case using certain techniques of word magic – such as quantum mystery – is perfectly legitimate and even natural.

Through quantum entanglement acting on memes at the microtubule level, a technology from our possible future which resembles the Star Trek holodeck has such a large psychic impact that it resonates backwards in time until it reaches and inhabits the brains of the writers of a futuristic science fiction show in the late 80’s and is introduced into the show as the Holodeck. Through television transmissions, the holodeck meme is then broadcast to millions of teenagers who eventually enter the tech industry, become leaders in the tech industry, and eventually decide to implement various aspects of the holodeck by creating better and better 3D sensors, 3D simulation tools and 3D visualization technologies – both augmented and virtual. In other words, the Holodeck reaches backwards in time to inspire others in order to effectively give birth to itself, ex nihilo. Those that have been touched by the transmission are what I am calling holocoders.

 

and/or

Alternatively, this theory of where holocoders come from can be taken as a metaphor only. In this case, holocoders are not people being pulled toward a common future but instead people being pushed forward from a common past. Holocoders are people inspired directly or indirectly by a television show from the late 80’s that involved a large room filled with holograms that could be used for entertainment as well as research. Holocoders work on any or all of the wide variety of technologies that could potentially be combined to recreate that imagined experience.

the dreamatorium

Anyways, that’s my theory and I’m sticking to it. More importantly, these technologies are deeply entangled and deserve a good name, whether you want to go with holocoding or something else (though the holodeck people from the future highly encourage you to use the terms “holocoder”, “holocoding” and “holodeck”).

 

appendix

There are two other important instances of environment simulators which for whatever reason do not have the same impact as the Star Trek holodeck but are nevertheless worth mentioning.

danger_room

The first is the X-Men Danger Room which is an elaborate obstacle course involving holograms as well as robots used to train the X-Men. While the Danger Room goes back to the 60’s, the inclusion of holograms actually didn’t happen until the early 90’s, and so actually comes after the Star Trek environment simulator.

WayStation(Simak)

Clifford D. Simak published Way Station in 1963 (and won a Hugo award for it). It actually anticipates two Star Trek technologies – transporters as well as an environment simulator. Enoch Wallace, the hero of the story, works the earth relay station for intergalactic aliens who transport travelers over vast distances by sending them over shorter hops between the way stations of the title. Because he is so isolated in his job, the aliens support him by allowing him to pursue a hobby. Because Wallace enjoys hunting, the aliens build for him an environment simulator that lets him do big game hunting for dinosaurs.

How to Read Star Wars Comics

wrecked_ship

Are you trying to find a guide to reading Star Wars comic books online? You’ve come to the right place. And obviously — Do. Or do not. There is no try.

Lucas Film just released the second teaser for Star Wars VII. My wife and I found ourselves tearing up as we watched it on her ipad, demonstrating that nostalgia is the only thing stronger than the Force.  We are of the generation that first saw Star Wars in a theater in the 70’s. We remember a time before Star Wars existed and yet it has always been the background myth of our lives as we grew up. What Gilgamesh was to Mesopotamians or Siegfried and Brunhilde to Germans, Luke, Han and Leia are to us.

Timed with the release of the teaser, Marvel Comics has added a ton of Star Wars related comic books to their digital comics service Marvel Unlimited. These comic books span a period from the 90’s to the present in which Dark Horse Comics started spinning up stories from the Star Wars expanded mythology (many based on the books) that fill in gaps left by the movies as well as extending the storyline beyond Star Wars VI. In 2015, Disney, which owns both the Star Wars franchise as well as Marvel Comics, moved the Star Wars publication rights from Dark Horse Comics to Marvel Comics, which is apparently how these classic Dark Horse comics are now appearing online.

 

accessing the digital comics

 

If you want to know what happens in Star Wars after the battle of Endor (but before J. J. Abrams retcons over it) then this is your opportunity. You can even do it for free if you want. Go to the Marvel Unlimited website and enter the promotion code starwars to get one free month – though I’d recommend skipping this and getting an annual subscription for $69. Marvel Unlimited has a decent web interface, but the best way to use the service is with the iPad app.

(Scott Hanselman has a good but critical review of the service written in 2011 that deserves to be read. It’s worth mentioning, though, that the service as well as the UX have greatly improved over the past four years.)

Once you have your subscription your main problem is going to be seeing the trees for the forest. There are thousands of comics in the Marvel catalog and they tend to be listed in alphabetical order. This is sensible, but not particularly helpful if you want to read the continuing Star Wars saga in mythologically chronological order. Additionally, while Marvel is offering large chunks of the Star Wars graphic novel canon, there are pieces missing. This is an additional difficulty in trying to get the full story straight.

 

reading in the correct order

 

While there are lots of comics available through the subscription that are contemporaneous with the events in the movies, in this post I’m just going to try to help you to read the Star Wars comics being offered through Marvel Unlimited in the correct order starting just after the battle of Endor.

TwinEnginesOfDestruction

The first comic of the New Republic Era available is Star Wars: Boba Fett – Twin Engines of Destruction (1997) — I’m using the titles as they are listed in the “browse” tab of the Marvel Unlimited app and including the year to highlight the difference between the chronology and the publication order. I’m also heavily indebted to Wookieepedia (you read it right) for all the correct timeline information.

The story picks up with what is known as the Thrawn Trilogy. In Marvel Unlimited, these are cataloged under three different series of about 5 comics each:

Heir6

Star Wars: Heir to the Empire (1995 – 1996)

 

Dfrtpb

Star Wars: Dark Force Rising (1997)

 

Tlctpb

Star Wars: The Last Command (1997 – 1998)

 

This leads us into the Dark Empire Trilogy in which the emperor turns out not to be as dead as he could be. Dark Horse’s first Dark Empire series is also in many ways what first made the Star Wars comics attractive as a vector for transmitting expanded universe stories. A few comics slip in between Dark Empire II and Empire’s End of which only the Boba Fett story is currently available on MU.

 

Darkempire1

Star Wars: Dark Empire (1991 – 1992)

 

De2_1

Star Wars: Dark Empire II (1994 – 1995)

 

Bobafett_agentofdoom

Star Wars: Boba Fett – Agent of Doom (2000)

 

1121447

Star Wars: Empire’s End (1995)

 

Boba Fett grew as a character mainly because he had an awesome costume and fans just wanted to see more of it. The same could be said of the main character in the next two series. Crimson Empire follows the exploits of one of Emperor Palpatine’s elite bodyguards.

 

001eb0bc_medium

Star Wars: Crimson Empire (1997)

 

978907-crimson_empire_ii

Star Wars: Crimson Empire II – Council of Blood (1998-1999)

 

Chewbacca_TPB

The Chewbacca series (2000) is a commemorative four issue run with stories told by Chewbacca’s friends because he is dead at this point in the Star Wars chronology (::sniff::) which will be overwritten by J. J. Abrams faster than you can unsay “Kaaahhhhhn” as J. J. retcons the expanded Star Wars universe.

At this point we leap a century forward and get into the Star Wars: Legacy comics where we follow the adventures of Cade Skywalker, Ania Solo and lots of other people with familiar-but-not-quite-right sounding names. MU lists three collections in the browse tab.

 

3starwarslegacy

Star Wars Legacy (2006-2010) – 50 issues!

 

Legacy_war_I

Star Wars: Legacy – War (2010 – 2011) – six issues

 

Legacy2

Star Wars: Legacy (2013 – 2014) [aka Star Wars: Legacy II] – 18 issues

 

then what … ?

 

And that’s as far as it goes for now. If you need more to read, you can go back in time and start pounding the 55 issues of Knights of the Old Republic digital comics which will provide the Jedi back story from thousands of years before the movies. On the other hand, you might also want to branch out and see what else MU has to offer. Here’s some other books on Marvel Unlimited that I would highly recommend.

 

nick fury

Nick Fury, Agent of SHIELD #1 (1968)

This single issue written and illustrated by legend Jim Steranko changed the game in comic books. Even as pop art was bringing high art low, Steranko lifted the comic book genre and opened the possibility to start considering comics an art form – or as we prefer to say today, start considering “graphic novels” an art form.

 

Marvel_1602

Marvel 1602 (2002-2003)

While there have been many takes on alternate Marvel timelines, Neil Gaiman’s turn with these eight issues is one of the most interesting. He imagines the classic Marvel heroes finding their place in 17th century Europe.

 

eternals

Eternals (2006)

In the 70’s, Marvel experimented with making Erich von Daniken’s Chariots of the Gods the basis for a comic book and had mixed success. Decades later, Neil Gaiman came along and wrote a seven issue series based on the earlier work to create an amazing story of aliens turning ancient humans into super heroes for their own mysterious purpose. The aliens in question, by the way, happen to be the Celestials who are part of the back story for James Gunn’s Guardians of the Galaxy movie. See – everything ties together in the Marvel universe.

 

gg

Guardians of the Galaxy (2008)

The comics are as good as the movie. There are a few more characters and the ones you know are slightly different. Rocket and Groot are the same, though.  This run of the comics basically revives a bunch of Silver Age characters, modernizes them and throws them together to amazing effect.

 

conquest

Annihilation: Conquest – Starlord (2007)

But if you want to do it right and find out how Peter Quill aka Starlord first meets Rocket, Groot and Bug (who’s Bug you ask?) then you might want to also read the four issues of Annihilation: Conquest – Starlord which is just a part of the much bigger Marvel space event called Annihilation: Conquest. Really, all of Annihilation: Conquest is worth reading because then you’ll get to know more about Quasar, Ronan the Accuser, the Heralds of Galactus, and the Nova Corps.

 

annihilation_4

Annihilation (2006 – 2007)

But if you really really want to do it right, then you’ll read the Annihilation comic event before you read either Annihilation: Conquest or Guardians of the Galaxy. This is where Peter Quill first gets retconned into the contemporary world. In the case that you are fully committing to the effort, the correct order for reading most of the back story for the Guardians movie would be:

Annihilation Prologue (2006), Annihilation (2006 – 2007), Annihilation: Quasar / Annihilation: Nova / Annihilation: Ronan / Annihilation: Silver Surfer / Annihilation: Super Skrull [these are all overlapping series], Annihilation: Conquest Prologue (2007), Annihilation: Conquest (2007), Annihilation: Conquest – Quasar / Annihilation: Conquest – Starlord / Annihilation: Conquest – Wraith / Annihilation: Conquest – Heralds of Galactus, Guardians of the Galaxy (2008), The Thanos Imperative: Ignition (2010), The Thanos Imperative (2010), The Thanos Imperative: Devastation (2010).

It’s totally worth it.

 

Agents_of_Atlas_Vol_1_1

Agents of Atlas (2006 — 2007)

Agents of Atlas, like Guardians of the Galaxy, is an instance of Marvel retconning characters that were abandoned in the 50’s and brought together for a series in the 00’s. Interestingly, this is the second time they have been retconned. The first time was in a What If? one-off from the 70’s. FBI agent Jimmy Woo leads a rag-tag team of super-powered beings against the nefarious criminal organization known as the Atlas Foundation. His team includes Namora of Atlantis, the goddess Venus, Marvel Boy the Uranian, Gorilla Man and M-11 the robot. Chronologically in the Marvel universe, Jimmy Woo’s team is actually considered the original Avengers formed to rescue President Dwight Eisenhower from the clutches of Atlas and then later mysteriously disbanded. This series of six issues from 2006 uncovers what really happened to the team. Another series of 11 issues of Agents of Atlas was released in 2009, which was followed up in 2010 by a five issue series simply titled Atlas.

 

nextwave

Nextwave: Agents of H.A.T.E.

A twelve issue S.H.I.E.L.D. parody written by comic legend Warren Ellis and beautifully drawn by Stuart Immonen. Super powered heroes discover that they aren’t working for the good guys after all, but that their organization are actually the baddies. They decide to do something about it. Ellis said of the series, “It’s an absolute distillation of the superhero genre. No plot lines, characters, emotions, nothing whatsoever. It’s people posing in the street for no good reason.”

 

runaways

Runaways (2003 — 2004)

Misfit, middle-class teenagers discover that their parents really are evil after all when they accidentally witness them performing a human sacrifice to Elder Gods. They also come to discover that, like their super-villain parents, they possess super powers.

 

journey

Journey Into Mystery (2011)

The god of mischief Loki is dead but a young boy appears claiming to be Loki reborn. He struggles however because, being Loki, everybody hates him and nobody trusts him. Written by Kieron Gillen, this is the story of how Loki attempts to redeem himself. It is by turns hilarious and heart breaking. Start with issue #622 if you can and try to at least get to issue #645 which wraps up the Loki story.

 

Secret_Avengers_Vol_1_20

Secret Avengers #20 (2010)

The entire Secret Avengers series is great. I would especially recommend that you read issue #20 which follows a single storyline in the life of Natalia Romanov, aka Black Widow.

 

marvel_zombies

Marvel Zombies (2005-2006)

Marvel Unlimited gives you every variation on Marvel zombies you could possibly want, from the original series to the five follow ups to Marvel Zombies Christmas Carol to Marvel Zombies vs. Marvel Apes. I recommend at least having a taste of the first five issue run.

 

secret warriors

Secret Warriors (2008 — 2011)

In the first comic of this 28 issue run, Nick Fury of S.H.I.E.L.D. discovers that for his entire career, his arch enemy Hydra (“Hail Hydra!”) has been secretly controlling  S.H.I.E.L.D. itself. The last 60 years of secret wars have been a farce scripted by the Nazi Baron Strucker. (This is actually the basis for the storyline in the film Winter Soldier as well as the Agents of S.H.I.E.L.D. TV series.) But Nick doesn’t give up. Instead, he pulls together a team to take Hydra down once and for all.

One Kinect to rule them all: Kinect 2 for XBox One

two_kinects

Yes. That’s a bit of a confusing title, but it seems best to lay out the complexity upfront. So far there have been two generations of the Kinect sensor which combine a color camera, a depth sensing camera, an infrared emitter (basically used for the depth sensing camera) and a microphone array which works as a virtual directional shotgun microphone. Additional software called the Kinect SDK then allows you to write programs that read these data feeds as well as interpolating them into 3D animated bodies that are representations of people’s movements.

Microsoft has just announced that they will stop producing separate versions of the Kinect v2, one for windows and one for the XBox One,  but will instead encourage developers to purchase the Kinect for Windows Adapter instead to plug their Kinects for XBox One into a PC. In fact, the adapter has been available since last year, but this just makes it official. All in all this is a good thing. With the promise that Universal Windows Apps will be portable to XBox, it makes much more sense if the sensors – and more importantly the firmware installed on them – are exactly the same whether you are on a PC running Windows 8/10 or an XBox running XBox OS.

This announcement also vastly simplifies the overall Kinect hardware story. Up to this point, there weren’t just two generations of Kinect hardware but also two versions of the current Kinect v2 hardware, one for the Xbox and one for Windows (for a total of four different devices). The Kinect hardware, both in 2010 and in 2013, has always been built first as a gaming device. In each case, it was then adapted to be used on Windows machines, in 2012 and 2014 respectively.

The now discontinued Kinect for Windows v2 differed from the Kinect for the Xbox One in both hardware and software. To work with Windows machines, the Kinect for Windows v2 device uses the specialized power adapter to pump additional power to the hardware (there is a splitter in the adapter that attaches the hardware to both a USB port as well as a wall plug). The Xbox One, being proprietary hardware, is able to pump enough juice to its Kinect sensor without needing special adapter. Additionally, the firmware for the original Kinect for Windows v1 sensor diverged over time from the Kinect for Xbox’s firmware – which led to differences in how the two versions of the hardware performed. It is now clear that this will not happen with Kinect v2.

Besides the four hardware devices and their respective firmware, the loose term “Kinect” can also refer to the software APIs used to incorporate Kinect functionality into a software program. Prior to this, there was a Kinect for Windows SDK 1.0 through 1.8 that was used to program against the original Kinect for Windows sensor. For the Kinect for XBox One with the Kinect for Windows Adapter, you will want to use the Kinect for Windows SDK 2.0 (“for Windows” is still part of the title for now, even though you will be using it with a Kinect for XBox One, though of course you can still use it with the Kinect for Windows v2 sensor if you happen to have bought one of those prior to their discontinuation). There are also other SDKs floating around such as OpenNI and Libfreenect.

[Much gratitude to Kinect MVP Bronwen Zande for helping me get the details correct.]


50 Shaders of Grey

This isn’t actually a blog post yet. Just a title and a promise. Someday I will backfill this post with amazingly deviant technical information about pixel shaders, vertex shaders, algorithmic drawing, matrix math, ray tracing, kernel convolutions and nipple clamps. Stay tuned.

Unity 5 and Kinect 2 Integration

pointcloud

Until just this month one of the best Kinect 2 integration tools was hidden, like Rappuccini’s daughter, inside a walled garden. Microsoft released a Unity3D plugin for the Kinect 2 in 2014. Unfortunately, Unity 4 only supported plugins (bridges to non-Unity technology) if you owned a Unity Pro license which typically cost over a thousand dollars per year.

On March 3rd, Unity released Unity 5 which includes plugin support in their free Personal edition – making it suddenly very easy to start building otherwise complex experiences like point cloud simulations that would otherwise require a decent knowledge of C++. In this post, I’ll show you how to get started with the plugin and start running a Kinect 2 application in about 15 minutes.

(As an aside, I always have trouble keeping this straight: Unity has plugins, openFrameworks as add-ins, while Cinder has bricks. Visual Studio has extensions and add-ins as well as NuGet packages after a confusing few years of rebranding efforts. There may be a difference between them but I can’t tell.)

1. First you are going to need a Kinect 2 and the Unity 5 software. If you already have a Kinect 2 attached to your XBox One, then this part is easy. You’ll just need to buy a Kinect Adapter Kit from the Microsoft store. This will allow you to plug your XBox One Kinect into your PC. The Kinect for Windows 2 SDK is available from the K4W2 website, though everything you need should automatically install when you first plug your Kinect into your computer. You don’t even need Visual Studio for this. Finally, you can download Unity 5 from the Unity website.

linktounityplugin

2. The Kinect 2 plugin for Unity is a bit hard to find. You can go to this Kinect documentation page and scroll half-way down to find the link called Unity Pro Packages. Aternatively, here is a direct link to the most current version of the plugin as of this writing.

unitypluginfolder

3. After you finish downloading the zip file (currently called KinectForWindows_UnityPro_2.0.1410.zip), extract it to a known location. I like to use $\Documents\Unity. Inside you will find three plugins as well as two sample scenes. The three Kinect plugins are the basic one, a face recognition plugin, and a gesture builder plugin, each wrapping functionality from the Kinect 2 SDK.

newunityproject

4. Fire up Unity 5 and create a new project in your known folder. In my case, I’m creating a project called “KinectUnityProject” in the $\Documents\Unity folder where I extracted the Kinect plugins and related assets.

import

5. Now we will add the Kinect plugin into our new project. When the Unity IDE opens, select Assets from the top menu and then select Import Package | Custom Package …

selectplugin

6. Navigate to the folder where you extracted the KinectforWindows_Unity components and select the Kinect2.0.xxxxx.unitypackage file. That’s our plugin along with all the scripts needed to build a Kinect-enabled Unity 5 application. After clicking on “Open”, an additional dialog window will open up in the Unity IDE called “Importing Package” with lots of files checked off. Just click on the “Import” button at the lower right corner of the dialog to finish the import process. Two new folders will now be added to your Project window under the Assets folder called Plugins and Standard Assets. This is the baseline configuration for any Kinect project in Unity.

unitywarning

7. Now we’ll get a Kinect with Unity project quickly going by simply copying one of the sample projects provided by the Microsoft Kinect team. Go into file explorer and copy the folder called “KinectView” out of the KinectforWindows_Unity folder where you extracted the plugins and paste it into the Assets directory in your project folder. Then return to the Unity 5 IDE. A warning message will pop up letting you know that there are compatibility issues between the plugin and the newest version of Unity and that files will automatically be updated. Go ahead and lie to the Unity IDE. Click on “I Made a Backup.”

added_assets

8. A new folder has been added to your Project window under Assets called KinectView. Select KinectView and then double click on the MainScene scene contained inside it. This should open up your Kinect-enabled scene inside the game window. Click on the single arrow near the top center of the IDE to see your application in action. The Kinect will automatically turn on and you should see a color image, an infrared image, a rendering of any bodies in the scene and finally a point cloud simulation.

allthemarbles

9. To build the app, select File | Build & Run from the top menu. Select Windows as your target platform in the next dialog and click the Build & Run button at the lower right corner. Another dialog appears asking you to select a location for your executable and a name. After selecting an executable name, click on Save in order to reach the final dialog window. Just accept the default configuration options for now and click on “Play!”. Congratulations. You’ve just built your first Kinect-enabled Unity 5 application!

les fruits dangereux

potato_clock

What ever happened to the potato clock? In a revived period of do-it-yourselfers, arduino artists and 3d printing presses – a bright new age of artisanal software and hardware development – what has happened to the epitome of nerdish home engineering? At one point in time you couldn’t escape your teenage years without an awareness of the potato clock – stashed somewhere between the x-ray glasses and the sea monkeys – and then poof, suddenly they vanish from the collective consciousness.

Over lunch with my good friends Joel and Nate, I raised this question and Nate was quick to identify the exact date on which the potato clock disappeared. It happened in 2001. On September 11th, to be precise. On that day, many seemingly innocent things were recognized for the danger they are. The test of this perceptual shift occurred on January 31, 2007 when a marketing campaign for the cartoon series Aqua Teen Hunger Force turned into a bomb scare. Whereas levity typically brings with it perspective and reestablishes old norms, in this case – five years after 9/11 – the guerrilla campaign involving led lights was seen as offensive, in poor taste and led to the resignation of Jim Samples, the man who built up Cartoon Network from nothing. Which led us to ask, over our fruit salads, what other common objects – objects like the lowly pototo — might be revealed to have a sinister aspect.

Children have always known the threatening qualities of fruits and vegetables. Adults, on the other hand, have always been somewhat oblivious to this implicit threat that children are aware of in their bones, and have been known to eat kale and Brussels sprouts with reckless abandon. We considered what it would take to capture the danger implicit in fruits and vegetables and realized all it required was a bit of tape and a phone.

WP_20150321_00_10_24_Pro

Consider the banana. At first appearance, it is a pleasant and unassuming fruit with a waxy peel and a soft, sweet interior. Attach a phone to it with some electrical tape and leave it in the lobby at the airport and its true nature reveals itself. Drop it in the mailbox at the post office and see just how innocent it really is.

WP_20150321_00_10_42_Pro

Leave this at the entrance to a police station and see who laughs.

WP_20150321_00_12_58_Pro

Context obviously matters. A banana in a fruit basket suggests a certain functional role. A banana with electronics taped to it sets up a different sort of context. Through experimentation, we found that even the type of tape used can shift the context in subtle but meaningful ways. Silvery duct tape, for instance, is much more threatening than glossy black electrical tape when attached to fruit.

WP_20150321_00_14_34_Pro

Which begs the question, what is the most threatening fruit? Here is an apple attached to an AT&T Pantech cell phone – one of the last free-with-contract AT&T phones that did not require a data plan and the automatic $25 data fee charged to your account whether you are using data or not – why can’t I just use it for phone calls and wifi, AT&T? In that scenario, the AT&T cellular contract is obviously the most frightening thing.

WP_20150321_00_17_18_Pro

But what happens when we exchange the Pantech for an HTC Windows Phone?

WP_20150321_00_19_33_Pro

Or an iPhone for that matter? Which is more intimidating: an iPhone 5 or an iPhone 6?

WP_20150321_00_44_07_Pro

Here is a Microsoft Band wrapped around an apple. There’s a metaphor in there somewhere.

WP_20150321_00_29_49_Pro

iPhone 5 and a kiwi.

WP_20150321_00_31_53_Pro

Samsung Galaxy 4 with onion and duct tape.

WP_20150321_00_32_01_Pro

I normally associate electrical tape with bombs and duct tape with kidnapping – which makes duct tape more viscerally terrifying for me. Some people claim they associate duct tape with ducks but I think that’s a canard. Onions make me want to cry.

WP_20150321_00_33_37_Pro

One of life’s riddles: is a coconut a fruit or a vegetable?

WP_20150321_00_35_01_Pro

No matter what I do, I can’t make strawberries look threatening.

WP_20150321_00_39_30_Pro

Carrots, on the other hand, are Nature’s terrorists.

WP_20150321_00_39_13_Pro

These carrots are organic, by the way. The extra cost is worth it for the additional fear factor.

WP_20150321_00_41_45_Pro

Rubber bands can be intimidating in the right context. Especially when that context is celery. The safety pin is overkill, maybe?

WP_20150321_00_37_44_Pro

This is a Nokia Lumia Windows Phone 7 developer unit with both front-facing and rear-facing cameras. It is attached to a red plum.

 WP_20150321_00_49_17_Pro

Kindle White meets cantaloupe.

WP_20150321_00_52_50_Pro

And of course, potatoes: fear incarnate.

The Next Book

min_lib

The development community deserves a great book on the Kinect 2 sensor. Sadly, I no longer feel I am the person to write that book. Instead, I am abandoning the Kinect book project I’ve been working on and off over the past year in order to devote myself to a book on the Microsoft holographic computing platform and HoloLens SDK. I will be reworking the material I’ve so far collected for the Kinect book as blog posts over the next couple of months.

As anyone who follows this blog will know, my imagination has of late been captivated and ensorcelled by augmented reality scenarios. The book I intend to write is not just a how-to guide, however. While I recognize the folly of this, my intention is to write something that is part technical manual and part design guide, part math tutorial, part travel guide and part cookbook. While working on the Kinect book I came to realize that it is impossible to talk about gestural computing without entering into a dialog with Maurice Merleau-Ponty’s Phenomenology of Perception and Umberto Eco’s A Theory of Semiotics. At the same time, a good book on future technologies should also cover the renaissance in theories of consciousness that occurred in the mid-90’s and which culminated with David Chalmers’ masterwork The Conscious Mind. Descartes, Bergson, Deleuze, Guattari and Baudrillard obviously cannot be overlooked either in a book dealing with the topic of the virtual, though  I can perhaps elide a bit.

A contemporary book on technology can no longer stay within the narrow limits of a single technology as was common 10 or so years ago. Things move at too fast a pace and there are too many different ways to accomplish a given task that choosing between them depends not only on that old saw ‘the right tool for the job’ but also on taste, extended community and prior knowledge. To write a book on augmented reality technology, even when sticking to one device like the HoloLens, will require covering and uncovering to the uninitiated such wonderful platforms as openFrameworks, Cinder, Arduino, Unity, the Unreal Engine and WPF. It will have to cover C#, since that is by and large the preferred language in the Microsoft world, but also help C# developers to overcome their fear of modern C++ and provide a roadmap from one to the other. It will also need to expose the underlying mathematics that developers need to grasp in order to work in a 3D world – and astonishingly, software developers know very little math.

Finally, as holographic computing is a wide new world and the developers who take to it will be taking up a completely new role in the workforce, the book will have to find its way to the right sort of people who will have the aptitude and desire to take up this mantle. This requires a discussion of non-obvious skills such as a taste for cooking and travel, an eye for the visual, a grounding in architecture and an understanding of how empty spaces are constructed, a general knowledge of literary and social theory. The people who create the next world, the augmented world, cannot be mere engineers. They will also need to be poets and madmen.

I want to write a book for them.

Screens, Sensors and Engines

Valve’s recent announcement about their new Vive headset for virtual reality as well as Epic’s announcement that the Unreal Engine is now free made me realize that it is time to once again catalog the current set of future technologies vying for our attention. Just as pre-NUI computer users need the keyboard and mouse, the post-NUI user needs sensors and just as the pre-NUI user required a monitor to see what she was doing, the post-NUI user needs a headset. Here is the list for 2015 from which, you will notice, Google Glass is now absent:

 

Virtual Reality Augmented Reality Sensors Development Platforms
       
Oculus Rift Microsoft HoloLens Microsoft Kinect 2 Unity 3D
Samsung Gear VR Magic Leap Leap Motion Unreal Engine
Google Cardboard castAR Myo WPF
Valve HTC Vive Epson Moverio Intel RealSense Cinder
Sony Project Morpheus   Orbbec openFrameworks
OSVR Razer   Eye Tribe Tracker  
Zeiss VR One      
       

The HoloLens Toolchain and XAML Grids

occlusion2occlusion

What tech stack will be used to develop applications for HoloLens, Microsoft’s innovative new augmented reality platform?

Keeping in mind that Alex Kipman is the visionary behind both HoloLens and the Kinect sensor, some clues can be gleaned from the current Kinect SDK. The Kinect SDK initially supported development in WPF and C++ with DirectX. Over time, however, and in line with Microsoft’s internal shift to be more open and embrace the tech stacks of non-Microsoft communities, the Kinect SDK has grown to include support for Unity3D, Cinder, OpenFrameworks, OpenCV and even MATLAB.

Legos vs Play-Doh

lego_vs_playdoh

This is a two-pronged approach to tooling that Microsoft luminary Rick Barraza has famously characterized as the distinction between building with legos and sculpting with play-doh.  Legos represent what Microsoft has traditionally been extremely good at: creating reusable components. Reusable components abstract the underlying technology layers so even beginning developers can accomplish difficult tasks without needing to understand the intricacies of networking, graphics cards, or memory management. As long as all you want to do is the 95% of tasks that Microsoft components support, legos may be all that you ever need. Microsoft has almost single-handedly created the modern enterprise software developer community based on component building, drag-and-drop IDEs, and copy /paste.

But what if you are not interested in building applications that look like everyone else’s? What if you want to do the 5% of things that reusable components do not allow you to do. This has typically been difficult. Not only does it require having a fine grain understanding of the underlying operating system but also a desire to hack around Microsoft’s safeguards. Because Microsoft had for so long embraced the component model of software development, it internally saw its role as one of safeguarding applications from attempts to follow any other model of software development. API classes are typically sealed to prevent extension and if one were to ask to have them unsealed, the inevitable reply was always “what would you use that for”?

Now lets consider a more playful software world in which it makes sense for classes to be unsealed by default so we can just start squishing them around in our hands to see what comes of it. This is software as exploration and in fact there is a whole community, frequently styled “creative coders”, who work in this way. The processing software programming platform created by Casey Reas and Ben Fry is the epitome of this movement. It was originally created as a better way to teach computer programming since it is built around drawing shapes rather than displaying text. From this simple and divergent starting point, all “hello world” applications are dramatically different.  Even more profound, however, it became clear that through simple loops and seed values, vastly different effects could be generated using only a few lines of code. Playing with processing feels like sculpting with play-doh. Other homologous tool chains were eventually created that shared processing’s emphasis on the visual rather than the textual: OpenFrameworks, Arduino, Cinder and Unity3D.

Which is a long-about way of saying that for premium experiences, these creative coding tool chains will likely be the tools of choice. If you want to get a jump start on HoloLens development, go learn these platforms:

 

HologramFramwork APIs

 hologramAPIs

I mentioned above that Microsoft has a two-pronged approach to tooling software developers. So far I have only mentioned the play-doh side. As we come closer to the Windows 10 release, however, there has been increased activity in the long dormant WPF platform team – enough to suggest that some sort of Holographic support might be released with a new version of WPF. WPF, after all, is Microsoft’s premier platform for component-based development. If the marketing ideal for HoloLens is to make as many HoloLens supported applications as possible straight out of the gate (as it should be), then an easy to use platform for building new HoloLens applications as well as porting old ones to the new paradigm is an obvious pre-requisite.

The image above is from enterprising colleagues who inspected the Windows 10 symbol packages to find out what kind of holographic support would be natively built into the new OS.  The initial impression is that low-hanging integration will be possible by using some sort of texture mapping model. For example, Silverlight provided a component called the VideoBrush that allowed any control that supported brushes to use a video rather than a solid or gradient texture as a background image. This included even complex 3D shapes or skewed geometries.

HoloLens Grids

occlusion4

To me, this suggests a grid-based programming model for quickly painting applications as textures onto valid surfaces identified by the HoloLens’s built-in depth sensors. The depth sensors will use computer vision algorithms to identify and tag surfaces in a room that can be used to project digital content. The user’s movements and any desirable digital-realistic skewing will be taken care of by the underlying holographic framework. For now, let’s assume that interactions will also be taken care of automatically and will follow a mouse-like hover/press idiom for convenience.

 simplegrid

XAML-based languages like WPF have a unique layout component called a Grid. Unlike tables in html, XAML grids define the layout and the content separately. The layout is specified in ColumnDefinitions and RowDefinitions – for instance one may specify a grid that is 3 X 3, or 2 X 1 (as above), and so on. Content is written out (or dragged) below the column and row definitions. Their placement in the layout is then defined by attaching positional directives on the content as shown below.

 grid2

In this code snippet, I haven’t defined a row so there is just one row by default. I have defined two columns which are a zero based array. Finally, I’ve specified on the green panel that I want it to be positioned in the first column by writing Grid.Column=”0” . The red panel, in turn, is placed in column 1, the second column in the series. The resulting WPF application is shown below:

grid3

In this case, the application is not particularly impressive. We can imagine the same code being written against the Holographic Framework with the following results, however:

occlusion3

And here is what the code for your first “Hello, World” application might look like:

holoworld

occlusion5

It is, I hope, not difficult to see how we can then go from a standard XAML Grid like the one above to a HoloLens-enabled Grid like the one below:

occlusion

At this point, given the dearth of information currently available (I’m dumpster diving through Windows 10 symbol packages, after all) this is obviously just a wild guess. I believe it is a plausible programming model, however, and would provide a royal road to quickly generate applications for Microsoft’s newest and brightest technology innovation.

The Coming Holo Wars and How to Survive Them

 

this is the way the RL world ends

cloud atlas: new seoul

We are the holo men,

We are the stuffed men.

Leaning together

Headpiece filled with straw. Alas!

Our dried voices, when

We whisper together,

Are quiet and meaningless

As wind in dry grass

Or rat’s feet over broken glass

In our dry cellar.

— T. S. Eliot

 

“Disruptive technology” is one of the most over-used phrases in contemporary marketing hyper-speech. Borrowing liberally from previous generations’ research into the nature of political and scientific revolutions (Leon Trotsky, Georges Sorel, Thomas Kuhn), self-promoting second raters have pillaged the libraries of these scholars of disruption and have co-opted their intellects in the service of filling the world with useless gadgets and vaporware. When everything is a disruptive technology, nothing is.

Just as Sorel drew on historical examples of general strikes to form his narrative of idealized proletarian revolution and Kuhn identified three examples of scientific revolution: the transition from the Ptolemaic to the Copernican model of the solar system, the abandoning of phlogiston theory, and the shift from Newtonian to relativistic physics – to distill his theory of the “paradigm shift”, we can similarly take one step back in order to find the treasure hidden in the morass of marketing opportunism.

There have been three* major shakeups in the tech sector over the past several decades; each one was marked by the invocation of the “war” metaphor, the leveraging of large sums of money and massive shifts in the fortunes of well known companies.

The PC Wars – the commoditization of the personal computer in the 80s led to the diminishing of IBM and a surprising victor, Microsoft, which realized that the key to winning the PC Wars lay not with the hardware but with the operating system that made the hardware accessible. Following that model, the mid- to late-90s saw the rise of the Internet, various attempts to create portal solutions, and a pitched battle between Netscape and Microsoft to produce the dominant browser. 

The Browser Wars – the Browser Wars saw the rise and fall of companies like Yahoo! and AOL and the eventual victor turned out not be the best browser but the best search engine: Google. More recently we’ve been going through the Mobile Wars in which Apple has been the clear winner – but also Amazon, Twitter and Facebook.

The Mobile Wars – covering both the rise of smart phones as well as tablet devices, the Mobile Wars have born fruit in the way we view consumer experiences, have shifted software development from desktop to web development, have made JavaScript a first class language, have made responsive design the de facto standard, have made the freelance creative designer the Renaissance person of the 21st century, and perhaps most important have accelerated geolocation technology. Geolocation, as will be shown below, is a key player in the next technology war.

 

between the idea and the reality

 

jupiter ascending

Shape without form, shade without color,

Paralyzed force, gesture without motion;

 

As a devotee of Adam Sandler movies, I was pleased to see him teamed with Judd Apatow and Seth Rogan in 2009’s Funny Men. Adam Sandler movies are up there with “Pretty Woman” and “Dumb and Dumber” in the cable industry as movies that can be shown at any time of day and still be guaranteed to draw viewers. There is a false moment in the middle of the movie, however, in which Adam Sandler and Seth Rogan are flown out to perform at a private party for MySpace. What’s MySpace you ask? It was a social network that was crushed in the dust by Facebook, of which you have probably heard, along with other even more obscure networks like Friendster and Bebo. MySpace are portrayed in the movie as an up-and-rising social network through a last-gasp cross-marketing placement with Universal Studios.

A major characteristic of today’s tech wars is that we do not remember the losers. It does not even matter how big these corporations were during their period of being winners. Once they are gone, it is as if they are completely erased from the timeline, their reputations liquidated in the same fashion as their Aeron chairs and stock options.

To be a winner in the tech wars is to be a survivor of the tech wars. This applies not just to corporations but also to the marketing, business and technical people who are carried in the wake of rising and falling technology trends. IT groups across the US now face the problem of trends they have ignored finally reaching the C-levels as they are being asked about their mobile strategies and why their applications are not designed to be responsive – and perhaps even whey they continue to be written in vb6 or delphi.

These casualties of the Mobile Wars must be wondering what choices they could have made differently over the past several years and what choices they should be making over the next. How does one survive the conflict that comes after the Mobile Wars?

 

between the motion and the act

 

2001 a space odyssey

Those who have crossed

With direct eyes, to death’s other kingdom

Remember us — if at all — not as lost

Violent souls, but only

As the holo men,

The stuffed men.

 

Surviving and even thriving in the coming Holo Wars is possible if you keep an eye out for the contours of future history – if you know what is coming. The first key is knowing who the major players are: Microsoft, Facebook, Google – though there is no guarantee any of them will still be standing when the Holo Wars are over.

Microsoft has catapulted to the front of the Holo Wars with its announcement of the HoloLens on January 21st. HoloLens is the brainchild of Alex Kipman, who also spearheaded the product development of the Kinect. It is expected to be built on some of the technology developed for the Kinect v2 sensor combined with new holographic display technology – possibly involving eye movement tracking – that has yet to be revealed.

Facebook became a participant in the Holo Wars when it bought Palmer Luckey’s company Oculus VR in mid-2014. The Oculus Rift, a virtual reality headset, is basically two mobile display screens placed in front of a user’s eyeballs in order to show stereoscopic digital visualizations. The key to this technology is John Cormack’s ingenious use of sensors to track and anticipate head movements to rotate and skew images in a realistic way in the virtual world revealed by the Rift.

Google participates in several ways. Even though the explorer program is now closed, Google Glass arrived with great fanfare and created excitement around the fashion and consumer uses of this heads-up display technology. Following Google’s major investment in Rony Abovitz’s Magic Leap in October 2014, a maker of mysterious augmented reality technology, it now appears that this is the more likely future direction of Google Glass or whatever it is eventually called. Magic Leap, in turn, has added some amazing names to its payroll including Gary Bradski of OpenCV fame and Neal Stephenson, the author of Snow Crash. The third leg of Google’s investment in a holographic future is the expertise in geolocation it has acquired over the past decade.

The next key to surviving the Holo Wars is to understand what skills will be needed when the fighting starts. The first skill is a deeper knowledge of computer graphics. Since the rise of the graphical user interface, software development platforms have increasingly abstracted away the details of generating pixels and managing human-computer interactions. Future demands for spatially aware pixels will force developers to relearn basic mathematical concepts, linear algebra, trigonometry and matrix math.

In addition to mathematics, machine learning will be important as a way of making overwhelming amounts of data manageable. Modern computer interactions are relatively simple. Users sit in one place, in a fixed position respective to the machine, and rarely deviate from this position. Input is passed through transducers that reduce desire and intent into simple signals. Digital reality experiences, on the other hand, not only receive gestural information which must be interpreted but also physical orientation, world coordinates, facial expressions and speech commands. A basic knowledge of Bayesian probability and stochastic calculus will be part of the tool chest of anyone who wants to successfully navigate the Holo joblists of the future.

To reforge ourselves with skills for surviving the next seven years, designers must also become better programmers and software programmers must become more creative. The freelance creative, a job role that expanded dramatically during the Mobile Wars, will have an even brighter future in a world pervaded by augmented reality experiences. In order to make the shift, however, creatives will need to move beyond their comfort zone of creating PSDs in Photoshop and learn motion graphics as well as basic computer programming. Programmers likewise will need to move beyond the conceit that coding is an inherently creative activity; moving data around from point A to point B is no more creative than moving books around a sprawling Amazon warehouse and then packing them up for shipping is a poetic.

Real creative coding involves learning how to construct digital-to-physical experiences with Arduino, how to program self-generating visual algorithms with Processing, how to create 3D worlds in Unity and how to create complex visual interactions with openFrameworks and Cinder. These activities will become the common vocabulary of the future programmers of augmented experiences. Hiring managers and recruiters will expect to find them on resumes and without them, otherwise experienced tech workers be unhireable or worse, relegated to maintaining legacy web applications.

 

not with a bang but a whimper

 

enders game

The eyes are not here

There are no eyes here

In this valley of dying stars

In this holo valley

This broken jaw of our lost kingdoms

In this last of meeting places

We grope together

 

How can one tell if these prescriptions for the future Holo Wars are real and actionable or simply more marketing hype attempting to take advantage of people’s natural gullibility regarding technical gadgets? Aren’t we always being burned by overly optimistic portrayals of the future that never come to pass? Where are our flying cars? Where are our remote work locations?

In order for the Holo Wars to play out, certain milestones need to be achieved. Consequently, if you start seeing these milestones realized, you will know that you are in fact living through a fight over the next disruptive technology that will destroy some major tech corporations while affirming others at the apex of the tech world, one that will also reward those that have positioned themselves with useful skills for this future economy and punish those who do not. These milestones are: technology, monetization, persistent holographic objects, belief circles, overlapping dissociative realities.

Technology: the first phase is occurring now with the three major players discussed above and several additional players such as Metaio, Qualcomm and Samsung engaged in building up consumer augmented reality hardware and supporting technologies such as geolocation and gestural interfaces.

Monetization: innovation costs money. The initial hardware and infrastructure effort will likely be subsidized by the major players. Over time, the monetization model will likely follow what we see on the internet with “free” consumer experiences being subsidized by ads. There will be a struggle between premium subscription based experiences offering to remove the ads while providing better, higher resolution experiences with better content. These portal solutions will also contend against free and low-cost plug-in content provided by hackers and freelance creatives. How this plays out will depend largely on whether the premium content providers will be able to block out independents through standards and compatibility issues as well as whether hackers will find ways to overcome these roadblocks. There is also the possibility that some of the players might be looking at a much longer game and will foster an open AR content generation community rather than attempt to crush it. If the AR economy opens up in this way, a new service sector will grow made up of one set of people generating digital worlds for another set to live in.

Persistent Holographic Objects: virtual worlds are typically subjective experiences. They can be made inter-subjective, as they are in MMOs, by creating virtual topology in which people co-exist and co-operate. In augmented worlds, on the other hand, shared topology is an inherent feature. AR shared topology is called reality. In order to make AR worlds truly inter-subjective, rather than simply objective or subjective, shared holo objects must be part of the experience. Pesistent holo objects such as a digital fountain, a digital garden, or a digital work of art will have a set location and orientation in the world. AR players will need to travel to these locations physically in order to experience them. Unlike private AR or VR experiences in which each player views copies of the same digital object, with a shared experience each player can be said to be looking at the same persistent holo object from different points of view. In order to achieve persistent holographic objects, we will require finer grained geolocation than we currently have. AR gear must also be improved to become more usable in direct sunlight.

Belief Circles: a healthy indie creative fringe-economy and persistent holographic objects will make it possible to customize intersubjective experiences. People have a natural tendency to form cliques, parties and communities. Belief circles, a term coined by Vernor Vinge, will provide coherent community experiences for different guilds based on shared interests and shared aspirations. Users will opt in and out of various belief circles as they see fit. The same persistent holographic objects may appear differently to members of different circles and yet be recognized as sharing a common space and perhaps a common purpose. For instance, the holosign in front of the local Starbucks will have a permanent location and consistent semantic purpose, in AR space, but a polymorphic appearance. To paraphrase a truism, beauty will be in the eye of one’s belief circle.

Overlapping Dissociative Realities: divergent intersubjectivities will produce both a greater awareness of synchronicity – and a sense of deja vu as AR content is copied freely into multiple locations — as well as an increased sense of cognitive dissonance. Consider the example of going into Starbucks for coffee. The people waiting in line will likely each be members of varying belief circles and consequently will be having different experiences of the wait. This is not a large departure since we typically do not care about what other people in line are doing and even avoid paying attention unless they take too long making a selection. In this case, divergent belief circles make it easier to follow our natural instinct to avoid each other. Everyone in the holo valley is anonymous if they want to be. When one arrives at the head of the line, however, something more interesting happens. Even though the customer and the barista likely belong to different belief circles, they must interact, communicate, and perform an economic exchange; these two creatures from different worlds. What will that be like? Will one then lift a corner of the holo lenses in order to rub a sore eye only to discover that this isn’t a Starbucks at all but really a Dunkin’ Donuts which had silently bought out the other chain in a hostile takeover the previous week? Will your coffee taste any different if it looks exactly the same?

* 1996 was witness to a small skirmish between OpenGL and Direct3D that has subsequently come to be known as the API Wars. While the API Wars have had long lasting ripples, I don’t see them as having the tectonic effect of the other historical phenomena I am describing – plus anyways Thomas Kuhn only provides three major examples of his thesis and I wanted to stick to that particular design pattern.

[Much gratitude to Joel and Nate for collaborating on these scenarios over a highly entertaining lunch.]