Tag Archives: Virtual Reality

Reminder to pre-buy your Black Panther tickets

black_panth

Tickets are actually already selling out for the new Black Panther movie, in part because of buy outs of complete theaters for children like these Ron Clark Academy students in Atlanta:

happy1

happy2

For those who don’t know (bad nerds) the Black Panther is a superhero in the Marvel Universe who is also the king of the African nation of Wakanda. Wakanda is secretly the most technologically advanced country in the world, the sole source of vibranium in the world (magic metal, Captain America’s shield is made of it), but projects an image of just being another African nation in order to avoid interference. In the marvel universe it had an all out war with the Submariner’s Atlantean army a few years ago and currently auteur Ta-Nehisi Coates is taking a turn at writing the series and problematizing it (which I don’t totally like but tastes and all that).

Deadly_Hands_of_Kung_Fu_29

The history of the series is basically the usual Marvel thing – Marvel takes advantage of racial trends and exploits them (like with Luke Cage, Iron Fist and Shang-chi) and end up creating something kind of miraculous. In this case, a kingdom of black people who are more advanced than anybody else, culturally and technologically.

wonder-woman-4

If I can talk race and gender a little (feel free to squirm) according to a friend, it does for black people what she assumes Wonder Women did for white women. You get to see yourself in an ideal way without any cultural or political baggage. How do you create a movie hero without any cultural baggage or identity politics attached – you create a fictional country like Themyscira or Wakanda and make your characters come from there – that way they don’t become walking political arguments but instead just _are_.

thorteamup

So after seeing Wonder Woman, my wife asked me if that’s what it’s like for men to see movies, and I think, yeah, pretty much. I’m not Thor, but he’s an ideal projection of myself when I watch the movie and he gets to drink and carouse and hang out with his buddies and women admire him and no one ever neggs him for it. And my wife said she’s learned to watch and enjoy those kinds of movies but Wonder Woman showed her what that experience could really be like.

fury

At the risk of overselling — Black Panther is going to do that for race, according to a friend who got to go to the Hollywood premiere. No white guilt, no resentment, no countries getting called sh* holes, just gorgeous, powerful black people and a reprieve from our crazy mixed up world for a while. Plus, again according to the friend, it’s also another fun Marvel movie.

b_mirror

And here’s the catch for lovers of VR and AR – obviously there’s going to be lots of great Cinema4D faux-holograms used to show how advanced Wakanda is. Not only did Marvel movies pioneer this, but holograms are the chief way movies and tv show “advanced” societies (e.g. Black Mirror, Electric Dreams).

vr

But more importantly, when we talk about “virtual” immersive experiences I think we implicitly know it means more than just having objects in a 3D space. The world is a given and stuck thing, while virtual reality frees us from that and lets us see it differently. The killer AR/VR app is going to do that at a very deep level. I think Black Panther is going to provide an ideal/target/goal for what we want to achieve with all of our headgear. An artificial experience that alters the way we see reality – if only for a few hours plus the afterglow period. Great virtual reality needs to alter our real reality – and make it better.

A User’s Guide to the terms VR, AR, MR and XR – with a tangent about pork

IMG_0083

Virtual reality, augmented reality, mixed reality and XR (or xR) are terms floating around that seem to describe the same things – but maybe not – and sometimes people get very angry if you use the terms incorrectly (or at least they say they do).

The difficulty is that these terms come from different sources and for different reasons, yet the mind naturally seeks to find order and logic in the world it confronts. A great English historical example is the way Anglo-Saxon words for animals have complementary Norman words for the cooked versions of those beasts: cow and beef (boeuf), pig and pork (porc), sheep and mutton (mouton). It is how the mind deals with a superfluity of words – we try to find a reason to keep them all.

vietnamese-pork-noodle

So as an experiment and a public service, here’s a guide to using these terms in a consistent way. My premise is that these terms are a part of natural language and describe real things rather than marketing terms meant to either boost products or boost personal agendas (such as the desire to be the person who coined a new term). Those constraints actually make it pretty easy to fit all these phrases into a common framework and uses grammar to enforce semantic distinctions:

1) Virtual reality is a noun for a 3D simulated reality that you move through by moving your body. A sense of space is an essential component of VR. VR includes 360 videos as well as immersive 3D games on devices like the Oculus Rift, HTC Vive and Microsoft Immersive headsets.

2) Augmented reality is a noun for an experience that combines digital objects and the real world, typically by overlaying digital content on top of a video of a real world (e.g. Pokémon Go) or by overlaying digital content on top of a transparent display (e.g. HoloLens, Meta, Magic Leap, Daqri).

3) Mixed reality is an adjective that modifies nouns in order to describe both virtual and augmented reality experiences. For instance:

a. A mixed reality headset enables virtual reality to be experienced.

b. The Magic Leap device will let us have mixed reality experiences.

4) xR is an umbrella term for the nouns virtual reality and augmented reality. You use xR generically when you are talking about broad trends or ambiguously when you are talking in a way that includes both VR and AR (for instance, I went to an event about xR where different MR experiences were on display). xR may, optionally, also cover AI and ML (aren’t they the same thing?).

This isn’t necessarily how anyone has consistently used these terms in 2017, but I feel like there is a trend towards these usages. I’m going to try to use them in this way in 2018 and see how it goes.

Hitchhiking the Backroads to Augmented Reality

To date, Microsoft has been resistant to sharing information about the HoloLens technology. Instead, they have relied on shock and awe demos to impress people with the overall experience rather than getting mired down in the nitty-gritty of the software and hardware engineering. Even something as simple as the field-of-view is never described in mundane numbers but rather in circumlocutions about tv screens X distance from the viewer. It definitely builds up mystery around the product.

Given the lack of concrete information, lots of people have attempted to fill in the gaps with varying degrees of success which, in their own way, make it difficult to navigate the technological true true. In an effort to simplify the research one typically has to do on one’s own in order to understand HoloLens and AR, I’ve made a sort of map for those interested in making their way. Here are some of the best resources I’ve found.

1. You should start with the Oculus blog, which is obviously about the Oculus and not about HoloLens. Nevertheless, the core technology the makes the Oculus Rift work is also in the HoloLens in some form. Moreover, the Oculus blog is a wonderful example of sharing and successfully explaining complicated concepts to the layman. Master these posts about how the Rift works and you are half way to understanding how HoloLens works:

2. Next, you should really read Oliver Kreylos’s (Doc OK) brilliant posts about the HoloLens field of view and waveguide display technology. Many disagreements around HoloLens would evaporate if people would simply invest half an hour into reading OK’s insights :

3. If you’ve gone through these, then you are ready for Dr. Michael J. Gourlay’s youtube discussion of surface reconstruction, occlusion, tracking and mapping. Sadly the audio drops out at key moments and the video drops out for the entire Q & A, but there’s lots of gold for everyone in this mine. Also check out his audio interview at Georgia Tech:

4. There have been lots of first-impression blog posts concerning the HoloLens, but Jasper Brekelmans provides far-and-away the best of these by following a clear just-the-facts-ma’am approach:

5. HoloLens isn’t only about learning new technology but also discovering a new design language. Mike Alger’s video provides a great introduction into the problems as well as some solutions for AR/VR interface and usability design:

6. Oculus, Leap Motion and others who have been designing VR experiences provide additional useful tips about what they have discovered along the way in articles like the now famous “Swayze Effect” (yes, that Swayze):

7. Finally, here are some video parodies and inspirational videos of VR and AR from the tv show Community and others:

I know I’ve left a lot of good material out, but these have been some of the highlights for me over the past year while hitchhiking on the backroads leading to Augmented Reality. Drop them in your mental knapsack, stick out your thumb and wait for the future to pick you up.

Virtual Reality Device Showdown at CES 2016

WP_20160108_10_04_05_Pro

Virtual Reality had its own section at CES this year in the Las Vegas Convention Center South Hall. Oculus had a booth downstairs near my company’s booth while the OSVR (Open Source Virtual Reality) device was being demonstrated upstairs in the Razer booth. The Project Morpheus (now Playstation VR) was being demoed in the large Sony section of North Hall. The HTC Vive Pre didn’t have a booth but instead opted for an outdoor tent up the street from North Hall as well as a private ballroom in the Wynn Hotel to show off their device.

WP_20160105_18_24_03_Pro

It would be convenient to be able to tell you which VR head mounted display is best, but the truth is that they all have their strengths. I’ll try to summarize these pros and cons first and then go into details about the demo experiences further down.

  • HTC Vive Pre and Oculus Rift have nearly identical specs
  • Pro: Vive currently has the best peripherals (Steam controllers + Lighthouse position tracking), though this can always change
  • Pro: Oculus is first out of the gate with price and availability of the three major players
  • Con: Oculus and Vive require expensive latest gen gaming computers to run in addition to the headsets ($900 US +)
  • Pro: PlayStation VR works with a reasonably priced PlayStation
  • Pro: PlayStation Move controllers work really well
  • Pro: PlayStation has excellent relationships with major gaming companies
  • Con: PlayStation VR has lower specs than Oculus Rift or HTC Vive Pre
  • Con: PlayStation VR has an Indeterminate release date (maybe summer?)
  • Pro: OSVR is available now
  • Pro: OSVR costs only $299 US, making it the least expensive VR device
  • Con: OSVR has the lowest specs and is a bit DIY
  • Pro: OSVR is a bit DIY

You’ll also probably want to look at the numbers:

  Oculus Rift HTC Vive Pre PlayStation VR OSVR Oculus DK2
Resolution 2160 x 1200 2160 x 1200 1920 x 1080 1920 x 1080 1920 x 1080
Res per eye 1080 x 1200 1080 x 1200 960 x 1080 960 x 1080 960 x 1080
FPS 90 Hz 90 Hz 120 Hz 60 Hz 60 / 75 Hz
Horizontal FOV 110 degrees 110 degrees 100 degrees 100 degrees 100 degrees
Headline Game Eve: Valkyrie Elite: Dangerous The London Heist Titans of Space
Price $600 ? ? $299 $350/sold out

 

WP_20160105_19_34_23_Pro

Let’s talk about Oculus first because they started the current VR movement and really deserve to be first. Everything follows from that amazing initial Kickstarter campaign. The Oculus installation was an imposing black fortress in the middle of the hall with lines winding around it full of people anxious to get a seven minute demo of the final Oculus Rift. This was the demo everyone at CES was trying to get into. I managed to get into line half an hour early one morning because I was working another booth. Like at most shows, all the Oculus helpers were exhausted and frazzled but very nice. After some hectic moments of being handed off from person to person, I was finally led into a comfortable room on the second floor of Fortress Oculus and got a chance to see the latest device. I’ve had the DK2 for months and was pleased to see all the improvements that have been made to the gear. It was comfortable on my head and easy to configure, especially compared to the developer kit model that I need a coin in order to adjust. I was placed into a fixed-back chair and an Xbox controller was put into my hand (which I think means Oculus Rift is exclusively a PC device until the Oculus Touch is released in the future) and I was given the choice of eight or so games including a hockey game in which I could block the puck and some pretty strange looking games. I was told to choose carefully as the game I chose would be the only game I would be allowed to play. I chose the space game, Eve Valkyrie, and until my ship exploded I flew 360 degrees through the void fighting off an alien armada while trying to protect the carriers in my space fleet.

WP_20160109_15_41_38_Pro

What can one say? It was amazing. I felt fully immersed in the game and completely forgot about the rest of the world, the marketing people around me, the black fortress, the need to get back to my own booth, etc. If you are willing to pay $700 – $800 for your phone, then paying $600 for the Oculus Rift shouldn’t be such a big deal. And then you need to spend another $900 or more for a PC that will run the rift for you, but then at least you’ll have an awesome gaming machine.

Or you could also just wait for the HTC Vive Pre which has identical specs and feels just as nice and even has its own space game at launch called Elite: Dangerous. While the Oculus booth was targeted at fans, in large part, the Vive was shown in two different places to different audiences. A traveling HTC Vive bus pulled out tents and set up on the corner opposite Convention Hall North. This was for fans to try out the system and involved an hour wait for outdoor demos while demos inside the bus required signing up. I went down the street the the Wynn Hotel where press demos run by the marketing team were being organized in one of the hotel ballrooms. No engineers to talk to, sadly.

Whereas Oculus’s major announcement was about pricing and availability as well as opening up pre-orders, HTC’s announcement was about a technology breakthrough that didn’t really seem like much of a breakthrough. A color camera was placed on the front of HMD that outlines real-world objects around the player in order, among other things, to help the player avoid bumping into things when using the Vive Pre with the Lighthouse peripherals in order to walk around a VR experience.

vive pre

The Lighthouse experience is cool but the experience I most enjoyed was playing Elite: Dangerous with two mounted joysticks. This is a game I’ve played on the DK2 until it stopped working with the DK2 following my upgrade to Windows 10 (which as a Microsoft MVP I’m pretty much required to do) so I was pretty surprised to see the game in the HTC press room and even more surprised when I spent an hour chatting away happily to one of ED’s marketing people.

So this is a big tangent but here’s what I think happened and why the ED Oculus support became rocky a few months ago. Oculus appears to have started courting Eve: Valkyrie a while back, even though Elite: Dangerous was the more mature game. Someone must have decided that you don’t need two space games for one device launch, and so ED drifted over to the HTC Vive camp. And suddenly, support for the DK2 went on the backburner at ED while Oculus made breaking changes in their SDK release and many people who had gotten ED to play with the Rift or gotten the Rift to play with ED were sorely disappointed. At this point, you can make Elite: Horizons (the upgrade from ED) work in VR with Oculus but it is tricky and not documented. You have to download SteamVR, even if you didn’t buy Elite: Horizons from Stream, and jury rig your monitor settings to get everything running well in the Oculus direct mode. Needless to say, it’s clear that Elite’s games are going to run much more nicely if you buy Steam’s Vive and run it through Steam.

As for comparing Oculus Rift and HTC Vive Pre, it’s hard to say. They have the same specs. They both will need powerful computers to play on, so the cost of ownership goes beyond simply buying the HMD. Oculus has the touch controllers, but we don’t really know when they will be ready. HTC Vive has the Lighthouse peripherals that allow you to walk around and the specialized Steam controllers, but we don’t know how much they will cost.

For the moment, then, the best way to choose between the two VR devices comes down to which space flying game you think you would like more. Elite: Dangerous is mainly a community exploration game with combat elements. Eve: Valkyrie is a space combat game with exploration elements. Beyond that, Palmer Luckey did get the ball rolling on this whole VR thing, so all other things being equal, mutatis mutandis, you should probably reward him with your gold. Personally, though, I really love Elite: Horizons and being able to walk around in VR.

WP_20160109_09_39_20_Pro

But then again, one could always wait for PlayStation VR (the head-mounted display formerly known as Project Morpheus). The PlayStation VR demo was hidden in the back of the PlayStation demos, which in turn was at the back of the Sony booth which was at the far corner of the Las Vegas Convention Center North Hall. In other words, it was hard to find and a hike to get to. Once you go to it, though, it became clear that this was, in the scheme of things, a small play for the extremely diversified Sony. There wasn’t really enough room for the four demos Sony was showing and the lines were extremely compressed.

Which is odd because, for me at least, the PlayStation VR was the only thing I wanted to see. It’s by far the prettiest of the four big VR systems. While the resolution is slightly lower than that of the Oculus Rift or HTC Vive Pre, the frame rate is higher. Additionally, you don’t need to purchase a $900 computer to play it. You just need a PlayStation 4. The PlayStation Move controllers, as a bonus, finally make sense as VR controllers.

Best of all, there’s a good chance that PlayStation will end up having the best VR games (including Eve: Valkyrie) because those relationships already exist. Oculus and HTC Vive will likely clean up on the indie-game market since their dev and deployment story is likely going to be much simpler than Sony’s.

WP_20160108_10_04_34_Pro__highres

I waited forty minutes to play the newest The London Heist demo. In it, I rode shotgun in a truck next to a London thug as motorcycles and vans with machine gun wielding riders passed by and shot at me. I shot back, but strangely the most fascinating part for me was opening the glove compartment with the Move controllers and fiddling with the radio controls.

Prepare for another digression or just skip ahead if you like. While I was using Playstion Move controllers (those two lit up things in the picture above that look like neon ice-cream cones) in the Sony booth to change the radio station in my virtual van, BMW had a tent outside the convention center where they demoed a radio tuner in one of their cars that responded to hand gestures. One spun ones finger clockwise to scan through the radio channels. Two fingers pressed forward would pause a track. Wave would dismiss. Having worked with Kinect gestures for the past five years, I was extremely impressed with how good and intuitive these gestures were. They can even be re-programmed, by the way, to perform other functions. One night, I watched my boss close his eyes and perform these gestures from memory in order to lock them into his motor memory. They were that good, so if you have a lot of money, go buy all four VR sets as well as a BMW Series 7 so you can try out the radio.

But I digress. The London Heist is a fantastic game and the Playstation VR is pretty great. I only wish I had a better idea of when it is being released an how much it will cost.

Another great thing about the Sony PlayStation VR area was that it was out in the open unlike the VR demos from other companies. You could watch (for about 40 minutes, actually) as other people went through their moves. Eventually, we’ll start seeing a lot of these shots contrasting what people think they are doing in VR with what they are really doing. It starts off comically, but over time becomes very interesting as you realize the extent to which we are all constantly living out experiences in our imaginations and having imaginary conversations that no one around us is aware of – the rich interior life that a VR system is particularly suited to reveal to us.

WP_20160106_12_33_29_Pro

I found the OSVR demo almost by accident while walking around the outside of the Razer booth. There was a single small room with a glass window in the side where I could spy a demo going on. I had to wait for Tom’s Hardware to go though first, and also someone from Gizmodo, but after a while they finally invited me in and I got to talk to honest to goodness engineers instead of marketing people! OSVR demoed a 3D cut scene rather than an actual game and there was a little choppiness which may have been due to IR contamination from the overhead lights. I don’t really know. But for $299 it was pretty good and, if you aren’t already the proud owner of an Oculus DK2, which has the same specs, it may be the way to go. It also has upgradeable parts which is pretty interesting. If you are a hobbyist who wants to get a better understanding of how VR devices work – or if you simply want a relatively inexpensive way to get into VR – then this might be a great solution.

You could also go even cheaper, down to $99, and get a Samsung Gear VR (or one of a dozen or so similar devices) if you already have a $700 phone to fit into it. Definitely demo a full VR head-mounted display first, though, to make sure the more limited Gear VR-style experience is what you really want.

I also wanted to make quick mention of AntVR, which is an indie VR solution and Kickstarter that uses fiducial markers instead of IR emitters/receivers for position tracking.  It’s a full walking VR system that looked pretty cool.

If walking around with VR goggles seems a bit risky to you, you could also try a harness rig like Omni’s. Ignoring the fact that it looks like a baby’s jumporee, the Omni now comes with custom shoes so running inside it is easier. With practice, it looks like you can go pretty fast in one of these things and maybe even burn some serious calories. There were lots of discussions about where you would put something like this. It should work with any sort of VR setup: the demo systems were using Oculus DK2. While watching the demo I kept wanting to eat baby carrots for some reason.

jumporee

According to various forecasters, virtual reality is going to be as important a cultural touchstone for children growing up today as the Atari 2600 was for my generation.

To quickly summarize (or at least generalize) the benefits of each of the four main VR systems coming to market this year:

1. Oculus Rift – first developed and first to release a full package

2. HTC Vive Pre – best controllers and position tracking

3. PlayStation VR – best games

4. OSVR – best value

Ceci n’est pas une pipe bombe

homemade_clock

This is the picture of the homemade clock Ahmed Mohamed brought to his Irving, Texas high school. Apparently no one ever mistook it for a bomb, but they did suspect that it was made to look like a bomb and so they dragged the hapless boy off in handcuffs and suspended him for three days.

This is a strange case of perception versus reality in which the virtual bomb was never mistaken for a real bomb. Instead, what was identified was the fact that it was, in fact, only a bomb virtually and, as with all things virtual, therefore required some sort of explanation.

The common sympathetic explanation is that this isn’t a picture of a virtual bomb at all but rather a picture of a homemade clock. Ahmed recounts that he made the clock, in maker fashion, in order to show an engineering teacher because he had done robotics in middle school and wanted to get into a similar program in high school. Homemade clocks, of course, don’t require an explanation since they aren’t virtually anything other than themselves.

MagrittePipe

It turns out, however, that the picture at the top does not show a homemade maker clock. Various engineering types have examined the images and determined that it is in fact a disassembled clock from the 80’s.

The telling aspect is the DC power cord which doesn’t actually get used in homemade projects. Instead, anyone working with arduino projects typically (pretty much always) uses AA batteries. The clock components have also been tracked back to their original source, however, so the evidence seems pretty solid.

battery

The photo at the top shows not a virtual bomb nor a homemade clock but, in fact, a virtual homemade clock. That is, it was made to look like a homemade clock but was mistakenly believed to be something made to look like a homemade bomb.

[As a disclaimer about intentions, which is necessary because getting on the wrong side of this gets people in trouble, I don’t know Ahmed’s intentions and while I’m a fan of free speech I can’t say I actually believe in free speech having worked in marketing and I think Ahmed Mohammed looks absolutely adorable in his NASA t-shirt and I have no desire to be placed in company with those other assholes who have shown that this is not a real homemade clock but rather a reassembled 80’s clock and therefore question Ahmed’s motives whereas I refuse to try to get into a high schooler’s head, having two of my own and knowing what a scary place that can be … something, something, something … and while I can’t wholeheartedly support every tweet made by Richard Dawkins and have at times even felt in mild disagreement with things he and others have tweeted on twitter I will say that I find his book The Selfish Gene a really good read … etc, etc, … and for good measure fuck you FoxNews.]

 

micronta

The salient thing for me is that we all implicitly know that a real bomb isn’t supposed to look like a bomb. The authorities at Ahmed’s high school knew that immediately. Bombs are supposed to look like shoes or harmless tourist knickknacks. If you think it looks like a bomb, it obviously isn’t. So what does it mean to look like a bomb (to be virtually a bomb) but not be an actual bomb?

banana_tape

I covered similar territory once before in a virtual exhibit called les fruits dangereux and at the time concluded that virtual objects, like post-modern novels, involve bricolage and the combining of disparate elements in unexpected ways. For instance combining phones, electrical tape and fruit or combining clock parts and pencil cases. Disrupting categorical thinking at a very basic level makes people – especially authority people – suspicious and unhappy.

Which gets us back to racism which is apparently what has happened to Ahmed Mohammed who was led out of school in handcuffs in front of his peers – and we’re talking high school! and he wasn’t asking to be called “McLovin.” It’s pretty cruel stuff. The fear of racial mixing (socially or biologically) always raises it’s head and comes from the same desire to categorize people and things into bento box compartments. The great fear is that we start to acknowledge that we live in a continuum of types rather than distinct categories of people, races and objects. In the modern age, mass production makes all consumer objects uniform in a way that artisanal objects never were while census forms do the same for people.

Virtual reality will start by copying real world objects in a safe way. As with digital design, it will start with isomorphism to make people feel safe and comfortable. As people become comfortable, bricolage will take hold simply because, in a digital world rather than a commoditized/commodified world, mashups are easy. Irony and a bit of subversiveness will lead to bricolage with purpose as we find people’s fantasies lead them to combine digital elements in new and unexpected ways.

We can all predict augmented and virtual ways to press a digital button or flick through a digital menu projected in front of us in order to get a virtual weather forecast. Those are the sorts of experiences that just make people bored with augmented reality vision statements.

hieronymus.63a

The true promise of virtual reality and augmented reality is that they will break down our racial, social and commodity thinking. Mixed-reality has the potential to drastically change our social reality. How do social experiences change when the color of a person’s avatar tells you nothing real about them, when our social affordances no longer provide clues or shortcuts to understanding other people? In a virtual world, accents and the shoes people wear no longer tell us anything about their educational background or social status. Instead of a hierarchical system of discrete social values, we’ll live in a digital continuum.

That’s the sort of augmented reality future I’m looking forward to.

The important point in the Ahmed Mohammed case, of course, is that you shouldn’t arrest a teenager for not making a bomb.

How Will Battlestar Galactica End?

apollo

I’m such a loser.  Hours away from the BSG finale and I am still blogging about code.

So how will Battlestar Galactica end tonight?  I am hoping for a classic Dallas ending.  Apollo wakes up on the 1978 Galactica from a nightmare in which he is a sociopath politician named Tom Zarek on an alternate Galactica in which Dr. Z is never found and Earth is a wasteland.  It turns out that this is part of an evil plot devised by Count Iblis, but Starbuck, Apollo, Boomer, Lieutenant Zack and Muffit, Boxey’s Daggit, foil Iblis’s scheme and bring sanity back to the Galactica.  When Apollo tells Starbuck about their intimate relationship in his dream, it creates some awkwardness, but they work it out in a game of pyrimid and then go down to the gambling planet to get good and sloshed.

Can’t wait ’till tonight when I get to see how close I am.  It’s definitely a Crystal and Courvoisier kind of evening.

Rock Star

bobdylan 

Over the past week my family and I have been playing with an early Christmas present, an XBOX game called Rock Band.  This game one-ups the other popular rock simulation franchise, Guitar Hero, by allowing the player to perform on guitar, drums or with a mic.  A sort of guitar hero meets karaoke, it even allows different players to work together in forming a rock band either over the Internet or in their living room, side-by-side.

I have adopted the guitar as my own instrument for acquiring fame.  I have already mastered Don’t Fear the Reaper (no cowbell, unfortunately), Wanted Dead or Alive, Mississippi Queen and Blitzkrieg Bop, at medium difficulty, and am currently rehearsing Suffragette City, which has a wicked A-X-Y riff (XBOX control buttons, naturally, not notes) that I cannot seem to get the hang of.

The game is wonderful, though as I play through I am flabbergasted by the number of songs I do not recognize.  Who are Vagiant, or Anarchy Club, or Crooked X?  There are other groups I know by name, like Weezer, Radiohead, and Foo Fighters, but until now I couldn’t have named a song by any of them.   My knowledge of popular music seems to have ended sometime in the late eighties, and there is a decade and a half lacuna following that which I am loathe to fill.  Added to this the great number of metal anthems in this game combined with my ignorance of the Bon Jovi catalogue and the Metallica repertoire (though I do know — who wouldn’t — Rush’s Tom Sawyer and Black Sabbath’s Paranoid),  and you may get a sense of the cultural irrelevance that washes over me as each new playlist is thrown onto my screen.

Worse, I believe I threw out my back during the extra points phase of Nine Inch Nails’ The Hand That Feeds, and now rest supine, forbidden to play until my back heals or eventually snaps back into place.  How do aging rockers do it?  Keith Richards apparently has fresh blood transfused into his system every few years, but I believe his case is anomalous. 

Playing at being a rock star does have that element of grasping at one’s lost youth to it.  Like the elixir imbibed by Richards, rock rhythm and tonal inflections are absorbed by the air-guitarist and, for a brief time, he undergoes a spiritual transmutation — one that must be accomplished in privacy, of course, lest reality, or perchance an unfortunately placed mirror, dispel the glamour. 

Much of the current literature — often found in blogs, of course — discusses video games and virtual worlds as a sort of escapism.  The notion is that the unfulfilled middle-manager may find an emotional outlet for his work-induced frustrations in virtual games such as World of Warcraft, where everyone and anyone has the opportunity to be a hero.

I am not sure that this quite captures the phenomenon, however.  With Rock Band, the goal is not so much to escape one’s reality but rather to participate in a different one — more of a pull than a push, so to speak.  Plotinus, though in a different context, spoke of it as a hypostatic union; a union between oneself and a more perfect version of oneself; something that has the character of fulfilling one’s nature rather than erasing it.

Such is my feeling when playing air-guitar on the XBOX.  I don’t want to be someone else, but instead simply want to release an aspect of myself that requires lowering the bar a bit, through technology, and closing the gap between myself and the gods of rock and roll.  Surely this is the origin of all technology since Nimrod’s tower, brought down by Jehovah for its impertinence.  Technology lifts us up, giving us health and the promise of immortality through medicine, cleverness through information technology, courage through online role-playing, and happiness through pharmacology.  Technology removes our frailties, leaving us as we were meant to be before the fall: young and immortal.

Speaking of growing old, I attended a Bob Dylan concert a few months ago.  Bob has been going through a bit of a revival, his U.S. tour coinciding with a documentary by Martin Scorsese and even a feature length biopic.  I have not been able to find an adequate way to write about the event, which I and my companions walked out of.  To say that Dylan couldn’t sing seems to be missing the point, since this charge has always been leveled against him.  And to say that I did not like the new style he was playing also makes me sound like those who criticized Bob for going electric back in the day.

I might draw the contrast between Bob and Elvis Costello, who opened for him.  Elvis played several songs solo, both old and new, only changing guitars on occasion to fit the piece.  He was the Elvis I imagined, comfortable on stage, able to work the audience, singing in his off-key way to perfection.  He was the archetype of the solo college musician, fitting complex lyrics into a structured form meant to invoke interlaced feelings of sympathy and alienation — the alienation of the artist — in his audience.  Like a great beast, the audience responded to his coaxing, and he guided us through a peculiar journey and then deposited us gently when his set was over.

According to Jung, archetypes recur across cultures, and we seek them out, attempting to fill the recesses established in our minds.  Most of all we seek heroes, not because we seek to be heroes ourselves, necessarily, but because we have an inchoate sense that there must always be heroes.  Shakespeare provided heroes, sometimes twisted, sometimes broken, but heroes nonetheless, on the stage.  The modern rock star transforms himself to fulfill this role established for him.  The form required for the hero changes over time, and changes with context; Dylan was always able to adapt to these changing roles.  He served as a cipher, reflecting the image that people required of him.

The Scorsese documentary, built around a ten hour interview with Dylan, portrays a man who sees none of the depth in his own lyrics that others impute to them.  Perhaps he is simply being playful.  He discusses a radio interview in which the host asks Dylan whether — actually, insists that — A Hard Rain’s a-Gonna Fall is about nuclear ash and the madness of nuclear proliferation, to which Dylan responds that the song is about rain, which is sometimes heavy, you know. 

Again and again, however, Dylan was able to capture the spirit of his times, several times, and people found in his lyrics answers to their questions and anxieties.  My favorite Dylan album is Desire, which is a combination of electric protest songs and wild fantasies, all within a cowboy motif.  I can’t say what questions he answers for me in this, but I do know that I return to it again and again, and it puts me in a happy place.

According to Giambattista Vico, the 18th century Neapolitan philologist, the secret of metaphor is that it is not based on similarity, but rather on identity.  The secret of the hero is not that he makes himself resemble the classic hero.  He becomes that hero, transforming himself as needed, internally, until identity is achieved.  “The true war chief … is the Godfrey that Torquato Tasso imagines; and all the chiefs who do not conform throughout to Godfrey are not true chiefs of war” as Vico said.

Dylan accomplished something similar, adopting the argot of protest singers like Pete Seeger when this was required of him.  As the times changed, he transformed himself again into the rebellious Bob, with dark glasses and an attitude, indifferent to the complaints about his going electric and “selling out,” when in fact his ear was simply better attuned than that of those around him.  The Desire period marked him as a reclusive genius, when that was the thing to be.  In the 80’s and 90’s, he identified himself with the Christian revival, while his music turned bluesy in a time when we wanted heroes whose rough voices where inhabited by old wisdom and hard-earned experience.

The new Dylan (which is your favorite Dylan?) doesn’t speak to me, however.  He has a sort of be-bop band in conservative zoot-suits accompanying him, and the music matches the look.  The music is pleasant enough, punctuated by Bob’s gravelly voice hammering out quick phrases like “dondenktwysanizalri” or “juzlikawama.”  But what it means, and who Bob Dylan represents, is unclear to me.  I hold out the possibility that he is simply ahead of the curve once again — but what sort of culture requires a hero who is mostly pleasant and barely comprehensible?  Is this the zeitgeist of the 00’s?

Like my back, my plastic XBOX Stratocaster has also given out.  After only a few hours of playing, the strummer is now mushy, and I am unable to get through the rapid 12 note riffs that seem to infest all the metal songs currently on my playlist like little roaches.  I have gone online and found a fix that requires me to remove the back of the guitar (there are 20 odd screws, and I am grateful for the gift of a Christmas past: a power drill with bit attachments) and than reset a tension bar inside the guitar mechanism, but this seems to only work for about five hours before the strummer becomes mushy again.  Apparently there is a known problem with the early Rock Band Stratocasters, and Activision is allowing people to send their bad guitars in for a replacement.  Since I am currently in a state of rock star disability, I think I may take advantage of this, and with luck by the time my guitar is healed, my back will be, too.

SophiaBot: What I’ve been working on for the past month…

I have been busy in my basement constructing a robot with which I can have conversations and play games.  Except that the robot is more of a program, and I didn’t build the whole thing up from scratch, but instead cobbled together pieces that other people have created.  I took an Eliza-style interpreter written by Nicholas H.Tollervey (this is the conversation part) along with some scripted dialogs by Dr. Richard S. Wallace and threw it together with a Z-machine program written by Jason Follas, which allows my bot to play old Infocom games like Zork and The Hitchhiker’s Guide to the Galaxy.  I then wrapped these up in a simple workflow and added some new Vista\.NET 3.0 speech recognition and speech synthesis code so the robot can understand me.

I wrote an article about it for CodeProject, a very nice resource that allows developers from around the world to share their code and network.  The site requires registration to download code however, so if you want to play with the demo or look at the source code, you can also download them from this site.

Mr. Tollervey has a succint article about the relationship between chatterboxes and John Searle’s Chinese Box problem, which obviates me from responsibility for discussing the same.

Instead, I’ll just add some quick instructions:

 

The application is made up of a text output screen, a text entry field, and a default enter button. The initial look and feel is that of an IBX XT theme (the first computer I ever played on). This can be changed using voice commands, which I will cover later. There are three menus initially available. The File menu allows the user to save a log of the conversation as a text file. The Select Voice menu allows the user to select from any of the synthetic voices installed on her machine. Vista initially comes with “Anna”. Windows XP comes with “Sam”. Other XP voices are available depending on which versions of Office have been installed over the lifetime of that particular instance of the OS. If the user is running Vista, then the Speech menu will allow him to toggle speech synthesis, dictation, and the context-free grammars. By doing so, the user will have the ability to speak to the application, as well as have the application speak back to him. If the user is running XP, then only speech synthesis is available, since some of the features provided by .NET 3.0 and consumed by this application do not work on XP.

The appearance menu will let you change the look and feel of the text screen.  I’ve also added some pre-made themes at the bottom of the appearnce menu.  If, after chatting with SophiaBot for a while, you want to play a game, just type or say “Play game.”  SophiaBot will present you with a list of the games available (you can add more, actually, simply by dropping additional game files you find on the internet into the Program Files\Imaginative Universal\SophiaBot\Game Data\DATA folder (Jason’s Z-Machine implementation plays games that use version 3 and below of the game engine.  I’m looking (rather lazily) into how to support later versions.  You can go here to download more Zork-type games.  During a game, type or say “Quit” to end your session. “Save” and “Restore” keep track of your current position in the game, so you can come back later and pick up where you left off.

Speech recognition in Vista has two modes: dictation and context-free recognition. Dictation uses context, that is, an analysis of preceding words and words following a given target of speech recognition, in order to determine what word was intended by the speaker. Context-free speech recognition, by way of contrast, uses exact matches and some simple patterns in order to determine if certain words or phrases have been uttered. This makes context-free recognition particularly suited to command and control scenarios, while dictation is particularly suited to situations where we are simply attempting to translate the user’s utterances into text.

You should begin by trying to start up a conversation with Sophia using the textbox, just to see how it works, as well as her limitations as a conversationalist. Sophia uses certain tricks to appear more lifelike. She throws out random typos, for one thing. She also is a bit slower than a computer should really be. This is because one of the things that distinguish computers from people is the way they process information — computers do it quickly, and people do it at a more leisurely pace. By typing slowly, Sophia helps the user maintain his suspension of disbelief. Finally, if a text-to-speech engine is installed on your computer, Sophia reads along as she types out her responses. I’m not certain why this is effective, but it is how computer terminals are shown to communicate in the movies, and it seems to work well here, also. I will go over how this illusion is created below.

In Command\AIML\Game Lexicon mode, the application generates several grammar rules that help direct speech recognition toward certain expected results. Be forewarned: initially loading the AIML grammars takes about two minutes, and occurs in the background. You can continue to touch type conversations with Sophia until the speech recognition engine has finished loading the grammars and speech recognition is available. Using the command grammar, the user can make the computer do the following things: LIST COLORS, LIST GAMES, LIST FONTS, CHANGE FONT TO…, CHANGE FONT COLOR TO…, CHANGE BACKGROUND COLOR TO…. Besides the IBM XT color scheme, a black papyrus font on a linen background also looks very nice. To see a complete list of keywords used by the text-adventure game you have chosen, say “LIST GAME KEYWORDS.” When the game is initially selected, a new set of rules is created based on different two word combinations of the keywords recognized by the game, in order to help speech recognition by narrowing down the total number of phrases it must look for.

In dictation mode, the underlying speech engine simply converts your speech into words and has the core SophiaBot code process it in the same manner that it processes text that is typed in. Dictation mode is sometimes better than context-free mode for non-game speech recognition, depending on how well the speech recognition engine installed on your OS has been trained to understand your speech patterns. Context-free mode is typically better for game mode. Command and control only works in context-free mode.

48% of Americans Reject Darwinian Evolution

 

A new Newsweek poll reveals frightening data about the curious disjunct between faith and science among Americans.  Pundits have attributed these results to anything from poor science education in pre-K programs to global warming.  According to the poll, while 51% percent of Americans still ascribe to Darwin’s theory of gradual evolution through adaptation, an amazing 42% continue to cleave to Lamarkianism, while only 6% believe in Punctuated Equilibrium. 1% remain uncommitted and are waiting to hear more before they come to a final decision.

This has led me to wonder what else Americans believe:

The 2002 Roper Poll found that 48% of americans believe in UFO’s, while 37% believe that there has been first hand contact between aliens and humans.  25% of Americans believe in alien abductions, while approximately 33% believe that humans are the only intelligent life in the universe, and that all the UFO stuff is bunk.

The 33% of people who ascribe to the anthropocentric view of the universe corresponds numerically with the 33% of Americans who opposed the recent deadline for troop withdrawal from Iraq (PEW Research center poll).   According to the Gallup poll, in 1996 33% of Americans thought they would become rich someday.  By 2003, this number had dropped to 31%.  According to a Scripps Howard/Ohio University poll, 33% of the American public suspects that federal officials assisted in the 9/11 terrorist attacks or took no action to stop them so the United States could go to war in the Middle East.  A Harris poll discovered that in 2004, 33% of adult Americans considered themselves Democrats.

PEW says that as of 2004, 33 million American internet users had reviewed or rated something as part of an online rating system.  33 million Americans were living in povery in 2001, according to the U.S. Census Bureau.  According to PEW, in 2006 33 million Americans had heard of VOIP.  Each year, 33 million Americans use mental health services or services to treat their problems and illnesses resluting from alcohol, inappropirate use of prescription medications, or illegal drugs.  The New York Times says that out of 33 countries, Americans are least likely to believe in evolution.  Researchers estimate that 33% of Americans born in 2000 will develop diabetes.  In the same year, 33 million Americans lost their jobs.

CBS pollsters discovered that 22% of Americans have seen or felt the presence of a ghost.  48% believe in ghosts.  ICR says 48% of Americans oppose embryonic stem-cell research.  CBS finds that 61% support embryonic stem-cell research.  There is no poll data available on whether they believe that embryos used for stem-cell research will one day become ghosts themselves.

82% of Americans believe that global warming is occuring, according to Fox News/Opinion Dynamics.  79% believe people’s behavior has contributed to global warming.  89% do not believe the U.S. government staged or faked the Apollo moon landing, according to Gallup.  Gallup also found that 41% of Americans believe in ESP, 25% believe in Astrology, 20% believe in reincarnation, while only 9% believe in channeling.  A USA TODAY/ABC News/Stanford University Medical Center poll found that 5% of American adults have turned to acupuncture for pain relief.

According to Gallup, 44% of Americans go out of their way to see movies starring Tom Hanks.  34% go out of their way to avoid movies starring Tom Cruise.  Only 18% go out of their way to avoid Angelina Jolie.

Do Computers Read Electric Books?

In the comments section of a blog I like to frequent, I have been pointed to an article in the International Herald about Pierre Bayard’s new book,  How to Talk About Books You Haven’t Read.

Bayard recommends strategies such as abstractly praising the book, offering silent empathy regarding someone else’s love for the book, discussing other books related to the book in question, and finally simply talking about oneself.  Additionally, one can usually glean enough information from reviews, book jackets and gossip to sustain the discussion for quite a while.

Students, he noted from experience, are skilled at opining about books they have not read, building on elements he may have provided them in a lecture. This approach can also work in the more exposed arena of social gatherings: the book’s cover, reviews and other public reaction to it, gossip about the author and even the ongoing conversation can all provide food for sounding informed.

I’ve recently been looking through some AI experiments built on language scripts, based on the 1966 software program Eliza, which used a small script of canned questions to maintain a conversation with computer users.  You can play a web version of Eliza here, if you wish.  It should be pointed out that the principles behind Eliza are the same as those that underpin the famous Turing Test.  Turing proposed answering the question can machines think by staging an ongoing experiment to see if machines can imitate thinking.  The proposal was made in his 1950 paper Computing Machinery and Intelligence:

The new form of the problem can be described in terms of a game which we call the ‘imitation game.” It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either “X is A and Y is B” or “X is B and Y is A.” The interrogator is allowed to put questions to A and B thus:

C: Will X please tell me the length of his or her hair?

Now suppose X is actually A, then A must answer. It is A’s object in the game to try and cause C to make the wrong identification. His answer might therefore be:

“My hair is shingled, and the longest strands are about nine inches long.”

In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary. The object of the game for the third player (B) is to help the interrogator. The best strategy for her is probably to give truthful answers. She can add such things as “I am the woman, don’t listen to him!” to her answers, but it will avail nothing as the man can make similar remarks.

We now ask the question, “What will happen when a machine takes the part of A in this game?” Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, “Can machines think?”

The standard form of the current Turing experiments is something called a chatterbox application.  Chatterboxes abstract the mechanism for generating dialog from the dialog scripts themselves by utilizing a set of rules written in a common format.  The most popular format happens to be an XML standard called AIML (Artificial Intelligence Markup Language).

What I’m interested in, at the moment, is not so much whether I can write a script that will fool people into thinking they are talking with a real person, but rather whether I can write a script that makes small talk by discussing the latest book.  If I can do this, it should validate Pierre Bayard’s proposal, if not Alan Turing’s.