Speech Recognition And Synthesis Managed APIs in Windows Vista: Part III

Voice command technology, as exemplified in Part II, is probably the most useful and most easy to implement aspect of the Speech Recognition functionality provided by Vista.  In a few days of work, any current application can be enabled to use it, and the potential for streamlining workflow and making it more efficient is truly breathtaking.  The cool factor, of course, is also very high.

Having grown up watching Star Trek reruns, however, I can’t help but feel that the dictation functionality is much more interesting than the voice command functionality.  Computers are meant to be talked to and told what to do, as in that venerable TV series, not cajoled into doing tricks for us based on finger motions over a typewriter.  My long-term goal is to be able to code by talking into my IDE in order to build UML diagrams and then, at a word, turn that into an application.  What a brave new world that will be.  Toward that end, the SR managed API provides the DictationGrammar class.

Whereas the Grammar class works as a gatekeeper, restricting the phrases that get through to the speech recognized handler down to a select set of rules, the DictateGrammar class, by default, kicks out the jams and lets all phrases through to the recognized handler.

In order to make Speechpad a dictation application, we will add the default DicatateGrammar object to the list of grammars used by our speech recognition engine.  We will also add a toggle menu item to turn dictation on and off.  Finally, we will alter the SpeechToAction() method in order to insert any phrases that are not voice commands into the current Speechpad document as text.  Create an local instance of DictateGrammar for our Main form, and then instantiate it in the Main constructor.  Your code should look like this:

	#region Local Members
		
        private SpeechSynthesizer synthesizer = null;
        private string selectedVoice = string.Empty;
        private SpeechRecognitionEngine recognizer = null;
        private DictationGrammar dictationGrammar = null;
        
        #endregion
        
        public Main()
        {
            InitializeComponent();
            synthesizer = new SpeechSynthesizer();
            LoadSelectVoiceMenu();
            recognizer = new SpeechRecognitionEngine();
            InitializeSpeechRecognitionEngine();
            dictationGrammar = new DictationGrammar();
        }
        

Create a new menu item under the Speech menu and label it “Take Dictation“.  Name it takeDictationMenuItem for convenience. Add a handler for the click event of the new menu item, and stub out TurnDictationOn() and TurnDictationOff() methods.  TurnDictationOn() works by loading the local dictationGrammar object into the speech recognition engine. It also needs to turn speech recognition on if it is currently off, since dictation will not work if the speech recognition engine is disabled. TurnDictationOff() simply removes the local dictationGrammar object from the speech recognition engine’s list of grammars.

		
        private void takeDictationMenuItem_Click(object sender, EventArgs e)
        {
            if (this.takeDictationMenuItem.Checked)
            {
                TurnDictationOff();
            }
            else
            {
                TurnDictationOn();
            }
        }

        private void TurnDictationOn()
        {
            if (!speechRecognitionMenuItem.Checked)
            {
                TurnSpeechRecognitionOn();
            }
            recognizer.LoadGrammar(dictationGrammar);
            takeDictationMenuItem.Checked = true;
        }

        private void TurnDictationOff()
        {
            if (dictationGrammar != null)
            {
                recognizer.UnloadGrammar(dictationGrammar);
            }
            takeDictationMenuItem.Checked = false;
        }
        

For an extra touch of elegance, alter the TurnSpeechRecognitionOff() method by adding a line of code to turndictation off when speech recognition is disabled:

        TurnDictationOff();

Finally, we need to update the SpeechToAction() method so it will insert any text that is not a voice command into the current Speechpad document.  Use the default statement of the switch control block to call the InsertText() method of the current document.

        
        private void SpeechToAction(string text)
        {
            TextDocument document = ActiveMdiChild as TextDocument;
            if (document != null)
            {
                DetermineText(text);
                switch (text)
                {
                    case "cut":
                        document.Cut();
                        break;
                    case "copy":
                        document.Copy();
                        break;
                    case "paste":
                        document.Paste();
                        break;
                    case "delete":
                        document.Delete();
                        break;
                    default:
                        document.InsertText(text);
                        break;
                }
            }
        }

        

With that, we complete the speech recognition functionality for Speechpad. Now try it out. Open a new Speechpad document and type “Hello World.”  Turn on speech recognition.  Select “Hello” and say delete.  Turn on dictation.  Say brave new.

This tutorial has demonstrated the essential code required to use speech synthesis, voice commands, and dictation in your .Net 2.0 Vista applications.  It can serve as the basis for building speech recognition tools that take advantage of default as well as custom grammar rules to build adanced application interfaces.  Besides the strange compatibility issues between Vista and Visual Studio, at the moment the greatest hurdle to using the Vista managed speech recognition API is the remarkable dearth of documentation and samples.  This tutorial is intended to help alleviate that problem by providing a hands on introduction to this fascinating technology.

Speech Recognition And Synthesis Managed APIs In Windows Vista: Part II


Playing with the speech synthesizer is a lot of fun for about five minutes (ten if you have both Microsoft Anna and Microsoft Lila to work with)  — but after typing “Hello World” into your Speechpad document for the umpteenth time, you may want to do something a bit more challenging.  If you do, then it is time to plug in your expensive microphone, since speech recognition really works best with a good expensive microphone.  If you don’t have one, however, then go ahead and plug in a cheap microphone.  My cheap microphone seems to work fine.  If you don’t have a cheap microphone, either, I have heard that you can take a speaker and plug it into the mic jack of your computer, and if that doesn’t cause an explosion, you can try talking into it.


While speech synthesis may be useful for certain specialized applications, voice commands, by cantrast, are a feature that can be used to enrich any current WinForms application. With the SR Managed API, it is also easy to implement once you understand certain concepts such as the Grammar class and the SpeechRecognitionEngine.


We will begin by declaring a local instance of the speech engine and initializing it. 

	#region Local Members

private SpeechSynthesizer synthesizer = null;
private string selectedVoice = string.Empty;
private SpeechRecognitionEngine recognizer = null;

#endregion

public Main()
{
InitializeComponent();
synthesizer = new SpeechSynthesizer();
LoadSelectVoiceMenu();
recognizer = new SpeechRecognitionEngine();
InitializeSpeechRecognitionEngine();
}

private void InitializeSpeechRecognitionEngine()
{
recognizer.SetInputToDefaultAudioDevice();
Grammar customGrammar = CreateCustomGrammar();
recognizer.UnloadAllGrammars();
recognizer.LoadGrammar(customGrammar);
recognizer.SpeechRecognized +=
new EventHandler<SpeechRecognizedEventArgs>(recognizer_SpeechRecognized);
recognizer.SpeechHypothesized +=
new EventHandler<SpeechHypothesizedEventArgs>(recognizer_SpeechHypothesized);
}

private Grammar CreateCustomGrammar()
{
GrammarBuilder grammarBuilder = new GrammarBuilder();
grammarBuilder.Append(new Choices(“cut”, “copy”, “paste”, “delete”));
return new Grammar(grammarBuilder);
}


The speech recognition engine is the main workhorse of the speech recognition functionality.  At one end, we configure the input device that the engine will listen on.  In this case, we use the default device (whatever you have plugged in), though we can also select other inputs, such as specific wave files.  At the other end, we capture two events thrown by our speech recognition engine.  As the engine attempts to interpret the incoming sound stream, it will throw various “hypotheses” about what it thinks is the correct rendering of the speech input.  When it finally determines the correct value, and matches it to a value in the associated grammar objects, it throws a speech recognized event, rather than a speech hypothesized event.  If the determined word or phrase does not have a match in any associated grammar, a speech recognition rejected event (which we do not use in the present project) will be thrown instead.


In between, we set up rules to determine which words and phrases will throw a speech recognized event by configuring a Grammar object and associating it with our instance of the speech recognition engine.  In the sample code above, we configure a very simple rule which states that a speech recognized event will be thrown if any of the following words: “cut“, “copy“, “paste“, and “delete“, is uttered.  Note that we use a GrammarBuilder class to construct our custom grammar, and that the syntax of the GrammarBuilder class closely resembles the syntax of the StringBuilder class.


This is the basic code for enabling voice commands for a WinForms application.  We will now enhance the Speechpad application by adding a menu item to turn speech recognition on and off,  a status bar so we can watch as the speech recognition engine interprets our words, and a function that will determine what action to take if one of our key words is captured by the engine.


Add a new menu item labeled “Speech Recognition” under the “Speech” menu item, below “Read Selected Text” and “Read Document”.  For convenience, name it speechRecognitionMenuItem.  Add a handler to the new menu item, and use the following code to turn speech recognition on and off, as well as toggle the speech recognition menu item.  Besides the RecognizeAsync() method that we use here, it is also possible to start the engine synchronously or, by passing it a RecognizeMode.Single parameter, cause the engine to stop after the first phrase it recognizes. The method we use to stop the engine, RecognizeAsyncStop(), is basically a polite way to stop the engine, since it will wait for the engine to finish any phrases it is currently processing before quitting. An impolite method, RecognizeAsyncCancel(), is also available — to be used in emergency situations, perhaps.

        private void speechRecognitionMenuItem_Click(object sender, EventArgs e)
{
if (this.speechRecognitionMenuItem.Checked)
{
TurnSpeechRecognitionOff();
}
else
{
TurnSpeechRecognitionOn();
}
}

private void TurnSpeechRecognitionOn()
{
recognizer.RecognizeAsync(RecognizeMode.Multiple);
this.speechRecognitionMenuItem.Checked = true;
}

private void TurnSpeechRecognitionOff()
{
if (recognizer != null)
{
recognizer.RecognizeAsyncStop();
this.speechRecognitionMenuItem.Checked = false;
}
}


We are actually going to use the RecognizeAsyncCancel() method now, since there is an emergency situation. The speech synthesizer, it turns out, cannot operate if the speech recognizer is still running. To get around this, we will need to disable the speech recognizer at the last possible moment, and then reactivate it once the synthesizer has completed its tasks. We will modify the ReadAloud() method to handle this.


private void ReadAloud(string speakText)
{
try
{
SetVoice();
recognizer.RecognizeAsyncCancel();
synthesizer.Speak(speakText);
recognizer.RecognizeAsync(RecognizeMode.Multiple);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}

}

The user now has the ability to turn speech recognition on and off. We can make the application more interesting by capturing the speech hypothesize event and displaying the results to a status bar on the Main form.  Add a StatusStrip control to the Main form, and a ToolStripStatusLabel to the StatusStrip with its Spring property set to true.  For convenience, call this label toolStripStatusLabel1.  Use the following code to handle the speech hypothesized event and display the results:

private void recognizer_SpeechHypothesized(object sender, SpeechHypothesizedEventArgs e)
{
GuessText(e.Result.Text);
}

private void GuessText(string guess)
{
toolStripStatusLabel1.Text = guess;
this.toolStripStatusLabel1.ForeColor = Color.DarkSalmon;
}


Now that we can turn speech recognition on and off, as well as capture misinterpretations of the input stream, it is time to capture the speech recognized event and do something with it.  The SpeechToAction() method will evaluate the recognized text and then call the appropriate method in the child form (these methods are accessible because we scoped them internal in the Textpad code above).  In addition, we display the recognized text in the status bar, just as we did with hypothesized text, but in a different color in order to distinguish the two events.


private void recognizer_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
{
string text = e.Result.Text;
SpeechToAction(text);
}

private void SpeechToAction(string text)
{
TextDocument document = ActiveMdiChild as TextDocument;
if (document != null)
{
DetermineText(text);

switch (text)
{
case “cut”:
document.Cut();
break;
case “copy”:
document.Copy();
break;
case “paste”:
document.Paste();
break;
case “delete”:
document.Delete();
break;
}
}
}

private void DetermineText(string text)
{
this.toolStripStatusLabel1.Text = text;
this.toolStripStatusLabel1.ForeColor = Color.SteelBlue;
}


Now let’s take Speechpad for a spin.  Fire up the application and, if it compiles, create a new document.  Type “Hello world.”  So far, so good.  Turn on speech recognition by selecting the Speech Recognition item under the Speech menu.  Highlight “Hello” and say the following phrase into your expensive microphone, inexpensive microphone, or speaker: delete.  Now type “Save the cheerleader, save the”.  Not bad at all.

Speech Recognition And Synthesis Managed APIs In Windows Vista: Part I




VistaSpeechAPIDemo.zip – 45.7 Kb


VistaSpeechAPISource.zip – 405 Kb


Introduction


One of the coolest features to be introduced with Windows Vista is the new built in speech recognition facility.  To be fair, it has been there in previous versions of Windows, but not in the useful form in which it is now available.  Best of all, Microsoft provides a managed API with which developers can start digging into this rich technology.  For a fuller explanation of the underlying technology, I highly recommend the Microsoft whitepaper. This tutorial will walk the user through building a common text pad application, which we will then trick out with a speech synthesizer and a speech recognizer using the .Net managed API wrapper for SAPI 5.3. By the end of this tutorial, you will have a working application that reads your text back to you, obeys your voice commands, and takes dictation. But first, a word of caution: this code will only work for Visual Studio 2005 installed on Windows Vista. It does not work on XP, even with .NET 3.0 installed.

Background


Because Windows Vista has only recently been released, there are, as of this writing, several extant problems relating to developing on the platform.  The biggest hurdle is that there are known compatibility problems between Visual Studio and Vista.  Visual Studio.NET 2003 is not supported on Vista, and there are currently no plans to resolve any compatibility issues there.  Visual Studio 2005 is supported,  but in order to get it working well, you will need to make sure you also install service pack 1 for Visual Studio 2005.  After this, you will also need to install a beta update for Vista called, somewhat confusingly, “Visual Studio 2005 Service Pack 1 Update for Windows Vista Beta”.  Even after doing all this, you will find that all the new cool assemblies that come with Vista, such as the System.Speech assembly, still do not show up in your Add References dialog in Visual Studio.  If you want to have them show up, you will finally need to add a registry entry indicating where the Vista dll’s are to be found.  Open the Vista registry UI by running regedit.exe in your Vista search bar.  Add the following registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\AssemblyFolders\v3.0 Assemblies with this value: C:\\Program Files\\Reference Assemblies\\Microsoft\\Framework\\v3.0. (You can also install it under HKEY_CURRENT_USER, if you prefer.)  Now, we are ready to start programming in Windows Vista.

Before working with the speech recognition and synthesis functionality, we need to prepare the ground with a decent text pad application to which we will add on our cool new toys. Since this does not involve Vista, you do not really have to follow through this step in order to learn the speech recognition API.  If you already have a good base application, you can skip ahead to the next section, Speechpad, and use the code there to trick out your app.  If you do not have a suitable application at hand, but also have no interest in walking through the construction of a text pad application, you can just unzip the source code linked above and pull out the included Textpad project.  The source code contains two Visual Studio 2005 projects, the Textpad project, which is the base application for the SR functionality, and Speechpad, which includes the final code.


All the same, for those with the time to do so, I feel there is much to gain from building an application from the ground up. The best way to learn a new technology is to use it oneself and to get one’s hands dirty, as it were, since knowledge is always more than simply knowing that something is possible; it also involves knowing how to put that knowledge to work. We know by doing, or as Giambattista Vico put it, verum et factum convertuntur.


Textpad


Textpad is an MDI application containing two forms: a container, called Main.cs, and a child form, called TextDocument.csTextDocument.cs, in turn, contains a RichTextBox control.


Create a new project called Textpad.  Add the “Main” and “TextDocument” forms to your project.  Set the IsMdiContainer property of Main to true.  Add a MainMenu control and an OpenFileDialog control (name it “openFileDialog1”) to Main.  Set the Filter property of the OpenFileDialog to “Text Files | *.txt”, since we will only be working with text files in this project.  Add a RichTextBox control to “TextDocument”, name it “richTextBox1”; set its Dock property to “Fill” and its Modifiers property to “Internal”.


Add a MenuItem control to MainMenu called “File” by clicking on the MainMenu control in Designer mode and typing “File” where the control prompts you to “type here”.  Set the File item’s MergeType property to “MergeItems”. Add a second MenuItem called “Window“.  Under the “File” menu item, add three more Items: “New“, “Open“, and “Exit“.  Set the MergeOrder property of the “Exit” control to 2.  When we start building the “TextDocument” form, these merge properties will allow us to insert menu items from child forms between “Open” and “Exit”.


Set the MDIList property of the Window menu item to true.  This automatically allows it to keep track of your various child documents during runtime.


Next, we need some operations that will be triggered off by our menu commands.  The NewMDIChild() function will create a new instance of the Document object that is also a child of the Main container.  OpenFile() uses the OpenFileDialog control to retrieve the path to a text file selected by the user.  OpenFile() uses a StreamReader to extract the text of the file (make sure you add a using declaration for System.IO at the top of your form). It then calls an overloaded version of NewMDIChild() that takes the file name and displays it as the current document name, and then injects the text from the source file into the RichTextBox control in the current Document object.  The Exit() method closes our Main form.  Add handlers for the File menu items (by double clicking on them) and then have each handler call the appropriate operation: NewMDIChild(), OpenFile(), or Exit().  That takes care of your Main form.

        #region Main File Operations

private void NewMDIChild()
{
NewMDIChild(“Untitled”);
}

private void NewMDIChild(string filename)
{
TextDocument newMDIChild = new TextDocument();
newMDIChild.MdiParent = this;
newMDIChild.Text = filename;
newMDIChild.WindowState = FormWindowState.Maximized;
newMDIChild.Show();
}

private void OpenFile()
{
try
{
openFileDialog1.FileName = “”;
DialogResult dr = openFileDialog1.ShowDialog();
if (dr == DialogResult.Cancel)
{
return;
}
string fileName = openFileDialog1.FileName;
using (StreamReader sr = new StreamReader(fileName))
{
string text = sr.ReadToEnd();
NewMDIChild(fileName, text);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

private void NewMDIChild(string filename, string text)
{
NewMDIChild(filename);
LoadTextToActiveDocument(text);
}

private void LoadTextToActiveDocument(string text)
{
TextDocument doc = (TextDocument)ActiveMdiChild;
doc.richTextBox1.Text = text;
}

private void Exit()
{
Dispose();
}

#endregion


To the TextDocument form, add a SaveFileDialog control, a MainMenu control, and a ContextMenuStrip control (set the ContextMenuStrip property of richTextBox1 to this new ContextMenuStrip).  Set the SaveFileDialog’s defaultExt property to “txt” and its Filter property to “Text File | *.txt”.  Add “Cut”, “Copy”, “Paste”, and “Delete” items to your ContextMenuStrip.  Add a “File” menu item to your MainMenu, and then “Save“, Save As“, and “Close” menu items to the “File” menu item.  Set the MergeType for “File” to “MergeItems”. Set the MergeType properties of “Save”, “Save As” and “Close” to “Add”, and their MergeOrder properties to 1.  This creates a nice effect in which the File menu of the child MDI form merges with the parent File menu.


The following methods will be called by the handlers for each of these menu items: Save(), SaveAs(), CloseDocument(), Cut(), Copy(), Paste(), Delete(), and InsertText(). Please note that the last five methods are scoped as internal, so they can be called by the parent form. This will be particularly important as we move on to the Speechpad project.


#region Document File Operations

private void SaveAs(string fileName)
{
try
{
saveFileDialog1.FileName = fileName;
DialogResult dr = saveFileDialog1.ShowDialog();
if (dr == DialogResult.Cancel)
{
return;
}
string saveFileName = saveFileDialog1.FileName;
Save(saveFileName);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

private void SaveAs()
{
string fileName = this.Text;
SaveAs(fileName);
}

internal void Save()
{
string fileName = this.Text;
Save(fileName);
}

private void Save(string fileName)
{
string text = this.richTextBox1.Text;
Save(fileName, text);
}

private void Save(string fileName, string text)
{
try
{
using (StreamWriter sw = new StreamWriter(fileName, false))
{
sw.Write(text);
sw.Flush();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

private void CloseDocument()
{
Dispose();
}

internal void Paste()
{
try
{
IDataObject data = Clipboard.GetDataObject();
if (data.GetDataPresent(DataFormats.Text))
{
InsertText(data.GetData(DataFormats.Text).ToString());
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

internal void InsertText(string text)
{
RichTextBox theBox = richTextBox1;
theBox.SelectedText = text;
}

internal void Copy()
{
try
{
RichTextBox theBox = richTextBox1;
Clipboard.Clear();
Clipboard.SetDataObject(theBox.SelectedText);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

internal void Cut()
{
Copy();
Delete();
}

internal void Delete()
{
richTextBox1.SelectedText = string.Empty;
}

#endregion


Once you hook up your menu item event handlers to the methods listed above, you should have a rather nice text pad application. With our base prepared, we are now in a position to start building some SR features.


Speechpad


Add a reference to the System.Speech assembly to your project.  You should be able to find it in C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.0\.  Add using declarations for System.Speech, System.Speech.Recognition, and System.Speech.Synthesis to your Main form. The top of your Main.cs file should now look something like this:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Speech;
using System.Speech.Synthesis;
using System.Speech.Recognition;

In design view, add two new menu item to the main menu in your Main form labeled “Select Voice” and “Speech“.  For easy reference, name the first item selectVoiceMenuItem.  We will use the “Select Voice” menu to programmatically list the synthetic voices that are available for reading Speechpad documents.  To programmatically list out all the synthetic voices, use the following three methods found in the code sample below.  LoadSelectVoiceMenu() loops through all voices that are installed on the operating system and creates a new menu item for each.  VoiceMenuItem_Click() is simply a handler that passes the click event on to the SelectVoice() method. SelectVoice() handles the toggling of the voices we have added to the “Select Voice” menu.  Whenever a voice is selected, all others are deselected.  If all voices are deselected, then we default to the first one.


Now that we have gotten this far, I should mention that all this trouble is a little silly if there is only one synthetic voice available, as there is when you first install Vista. Her name is Microsoft Anna, by the way. If you have Vista Ultimate or Vista Enterprise, you can use the Vista Updater to download an additional voice, named Microsoft Lila, which is contained in the Simple Chinese MUI.  She has a bit of an accent, but I am coming to find it rather charming.  If you don’t have one of the high-end flavors of Vista, however, you might consider leaving the voice selection code out of your project.


private void LoadSelectVoiceMenu()
{
foreach (InstalledVoice voice in synthesizer.GetInstalledVoices())
{
MenuItem voiceMenuItem = new MenuItem(voice.VoiceInfo.Name);
voiceMenuItem.RadioCheck = true;
voiceMenuItem.Click += new EventHandler(voiceMenuItem_Click);
this.selectVoiceMenuItem.MenuItems.Add(voiceMenuItem);
}
if (this.selectVoiceMenuItem.MenuItems.Count > 0)
{
this.selectVoiceMenuItem.MenuItems[0].Checked = true;
selectedVoice = this.selectVoiceMenuItem.MenuItems[0].Text;
}
}

private void voiceMenuItem_Click(object sender, EventArgs e)
{
SelectVoice(sender);
}

private void SelectVoice(object sender)
{
MenuItem mi = sender as MenuItem;
if (mi != null)
{
//toggle checked value
mi.Checked = !mi.Checked;

if (mi.Checked)
{
//set selectedVoice variable
selectedVoice = mi.Text;
//clear all other checked items
foreach (MenuItem voiceMi in this.selectVoiceMenuItem.MenuItems)
{
if (!voiceMi.Equals(mi))
{
voiceMi.Checked = false;
}
}
}
else
{
//if deselecting, make first value checked,
//so there is always a default value
this.selectVoiceMenuItem.MenuItems[0].Checked = true;
}
}
}


We have not declared the selectedVoice class level variable yet (your Intellisense may have complained about it), so the next step is to do just that.  While we are at it, we will also declare a private instance of the System.Speech.Synthesis.SpeechSynthesizer class and initialize it, along with a call to the LoadSelectVoiceMenu() method from above, in your constructor:


#region Local Members

private SpeechSynthesizer synthesizer = null;
private string selectedVoice = string.Empty;

#endregion

public Main()
{
InitializeComponent();
synthesizer = new SpeechSynthesizer();
LoadSelectVoiceMenu();
}


To allow the user to utilize the speech synthesizer, we will add two new menu items under the “Speech” menu labeled “Read Selected Text” and “Read Document“.  In truth, there isn’t really much to using the Vista speech synthesizer.  All we do is pass a text string to our local SpeechSynthesizer object and let the operating system do the rest.  Hook up event handlers for the click events of these two menu items to the following methods and you will be up and running with an SR enabled application:


#region Speech Synthesizer Commands

private void ReadSelectedText()
{
TextDocument doc = ActiveMdiChild as TextDocument;
if (doc != null)
{
RichTextBox textBox = doc.richTextBox1;
if (textBox != null)
{
string speakText = textBox.SelectedText;
ReadAloud(speakText);
}
}
}

private void ReadDocument()
{
TextDocument doc = ActiveMdiChild as TextDocument;
if (doc != null)
{
RichTextBox textBox = doc.richTextBox1;
if (textBox != null)
{
string speakText = textBox.Text;
ReadAloud(speakText);
}
}
}

private void ReadAloud(string speakText)
{
try
{
SetVoice();
synthesizer.Speak(speakText);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}

}

private void SetVoice()
{
try
{
synthesizer.SelectVoice(selectedVoice);
}
catch (Exception)
{
MessageBox.Show(selectedVoice + “\” is not available.);
}
}

#endregion

Performative Sleep

I was busy writing away in a notebook last night when I suddenly realized that I was sleeping.  In the dream, I had been working on a commentary on Montaigne’s essay Of Cannibals.  The problem with discussing dreams, of course, is that it ofen leads one into an embarrassing consideration of one’s own inner life which tends to be self-stroking and not particularly revealing — or rather, it reveals some self-absorbed aspects of one’s own personality even as one is sure that it is revealing some great inner-truth, like John Marcher’s beast in the jungle.  Or even worse, it is like taking one of those IQ tests that occassionally pop-up in one’s browser and determining from it that one is quite intelligent.

When I woke up in the morning, there was no commentary written out at my bedside, or even prepared in my head.  I don’t even remember what I was trying to say about Of Cannibals.  This contrasts starkly with the experience of sleep coding, which occassionally overcomes software developers who have been working too hard on a particular problem, and in some circles is even considered to be a mark of particularly virtuous coding.

I am convinced, as many programmers are, that sleep coding really works — that is, that in sleep, programmers actually solve problems from their waking hours.  I have often spent hours on a particularly insidious problem only to find, after a good night’s sleep, that I am able to quickly fix the problem the next morning.  And it seems to be something different from simply taking a break.  Walking away from a problem for a few hours, while it can be helpful in reducing stress, has never had the revelatory effect that sleeping has had.  I’ve even come to the point that when I consider a problem to be particularly difficult, instead of trying to solve the problem right away, I plan on learning as much as I can about it in order to hand it over to my dream-coder to solve in the night.  I think of this as an occult offshoring.

It is quite possible, of course, that the experience of dreaming code and the phenomenon of having code solved in one’s sleep are two entirely different things.  The second can be true even if the first is essentially meaningless, a phantom caused by neurons misfiring in our sleep.  And this is, in some cases, the solution to the mind-body problem.  The theory goes that when awake, we are merely observers of a mechanistic process with epiphenomenal experiences that do not actually affect what is going on in the world.  We are merely passive observers, even though we think we are actually participating and making decisions — though this implies a rather inelegant duplication of entities in the world.  Why should it be that I can code in my sleep, and also observe myself coding in my sleep through my dreams, but these two things are not the same thing?

And also, is this something that only happens for computer programmers?  Is it the case that our bodies can perform high cognitive functions without us, but only for certain types of tasks?  I don’t recall ever waking up in college with an essay fully formed in my mind.  Then again, we are told that Coleridge woke up from a dream with Kubla Kahn fully formed in his mind, and only after being interrupted by a visitor and taking a break from it, did he lose it again.

Concerning Ladders

It is a commonplace that humor resists translation.  This was Pevear and Volokhonsky’s conceit when they came out with a new translation of The Brothers Karamazov in 1990, which they claimed finally brought across (successfully, I think) the deep humor of Dostoevsky’s masterpiece.  While accuracy is the goal in most translation efforts, to hold to accuracy when translating humor unavoidably leaves much untranslated.  Thus in translating Lewis Caroll into Russian, Vladimir Nabokov chose to replace English wordplay with completely different puns sensible only to the Russian speaker, all the better to capture the flavor of Caroll’s humor.


A former colleague at the infamous Turner Studios has posted the following joke to his blog, which I must admit I cannot decipher:



 


 Yet I know it is a joke, because he adds the following gloss to the image:


Let the hilarity ensue. Someone put up a site where you can build your own World of Warcraft talent trees. The priest one made me laugh, since it’s twoo, it’s twoo.


What is one required to know in order to decipher this particular joke?  Initially, of course, one must know that this is an artifact of the online game World of Warcraft, which is a complex virtual world people pay a monthly fee in order to gain access to.  Next, the artifact is a “talent tree”, which describes different abilities people can gain through accruing time in the virtual world.  The various talents form a tree in the sense that one must gain lower level talents before one may achieve the higher talents, and while the low level talents form a broad base, there are fewer high level talents to choose from when one gets to the top.  The choice of talents one chooses to acquire, in turn, determines what sort of person one is in the virtual World of Warcraft.


This is the formal aspect of the talent tree.  In order to understand the hilarity of this particular talent tree, however, one must further understand the pictorial vocabulary used to represent talents in this tree, a task requiring a Rosetta stone of sorts.  The use of pictures to tell stories is old, and certainly predates any written languages, with exemplars such as the cave drawings at Lascaux.  Long after the advent of written languages, images continued to exert a central role in the telling of stories and the transmission of culture in societies where the majority of people were illiterate.  It was even the main way that Christianity promulgated its teachings to the masses, and the eventual eclipse of the central role of images in religious life by way of the Protestant Reformation can be seen as a direct result of the  emphasis placed on reading the Bible for oneself, and hence the importance of literacy.


Beyond pratfalls and scatology, I’m not sure that pictures without words are a particularly effective means of transmitting humor.  The talent tree for the priest represented above has less to do with cave paintings at Lascaux than with the Renaissance emblem book tradition, which does attempt to treat images as language, and reached its height of artistic expression with the HYPNEROTOMACHIA POLIPHILI.  The traditional emblem book was made up of a series of 100 or so images that were explicated with poems and allegories.  What sets them apart from instructive religious images is that they require a high level of literacy in order to read and enjoy, whereas religious images during the same period were particularly useful for the illiterate.  In some cases, due to the expense of printing woodcuts, emblem books would even forgo actual images and instead would include mere descriptions of the emblems being explicated.  Implicit in all of this, however, was the understanding that whatever could be said about the emblems was originally and overabundantly expressed in the images themselves, and that the accompanying text merely offered a glimpse into their hidden meanings.


Athanasius Kircher, the 17th century polymath, pursued a similar approach toward deciphering Egyptian hieroglyphs.  An interesting website dedicated to and to some extent influenced by his work can be found here.  Following the work of 19th century Egyptologists like Jean-Fran├žois Champollion, we know today that Egyptian hieroglyphs alternately represent either phonetic elements or words, depending on how they are used.  In the case of cartouches, the series of symbols often found on monuments and usually placed in an oval  in order to set them apart, hieroglyphs were exclusively a phonetic alphabet used to spell out the personal names of Egyptian dignitaries.  For Kircher, however, they represented a language of images which, if not actually magical, were at least possessed of superabundant and secret meaning.  Kircher sought transcendence in his efforts to cull meaning from cartouches.  How far he fell short can be gathered from this gloss by Umberto Eco in The Search For The Perfect Language:


>Out of this passion for the occult came those attempts at decipherment which now amuse Egyptologists.  On page 557 of his Obeliscus Pamphylius, figures 20-4 reproduce the images of a cartouche to which Kircher gives the following reading: ‘the originator of all fecundity and vegetation is Osiris whose generative power bears from heaven to his kingdom the Sacred Mophtha.’  This same image was deciphered by Champollion (Lettre a Dacier, 29), who used Kircher’s own reporductions, as ‘AOTKRTA (Autocrat or Emperor) sun of the son and sovereign of the crown, Caesar Domitian Augustus)’.  The difference is, to say the least, notable, especially as regards the mysterious Mophtha, figured as a lion, over which Kircher expended pages and pages of mystic exegesis listing its numerous properties, while for Champollion the lion simply stands for the Greek letter lambda.


 


 Whereas Kircher’s search for transcendence requires great learning, the icon to the right, of the Ladder of Divine Ascent, is accessible to the unliterate.  Most icons in the Eastern Orthodox tradition are of saints, and are used in prayer to the saints.  Icons that depict stories, such as this icon, are somewhat rare, though there is evidence that this was in fact the prior tradition, and the earliest Christian images, found in the catacombs of Rome, typically depict stories from the Bible.  The icon of the Ladder of Divine Ascent is based on the ladder described by John Climacus in the 7th century book of the same name, and the Orthodox saint can in fact be found at the lower left corner of the image.  Climacus, in turn, borrowed his ladder from the image of a ladder that Jacob dreamed about, a ladder extending from earth to heaven.  In this icon, Christ stands at the top of the ladder, welcoming anyone who can make the full ascent.  At the bottom are monks lining up to attempt the climb, while in between we see ascetics being diverted, distracted, and pulled off of the ladder by demonic beings.  The message is fairly straight forward.  Transcendence and salvation are possible, but very difficult.  The ladder represents the journey, but also the mediation required to ascend from the cthonic to the celestial.


I find the metaphor of the ladder striking because 1) it is man-made, and 2) it is something that one steps off of when one reaches the top.  These two features explain why the talent tree depicted above could never be a talent ladder, even though both are things that one climbs.  The tree is something made of the same earthly material that it grows out of.  It reaches for the sky, but because it is not truly a mediator, it cannot allow one to step off of it, and in fact the higher one climbs, the less stable one’s purchase is.  Just as a pier is a disappointed bridge, as James Joyce indicated, a tree is a disappointed ladder.  It goes nowhere.


This, I take it, is the humor inherent in the talent tree above.  The talent tree provides a semblance of movement upwards, but ultimately disappoints.  It always provides more, but the more turns out to be more of the same.  For an interesting unpacking of this phenomenon, one could do worse than read this cautionary blog about the dangers of playing World of Warcraft:


>60 levels, 30+ epics, a few really good “real life” friends, a seat on the oldest and largest guild on our server’s council, 70+ days “/played,” and one “real” year later…

It took a huge personal toll on me. To illustrate the impact it had, let’s look at me one year later. When I started playing, I was working towards getting into the best shape of my life (and making good progress, too). Now a year later, I’m about 30 pounds heavier that I was back then, and it is not muscle. I had a lot of hobbies including DJing (which I was pretty accomplished at) and music as well as writing and martial arts. I haven’t touched a record or my guitar for over a year and I think if I tried any Kung Fu my gut would throw my back out. Finally, and most significantly, I had a very satisfying social life before.

These changes are miniscule, however, compared to what has happened in quite a few other people’s lives. Some background… Blizzard created a game that you simply can not win. Not only that, the only way to “get better” is to play more and more. In order to progress, you have to farm your little heart out in one way or another: either weeks at a time PvPing to make your rank or weeks at a time getting materials for and “conquering” raid instances, or dungeons where you get “epic loot” (pixilated things that increase your abilities, therefore making you “better”). And what do you do after these mighty dungeons fall before you and your friend’s wrath? Go back the next week (not sooner, Blizzard made sure you can only raid the best instances once a week) and do it again (imagine if Alexander the Great had to push across the Middle East every damn week).

 


The burden of Sisyphus is a perennial staple of humorists, and not a tragedy at all.  Consider the most famous Laurel and Hardy short, The Music Box, in which the conceit of the whole film is the two bunglers trying to move a piano to a house on top of a hill.  Perhaps the most iconic example of this sort of humor is Nigel’s amplifier from This Is Spinal Tap, which “goes to eleven”.  For Nigel, eleven is a transcendent level of amplification, while for the mock interviewer, it is just one more number.  Why not just re-calibrate the amplifier and make ten eleven?  Nigel believes that eleven transforms the amplifier into a ladder, whereas the audience recognizes that it is just a tree.


I am at a point in my life where I see trees and ladders everywhere.  For instance, the constant philosophical debates around the mind-body problem can be broken down into a simple question about whether consciousness is a tree or a ladder.  If consciousness is the complex accumulation of basically simple brain processes, then it is a tree.  If aggregating various physical processes never can achieve true consciousness, then consciousness is a ladder.  And then from these two basic theses, we can arrive at all the other combinations of mind-body solutions, for instance that it is a tree that thinks it is a ladder, or a ladder that thinks it is a tree, or that ladder and tree are simply two equivalent modes of describing the same phenomenon, depending possibly on whether one is in fact a tree or a ladder.


Science fiction plots, in turn, can be broken down into two types: those in which ladders pretend to be trees, and those in which trees pretend to be ladders.  Virtual worlds, finally, are the culmination of a historical weariness over these problems, and a consequent ambivalence about whether trees and ladders make any difference, anymore.  For those who have chosen to forgo the search for ladders, virtual worlds provide a world of trees, which simulate the experience of climbing ladders — virtual ladders, so to speak.


Having had several years of success, Blizzard, the makers of World of Warcraft, have recently released a new expansion to their online world called The Burning Crusade.  Whereas up to this point, players have been limited to a maximum level of 60, those who buy The Burning Crusade will have that ceiling lifted.  With The Burning Crusade, World of Warcraft goes to level 70.