WCF, Arrays and the ExecutionEngineException

This is an anti-recipe for the ExecutionEngineException.  By an anti-recipe, I mean I will demonstrate how to get a certain result that you never want to get, with the expectation that it will help you steer clear of it.

ExecutionEngineException messages are a bit scary. They happen at a pretty low level, first of all, and aren’t easy to debug.  Second, the exception remarks provided on MSDN are:

“Execution engine errors are fatal errors that should never occur. Such errors occur mainly when the execution engine has been corrupted or data is missing.”

I want to point out one scenario which might provoke your code to throw this exception on the principle that forewarned is forearmed.  More importantly, don’t freak out if you run into this – there are some easy workarounds.

When writing a service contract for a WCF web service, you may want to use a IEnumerable<T> as the return type.  Generally this is fine.  If you try to return an array as the underlying type, however, you will get the ExecutionEngineException in .NET 3.5.

This is simply a bug, so you will need to work around it.  You have two choices: either change the signature of your service method or change the underlying type.

If you use the extension method ToList() on your array, everything will work fine.

Alternatively, if you are able to rewrite your service contract, just return the array type rather than trying to use IEnumerable<T>, ICollection<T>, or IList<T>, all of which will cause the same scary exception to be thrown if the underlying type is an array of custom objects (e.g. CustomType[]).

Arrays of simple types (e.g. string[]) do not seem to be a problem.

Arrays of custom objects will generate the exception whether they are the return type on art of the signature of the service method or if they are members of an object returned by the service method. 

I said there are two workarounds, but there are actually three.  This exception seems to only be thrown when you host your service in IIS or Cassini.  If you self-host your service, even using the BasicHttpBinding type, the problem doesn’t show up. 

To reiterate, the problem occurs when a web service contract specifies an IEnumerable<T>, ICollection<T> or IList<T> as the return type and the underlying type returned is actually an array of custom objects.

This has been noted as a bug by Microsoft (https://connect.microsoft.com/wcf/feedback/ViewFeedback.aspx?FeedbackID=433569).  It also appears to be fixed in .NET 4.0.  I have ported my troubled code to VS 2010 Beta 1, and the code works fine.

I don’t think this problem is actually encountered that often for two reasons.  First, we typically don’t use raw arrays that often.  Instead we use generic Lists or even ArrayLists which, even if they are a bit fatter than simple arrays, are awfully convenient.  Second, while it is considered to be a best practice to always return least derived types in methods and properties – in practice this seems to be the exception rather than the rule.

Finally, it doesn’t appear to be a problem with the DataContractSerializer as some have suggested elsewhere.  I can serialize my data contracts just fine when I use the DataContractSerializer on its own.  The problem only seems to happen for me when I try to expose these contracts through a WCF service.  Also, if it were a DataContractSerializer problem, changing to self-hosting shouldn’t help.

Reporting Services and WCF

Ingredients: Visual Studio 2008, SSRS 2008, AdventureWorks2008

While stand alone *.rdlc reports allow one to use data objects as a data source, it would be convenient to be able to do the same thing using published reports.  The XML data source indirectly makes this possible by accepting a path to an *.asmx web service as a valid “connection string” value. 

There seems to be some confusion over whether this can be done with WCF Services as well.  The short answer is yes, as long as the WCF service endpoint appears as a typical web service.  In other words, it must use basicHttpBinding as its binding type.  On the client side, consequently, the reporting services data source must use integrated security (Windows Authentication) for its credential type. 

Given these strictures, if you already have web services exposed for your data, it should take little to no effort to consuming those WCF services from a report.  I will walk through some simple examples on how to do this.  Ultimately the complications are derived not from structuring the WCF service methods themselves, but rather in figuring out how to use the Query syntax that comes with reporting services – which basically serves as a declarative style for generating SOAP service calls.  This problem, it should be pointed out, is not peculiar to WCF services, but was already a road block in figuring out how to make reporting services talk to *.asmx web services.

In working through the details of making reporting services consume a WCF Service, I was greatly aided by two sources.  The first is Jesse Ezell’s blog which beautifully summarizes the steps required, although he makes the task seem rather harder than it actually is and almost convinced me to abandon the effort early on.  The second is an MSDN article written by Jonathan Heide that deals with the query syntax for using reporting services with XML data sources.  I highly recommend both.

The following walkthrough will introduce five services and five reports, each progressively more complicated.  All examples are included in the download linked at the top.

1. A Simple Service

The simplest meaningful service for reporting returns data without taking any parameters.  For this example, we will not implement our own message contract, although the next example will demonstrate some advantages to doing so.

The service method in this example returns an enumeration of Employee types.  Below is the Employee Data Contract and the Interface definition for the service method:

namespace MyReportingService
{
    [DataContract]
    public class Employee
    {
        [DataMember]
        public string FirstName { get; set; }
        [DataMember]
        public string MiddleName { get; set; }
        [DataMember]
        public string LastName { get; set; }
        [DataMember]
        public string JobTitle { get; set; }
        [DataMember]
        public string Email { get; set; }
        [DataMember]
        public int EmailPromotion { get; set; }
    }
}
namespace MyReportingService
{
    [ServiceContract]
    public interface IEmployeesService
    {
        [OperationContract]
        IEnumerable<Employee> GetAllEmployeesRPCStyle();
    }
}

In the actual implementation of the service method, I happen to be retrieving records from the AdventureWorks2008 database.  For our purposes, however, we now have everything we need with just the interface and the data contract to hook reporting services up to our WCF service.  Make sure the binding is set to basicHttpBinding as Visual Studio likes to default to wsHttpBinding when one creates a new service.

Make sure the service is running before continuing to the next step.  If the Reports project and the WCF project are in the same solution, the easiest way to do this is to right click on the service project and select Debug|Start new instance.

The first step in setting up a report that consumes a WCF service is to create an XML Data Source in Visual Studio.  When we do so, we will be prompted for the following information.  Be sure to set XML as the Type and enter the URI of the service as the Connection String:

datasource

We also will need to set the Credentials used for your data source.  As far as I know, web service calls through reporting services always requires that Windows Authentication be used, though perhaps a future release of SSRS will allow us to create a data source that talks to a secured service:

credentials

Having done that, we can now use the Report Wizard to create a new report.  On the first screen, set the data source we just created as the Shared Data Source for the report.

SetDataSource

The next screen, the query screen, is where we’ll be spending most of our time from now on.  Select the Query Builder… at the top of the query screen.  Using the query builder will allow us to test our XML query as we go along.  In the Query Builder window, choose Text as the Command type if it isn’t already selected.

The basic syntax for making a service call will include a Query tag with a nested Method, SoapAction, and ElementPath.

<Query>
<Method Name="GetAllEmployeesRPCStyle" Namespace="http://tempuri.org/"/>
<SoapAction>http://tempuri.org/IEmployeesService/GetAllEmployeesRPCStyle</SoapAction>
<ElementPath IgnoreNamespaces="true">*</ElementPath>
</Query>

There are some things worth noticing here. 

  • We are using the default namespace for our service.  If one overrides the default namespace, the custom namespace will need to go into the Method and the SoapAction.
  • The Method Name, for a service method that doesn’t use its own message contract, is just the name of the service method.
  • The SoapAction specifies the full path to the service, which includes the service namespace, the service name (we are using an interface for our *.svc service, so we want the name of the interface rather than the implementing class), and the service method.
  • ElementPath, which is a specialized version of XPath, parses the data returned from the call.  IgnoreNamespaces should be set to true to enable us to be ignorant of some of the specifics of the SOAP message that has been returned by the service.
  • On this first swipe we will use the “*” as a wildcard syntax to return any repeating data that is found in the result.

If we click on the red bang (or hit F5) the query will run and we can immediately see if the query is written correctly.  The query above will return the following data in the Query Builder:

query1

This is basically the data we want returned but it isn’t shaped quite right.  The “xmlns”, “a” and “i” columns are present because the enumeration we want is actually wrapped up in several other elements, each with attributes that show up in the flattened data.  To get rid of these, we need to specify more precisely the data we want in the ElementPath.

The enumeration of Employee types is wrapped in a Result element, which in return is wrapped in a Response element.  By convention, the names of these elements are based on the name of the service method like this: [ServiceMethodName]Response/[ServiceMethodName]Result.

Instead of simply wildcarding it, we can rewrite the ElementPath to fully specify the data we want:

<Query>
<Method Name="GetAllEmployeesRPCStyle" Namespace="http://tempuri.org/"/>
<SoapAction>http://tempuri.org/IEmployeesService/GetAllEmployeesRPCStyle</SoapAction>
<ElementPath IgnoreNamespaces="true">
GetAllEmployeesRPCStyleResponse/GetAllEmployeesRPCStyleResult/Employee
</ElementPath>
</Query>

This query actually still returns the same data in the image above, but it puts in a good position to start shaping the data the way we want.  Squiggly brackets can be appended to any element in our ElementPath in order to specify the attributes we want to be displayed.   In this case, however, we will use empty squiggly brackets to indicate that we do not want any attributes to be returned for our wrapper elements.  The following ElementPath:

<ElementPath IgnoreNamespaces="true">
GetAllEmployeesRPCStyleResponse{}/GetAllEmployeesRPCStyleResult{}/Employee
</ElementPath>

will return us the following shaped result:

query2

This gets us closer to the shape we want.  You may notice, however, that the columns are in the wrong order.  To be more precise, they are in alphabetical order, which may not be exactly what we want.  There are two ways to fix this.  First, in the Data Contract for the Employee type, we can set a particular order for the Employee properties to be rendered in the returned SOAP message.  This is done by using the Order property off of the DataMember attribute of our properties:

 

[DataMember(Order = 0)]
public string FirstName { get; set; }
[DataMember(Order = 1)]
public string MiddleName { get; set; }
[DataMember(Order = 2)]
public string LastName { get; set; }
[DataMember(Order = 3)]
public string JobTitle { get; set; }
[DataMember(Order = 4)]
public string Email { get; set; }
[DataMember(Order = 5)]
public int EmailPromotion { get; set; }

While this is typically a good idea for versioning Data Contracts, there is a simpler way to order the columns using, once again, the squiggly brackets:

<ElementPath IgnoreNamespaces="true">
GetAllEmployeesRPCStyleResponse{}/GetAllEmployeesRPCStyleResult{}/
Employee{FirstName, MiddleName, LastName, JobTitle,Email, EmailPromotion}
</ElementPath>

There is a final thing we will want to do with out ElementPath.  The underlying type for EmailPromotion is actually an Integer.  If we use the ElementPath as it currently stands, however, EmailPromotion will be used as a String (the default type) when it is consumed by the report.  Sometimes this doesn’t matter.  If we wanted to aggregate on EmailPromotion, however, in order to do sum values or average values, we would have to convert the value of EmailPromotion in an expression to make it possible.

Fortunately, we can also specify the type of each Employee column in the ElementPath.  Return types are indicated by placing the Visual Basic type name – Short, Integer, Long, Decimal, Single, Double, Date, String, Boolean, etc – between ellipses after the column name.

I should point out that since we are simply parsing XML using the ElementPath syntax, and since the SOAP message tells us nothing about the underlying types of the data that is returned, the actual property types of the Employee Data Contract are in effect irrelevant as far as reporting services is concerned.  They could all be typed as objects and it would not matter to the design of the report.  The typing occurs in the ElementPath and nowhere else.

The query for our report should now look like this:

<Query>
<Method Name="GetAllEmployeesRPCStyle" Namespace="http://tempuri.org/"/>
<SoapAction>http://tempuri.org/IEmployeesService/GetAllEmployeesRPCStyle</SoapAction>
<ElementPath IgnoreNamespaces="true">
GetAllEmployeesRPCStyleResponse{}/GetAllEmployeesRPCStyleResult{}/
Employee{FirstName, MiddleName, LastName, JobTitle,Email, EmailPromotion(integer)}
</ElementPath>
</Query>

and our data will look like this:

query3

I will continue this series by talking about WCF Message Contracts and how to pass parameters in the next post.

Ajax AutoComplete Extender with WCF

blackstone

The problem with conjuring tricks is that they lose practically all their glamour once you find out how they are done.  It’s very cool to see David Blaine walk down the street, do a few passes over his hand, and resurrect a fly which proceeds to flee.  It’s rather disappointing to do a google search and discover that in order to prepare for this trick, the first requirement is that you freeze a fly.

My trick is to make an autocomplete extender from the Ajax Control Toolkit call a WCF service instead of an asmx service.  For this recipe, I assume that you are already familiar with the autocomplete extender, and that you are using Visual Studio 2008.  I warn you in advance — my trick disappoints.  It is so trivially easy that, once the technique spreads, it is very unlikely to impress your colleagues at work, much less get you a date with a supermodel.

Start by creating a new web project called AutocompleteWCF.  Add a reference to the AjaxControlToolkit.dll.  Open up the default aspx page that is generated with your project, and add the following code to:

    <form id="form1" runat="server">
    <asp:ScriptManager ID="ScriptManager1" runat="server">
    </asp:ScriptManager>
    <div>
            <asp:TextBox runat="server" ID="myTextBox" Width="300" autocomplete="off" />
            <ajaxToolkit:AutoCompleteExtender
                runat="server" 
                BehaviorID="AutoCompleteEx"
                ID="autoComplete1" 
                TargetControlID="myTextBox"
                ServicePath="Autocomplete.svc" 
                ServiceMethod="GetCompletionList"
                MinimumPrefixLength="0" 
                CompletionInterval="1000"
                EnableCaching="true">
            </ajaxToolkit:AutoCompleteExtender>
    </div>
    </form>

 

This is the standard demo code that is shipped with the Ajax Control Toolkit Sample Website.  I’ve simplified it a bit by removing the animations.  The only significant change I’ve made is to change the ServicePath from Autocomplete.asmx to Autocomplete.svc, the latter being the extension for a WCF service.

The next step is to create our service and add a GetCompletionList operation to it.  The easiest way to do this is to go to Add | New Item and just select the Ajax-enabled WCF Service item template, but this would be so easy that it is hardly worth doing.

Instead, create a new WCF Service using the WCF Service Item Template and call it Autocomplete.svc.  Visual Studio will automatically generate a service interface for you.  Delete the interface.  We don’t need it.  (To be more specific, I don’t know how to get this to work with an interface, so I’m just going to ignore that it is possible.)

Again, I am going to rip off the ACT sample app and just borrow the code from their webservice and place it in our WCF service.  The WCF service class (Autocomplete.svc.cs) will look like this:

    [ServiceContract(Namespace = "")]

    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]

    public class Autocomplete

    {

 

        [OperationContract]

        public string[] GetCompletionList(string prefixText, int count)

        {

            if (count == 0)

            {

                count = 10;

            }

 

            if (prefixText.Equals("xyz"))

            {

                return new string[0];

            }

 

            Random random = new Random();

            List<string> items = new List<string>(count);

            for (int i = 0; i < count; i++)

            {

                char c1 = (char)random.Next(65, 90);

                char c2 = (char)random.Next(97, 122);

                char c3 = (char)random.Next(97, 122);

 

                items.Add(prefixText + c1 + c2 + c3);

            }

 

            return items.ToArray();

        }

 

A few things worth noting:

1. Autocomplete does not implement the IAutocomplete Interface.  Even though this is generated automatically, with the WCF Service item template, you should remove it.

2. The service contract has a blank Namespace explicitly declared. 

3. The ASPNetCompatibilityRequirements attribute must be added to our class.

 

This takes care of the code that calls the WCF service, as well as the service itself.  We now have rig up the web.config file.  If you’ve been working with WCF for any length of time, then you know that this is where the problems usually occur.  Fortunately, the configuration is fairly simple.  You need to set up an endpoint behavior for your service that enables web scripting (much the way asmx web services must be decorated with the ScriptService attribute in order to be called from client-script).  You also will need to turn AspNetCompatibilityEanbled on for the hosting environment.

 

    <system.serviceModel>

        <behaviors>

            <endpointBehaviors>

                <behavior name="AjaxBehavior">

                    <enableWebScript/>

                </behavior>

            </endpointBehaviors>

        </behaviors>

        <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>

        <services>

            <service name="AutocompleteWCF.Autocomplete">

                <endpoint address="" behaviorConfiguration="AjaxBehavior" binding="webHttpBinding" contract="AutocompleteWCF.Autocomplete"/>

            </service>

        </services>

    </system.serviceModel>

 

And that is all you need to do make the AutoComplete Extender work with a WCF service instead of an asmx web service.  I told you it would be unimpressive.

Of course, using a WCF service for Ajax has all the limitations that using an asmx file for Ajax did.  First of all, you can’t call a service that is in a different domain than the page which hosts your client-code.  This is a security feature, to prevent malicious code from redirecting your harmless javascript to something nasty on the world wide web.

Second, you can’t call just any service from your client-side code.  The service must be explicitly marked as something that can be called from client code.  In asmx web services, we used ScriptService for this.  In WCF services, we similarly use EnableWebScript binding property.

Now I feel like I’ve wasted your time, so here’s a YouTube video of David Blaine to make up for it.  And remember, David Blaine is to Chris Angel what Daisy Duke was to Alexis Carrington.  It’s an existential thing, and at some point, you’ve just got to pick sides and stay put in a way that will determine who you are for the rest of your life.

Are you a David Blaine/Daisy Duke kind of person or are you a Chris Angel/Alexis Carrington sort?  Do some soul searching and please let me know what you learn about yourself.

Beyond Good and Endpoints

beyonder

Two important WCF resources came out last week.  The first is the source code for StockTrader 2.0, Microsoft’s reference app for .NET 3.5 using, in particular, CF and PF.  The download is available here http://msdn.microsoft.com/en-us/netframework/bb499684.aspx.  This is fairly significant since there are not (as far as I have been able to tell) any guidelines generally available on how to build a distributed application using the Communications Foundation.  In the project I am currently working on, we have been trying and stopping as we go, trading frequent emails with various Microsoft insiders to try to find out if what we are doing makes sense (so far it has). 

Now we have the opportunity to compare our app against StockTrader and see where the differences lie.  I’m not sure if this is an artifice or not, but I like being able to take two ideas and decide which one better as I work through an architecture.  This is a necessary exercise in working towards the "right" architecture.  The approach resembles, in some ways, the Mirror of Nature that Richard Rorty made his name on attacking, except that it is concerned with practice rather than with epistemology.  I think that there is a right way to do things of which I am unaware.  To use Don Rumsfeld’s memorable phrase, it is one of the things I know I don’t know.  I then work, through trial and error as well as a comparison of alternatives, towards an intellectual representation of what that correct way of getting things done might be. 

I’ve talked to other developers who feel that speaking about the "right way" to do things is simply a trap we all fall into or, worse, a fiction that serves only to generate artificial conflicts and slow down actual development.  As one developer told me, "There are many arguments on either side of any issue, and we’re not going to be able to resolve here which ones are better, since we come to it with preconceptions and prejudices we can’t get over easily."  While visiting my brother at college once, I attended a lecture in which Alan Dershowitz, the civil rights lawyer, argued very much the same case.  He insisted that he could go about seven levels behind any argument, and then seven levels behind the case against him.  The peeling of levels (usually it only takes two or three) gives the false impression that we are actually getting to the bottom of things, whereas Dershowitz himself had reached the point in his career where he felt it was merely an exercise in cleverness for lawyers and for philosophers an exercise in futility.

This may all be true, and yet I feel I need this fiction, if it is a fiction, in order to do my job well.

The second important release this week is Juval Lowy’s presention on WCF: Beyond the Endpoints (you will need to have Microsoft Live Meeting installed to listen to it).  In this presentation you will come to realize what many people already know: Mr. Lowy is either a genius or a madman, and had he lived in another era, he would have made a good Jules Verne villain.  Based on his WCF book, I had taken his position to be that CF solves many distributed programming problems, and in the big picture serves as the fulfillment (or successor) to component based programming — the notion that based upon common and established conventions we can pass messages between disparate technologies.

In Beyond the Endpoints, Mr. Lowy tops even those grand pronouncements.  Here he argues that CF is actually the successor to all of our current .NET programming conventions in C# and VB.NET, and is in fact a new programming model.  With specific examples from the Singleton pattern to class level isolation, he demonstrates various low level ways in which WCF provides better ways to do our most common programming tasks.  He makes clear in the Q&A session afterwards that he basically sees something in CF that even Microsoft doesn’t see in their technology.  His vision for WCF veers wildly away from Microsoft’s vision.  When asked what the performance penalty for using CF as a programming model would be, he insists that sending messages over a wire in WCF is actually faster than working with traditional in-proc objects (given certain unspecified conditions).  Moreover, the performance cost is irrelevant.  Mr. Lowy appeals to history in order to explain his vision for CF.  COM, for instance, started off as simply a way to embed files inside of Microsoft Word documents.  In order to get Object Linking and Embedding to work, however, Microsoft’s engineers were oblige to solve fundamental problems that in turn produced what we now know as COM.  CF, for Juval Lowy, is headed very much along the same track.

It is a truly amazing presentation, ambitious in scope and broad in vision, and I highly recommend it to anyone interested in the inherent potential of the Communications Foundation.

What is Service Oriented Architecture?

la_condition_humaine

What is SOA?  It is currently the hottest thing going on in corporate technology, and promises to simultaneously integrate disparate applications on multiple platforms as well as provide code reuse to all of those platforms.  According to Juval Lowy, it is the culmination of a 20 year project to enable true component-based design — in other words, the fulfillment of COM, rather than merely its replacement.  Others see it as a threat to object oriented programming. According to yet others, it is simply the wave of the future.  Rocky Lhotka recently remarked at a users-group meeting that it reminds him of mainframe programming.  In Windows Communication Foundation Unleashed, the authors write somewhat uncharitably:

"Thomas Erl, for instance, published two vast books ostensibly on the subject, but never managed to provide a noncircuitous definition of the approach in either of them."

This diversity of opinion, I believe, gives me an opening to offer my own definition of SOA.  SOA is, put simply, the triumph of the Facade pattern.

In the 90’s, Erich Gamma, Ralph Johnson, John Vlissides and Richard Helm popularized the notion of the 23 fundamental design patterns of object oriented programming.  I’ve often wondered why they came up with 23 patterns.  Some, such as the Flyweight pattern, are simply never used.  At the same time, one of the most popular patterns, MVP, doesn’t even make the canonical list.  How did they come up with 23?

Here’s an article on the significance of the number 23 which may or may not shed light on the Gang of Four’s motivation.  In Peter Greenaway’s A Zed and Two Noughts, the characters become obsessed with the number 23, and claim that there are 23 letters in the Greek alphabet and that Vermeer created 23 paintings (both false, by the way).  Perhaps the Gang of Four are Discordians — Discordians are fascinated by what they call the 23 Enigma.

In any case, they came up with 23 canonical (or "fundamental" or "classic") design patterns, and in the past decade, knowing these patterns has become the unofficial dividing line between the common run of code monkeys (I use the term affectionately) and so-called "true" developers — the initiation rite that turns boy programmers into men.  Anyone in development who wants to be anybody makes the attempt to learn them, but for whatever reason, the 23 patterns resist the attempt — sometimes because it is difficult to see how you would ever actually use them.  It helps, however, to remember that the StringBuilder type in C# is based on the Builder pattern, and that the Clone method on most types implements the Prototype pattern.  Delegates are built around the Observer pattern and collections are built around the Iterator pattern — but since these are both basically part of the C# language, among others, you don’t really need to learn them anymore.  In my opinion, the most useful patterns are the Template and the Factory Method.  The Singleton pattern, on the other hand, starts off seeming like a useful pattern but turns out not to be — a bit like a bad joke one eventually tires of.  It is, however, easy to remember, if somewhat tricky to implement.

The one pattern no one ever fails to remember is the Facade pattern.  It doesn’t do anything clever with abstract base classes or interfaces.  It doesn’t have tricky implementation details.  It simply takes the principle of encapsulation and goes crazy with it. Whatever complicated code you have, you place it behind a wall of code, called the Facade, which provides methods to manipulate your "real" code.  It’s the sort of pattern which, like Monsieur Jordan, once you find out about it you realize you’ve been doing it all your life.  The simplicity and ubiquity of the Facade makes it an unattractive pattern — it takes no programming acumen to learn it; it requires great effort to avoid it. It is the dumbest of the 23 canonical design patterns.

And Service Oriented Architecture is all built around it.  In some sense, SOA marks the democratization of architecture.  There are still tricks to planning a good SOA, and securing it may require some sophistication — but with SOA, anyone can be an architect.  Well … anyone who can build a Facade.