Authentically Virtual
As translated by Lady Charlotte Guest, excerpted from Robert Graves’s The White Goddess: A Historical Grammar of Poetic Myth:
Discover what it is:
The strong creature from before the Flood
Without flesh, without bone,
Without vein, without blood,
Without head, without feet …
In field, in forest…
Without hand, without foot.
It is also as wide
As the surface of the earth,
And it was not born,
Nor was it seen …
[Answer: Ventus]
Song
Go and catch a falling star,
Get with child a mandrake root,
Tell me where all past years are,
Or who cleft the devil’s foot,
Teach me to hear mermaids singing,
Or to keep off envy’s stinging,
And find
What wind
Serves to advance an honest mind.If thou be’st born to strange sights,
Things invisible to see,
Ride ten thousand days and nights,
Till age snow white hairs on thee,
Thou, when thou return’st, wilt tell me,
All strange wonders that befell thee,
And swear,
No where
Lives a woman true and fair.If thou find’st one, let me know,
Such a pilgrimage were sweet;
Yet do not, I would not go,
Though at next door we might meet,
Though she were true, when you met her,
And last, till you write your letter,
Yet she
Will be
False, ere I come, to two, or three.–John Donne
Why do software projects fail? is not a philosophical question in Descartes’ sense, that is one that can be decomposed in order to arrive at a practical solution. Rather, it is a contemplative problem in Aristotle’s sense: one that can be looked at in many ways in order to reveal something about ourselves.
Software projects fail because we are weak. We can make software projects better not by becoming stronger, but by minding our own weaknesses. The truth is that software programming has made a lot of progress over the past twenty years. It has not made it along the via positiva, however, through solutions such as OO, or Design Patterns, or Agile, or SOA, or a host of other white knight solutions that over time have turned out mostly to be hype.
Rather it has progressed through the via negativa, or by learning what not to do. The via negativa was originally a mode of theological discourse, mystical in essence, which strived to understand God not in terms of what He is, but in terms of what He is not. This mystical path serves well in pragmatic matters, also, and below I will outline some principles of the via negativa as applied to software development.
I should include the caveat that I am not sure these principles have ever worked well for me, and that I continue to underestimate the scope of a task and the amount of time it will take to finish it. On the positive side, I believe it all could have all been worse.
1. Programming is neither an Art nor a Science
The thing I like best about Hunt and Thomas’s book The Pragmatic Programmer is their attempt in the first chapter to analogize the act of programming to the Medieval journeyman tradition and the concept of craft. I liked it so much, in fact, that I never bothered to read the rest of their book. The point of thinking of programming as a craft is that we escape the trap of thinking of it in other ways.
Many programmers think of coding as an art, in the romantic sense. Every project is new and requires a new solutions. The solutions a programmer applies constitute a set of conceits that express the programmer’s own uniqueness. Rather than applying mundane and known solutions to a set of programming problems, the artist coder prefers to come up with solutions that will express his originality.
On the other side, some programmers think of coding as a science. There is a set of best ways to accomplish any given task and their role is to apply that perfect solution.
Between these two is the notion of programming as craft. There are a set of ways of doing things, such as retrieving data from a database or displaying them for data entry. The craftsman’s goal is to accomplish his tasks in the ways he knows how to do it, whether they are the best or not, with as little waste as possible. The craftsman’s motto is also Voltaire’s: Le mieux est l’ennemi du bien.
2. Don’t Invent when you can Steal
A corollary to the craftsman’s motto is not to invent what you can steal. One of the biggest mistakes that developers make is to assume that their problems are unique to them. In fact, most problems in programming have been faced before, and developers are unusually generous about publishing their solutions on the Internet for others to use. Good design should always involve, early in the process, using Google to find how others have solved your problems. This is somewhat different from the Patterns frame-of-mind which assumes that there are common theoretical solutions to recurring problems. What you really want to find are common implementations, specific to the language and platform you are using, to your programming needs.
This was a well-known and commonly applied rule in the early days of radio. Comics would freely steal jokes from one another without attribution, and occasionally would complain about how many of their jokes were being used on other stations. What the radio comics knew, and we need to relearn, is that the success of the punchline depends not on the setup, but on the delivery.
3. Smells are More Important than Patterns
Software changes too quickly for patterns to be useful. By the time patterns are codified, they are already obsolete. With experience, however, also comes a knowledge of what doesn’t work, and it is rarely the case that something that hasn’t worked in the past will somehow magically start working in the present.
A smell is a short-hand term for things you know will not work. While Richard Feynman was known as a gifted physicist at Los Alamos during the Manhattan Project, his peculiar talent was in intuiting what not to do. He would go from project to project and, without knowing all the details, be able to tell those involved wether they were on the right path or the wrong path. Eventually, his intuition about such matters became something other physicists respected and relied upon.
Feynman had a natural gift for sniffing out bad smells, which he couldn’t completely explain himself. With experience, we all come to develop a good nose for smells, which manifest as a sense that something is wrong. The greatest trick is to trust our respective noses and take smells into account, so that smells become identified as risks which deserve to be monitored.
Here are some common smells that emanate from doomed projects:
4. Shame Drives Good Code
The most successful innovations in software development over the past twenty years have been shame based. Pair-programming works because there is always a second set of eyes watching what we are doing. It is difficult to take shortcuts or ignore standards when someone else is likely to call one on it almost immediately.
Code reviews are a more scattershot approach to the same problem. Given that a code review can treat any piece of production code as a topic of analysis, it behooves the programmer to add comments, break up methods, and a host of other good coding practices in order to avoid having other developers point out obvious laziness.
QA groups are the ultimate wielders of shame as a tool to drive good development practices. QA brings obvious mistakes to light. A developer who might try to slip in bad business logic without first verifying that it works, on the self-assurance that it should work, is less likely to so if he knows his private malfeasance might be brought to public light. While QA is typically bad at catching problems with deep programming logic, they can instill a sense of shame in developers that will lead them to verify and thoroughly unit test their deep logic as they would test their shallow logic, knowing that others are watching.
5. Manage Risk, Not Progress
All the points above are programmer-centric. Manage risk, not progress is an essential rule for managers, project managers and business analyst. Because programming tasks can be shifted around, it is common to put off difficult problems till the end, simply because one can. This makes project plans, which measure all tasks with a common rule stick, notoriously bad at predicting the success level of a project at any given time.
Measuring risk is a better way to determine the success of a project. To my thinking, identifying risk and determining whether risks have been overcome is the chief role of a good project manager. If he spends the rest of his time surfing the Internet, I couldn’t care less. Unfortunately, many project managers insist on reading self-help books about leadership and interpret their role to be one of offering inspiration to others. They often view their roles in this way to such a degree that they tend to refrain from tracking risk, which is always a downer. And the best way to avoid tracking risk is to never identify it in the first place.
6. The Code Must Flow
Code must be treated as disposable. It must be treated as a commodity. It is a common feature of coding to treat every piece of code as special. On the other hand, we know that it isn’t true when we review our code months later. What is truly gained in coding through a problem is not the physical code itself, but the knowledge of how to solve the problem.
By constantly coding through every phase of a project, we gain knowledge of the problem domain. By disposing of our code when we know it doesn’t work, and starting over, we learn not to make fetishes of our code.
The second worst programming practice is when a prototype is turned into a real application. This is a problem for developers. It seems like it will save time to simply build on the same code that we started prototyping with, despite the fact that it fails to implement good practices or commenting or any of the other things we have come to expect from good code.
The worst programming practice is when management asks how long it will take to release a prototype, and demands the code be sent out as is. There is no answer to this worst of all programming practices, for while nothing can straighten out the crooked timber of humanity, poor management can always warp it further.
In the Theatetus, Plato writes that ‘philosophy begins in wonder’. Here is Jowett’s translation of the passage:
Soc: …I believe that you follow me, Theaetetus; for I suspect that you have thought of these questions before now.
Theaet. Yes, Socrates, and I am amazed when I think of them; by the Gods I am! and I want to know what on earth they mean; and there are times when my head quite swims with the contemplation of them.
Soc. I see, my dear Theaetetus, that Theodorus had a true insight into your nature when he said that you were a philosopher, for wonder is the feeling of a philosopher, and philosophy begins in wonder. He was not a bad genealogist who said that Iris (the messenger of heaven) is the child of Thaumas (wonder).
In the Metaphysics, Aristotle continues the same theme (tr. W. D. Ross):
For it is owing to their wonder {ex archês men ta procheira tôn atopôn thaumasantes} that men both now begin and at first began to philosophize; they wondered originally at the obvious difficulties, then advanced little by little and stated difficulties about the greater matters, e.g. about the phenomena of the moon and those of the sun and of the stars, and about the genesis of the universe. And a man who is puzzled and wonders thinks himself ignorant (whence even the lover of myth is in a sense a lover of Wisdom, for the myth is composed of wonders); therefore since they philosophized in order to escape from ignorance, evidently they were pursuing science in order to know, and not for any utilitarian end.
This wonder tradition in philosophy has two principles. One is that wonder must lead to the articulation of questions, for without questions and dialectic, wonder never goes any further. The second is that questioning must be non-utilitarian, and that its end must be contemplation, rather than the solution to a practical problem; in other words, questions must be open-ended in order to count as philosophical (i.e., pure scientific) problems.
Thus Aristotle continues, in Book II of the Metaphysics:
The investigation of the truth is in one way hard, in another easy. An indication of this is found in the fact that no one is able to attain the truth adequately, while, on the other hand, we do not collectively fail, but every one says something true about the nature of things, and while individually we contribute little or nothing to the truth, by the union of all a considerable amount is amassed. Therefore, since the truth seems to be like the proverbial door, which no one can fail to hit, in this respect it must be easy, but the fact that we can have a whole truth and not the particular part we aim at shows the difficulty of it.
Against this tradition, Descartes, an extremely practical man, argues in Articles 77 and 78 of Passions of the Soul (tr. Stephen H. Voss):
Furthermore, though it is only the dull and stupid who do not have any constitutional inclination toward Wonder {l’admiration}, this is not to say that those who have the most intelligence are always the most inclined to it. For it is mainly those who, even though they have a good deal of common sense, still do not have a high opinion of their competence [who are most inclined toward wonder].
And though this passion seems to diminish with use, because the more one encounters rare things one wonders at, the more one routinely ceases wondering at them and comes to think that any that may be presented thereafter will be ordinary, nevertheless, when it is excessive and makes one fix one’s attention solely upon the first image of presented objects without acquiring any other knowledge of them, it leaves behind a habit which disposes the soul to dwell in the same way upon all other presented objects, provided they appear the least bit new to it. This is what prolongs the sickness of the blindly curious — that is, those who investigate rarities only to wonder at them and not to understand them. For they gradually become so given to wonder that things of no importance are no less capable of engaging them than those whose investigation is more useful.
Descartes found many ways to break with the Aristotelian tradition that had dominated Western thought for over a millennium, but none, I think, more profound than this dismissal of the wonder tradition. In this brief passage, he places intelligence {l’esprit} above contemplation as the key trait of philosophizing.
A consequence of this is that the nature of philosophical questioning must also change. In a Cartesian world, questions must have a practical goal. Difficult problems can be broken down into their components, and if necessary those must be broken down into their components, until we arrive at a series of simple problems that can be solved easily.
Descartes’ position is only strengthened by what is often called the scandal of philosophy: why, given all this time, has philosophy failed to answer the questions it originally set for itself?
Another way to look at the scandal, however, is not as a problem of lacking answers to these questions, but rather as a problem of having an overabundance of answers. Philosophy, over the centuries, has answered the question of God’s existence with both a yes and a no. There are five canonical proofs of God’s existence, as well as a multitude of critical analyses of each of these proofs. We are told that Occam’s Razor, originally a tool of theological discourse, demands that we reject God’s existence. At the same time, we are told that Occam’s Razor, the principle that simple answers are preferable to complex answers, itself depends on a rational universe for its justification; for the only thing that can guaranty that the universe is simple and comprehensible rather than Byzantine in its complexity, is God Himself.
The scandal of philosophy is itself based on a presupposition: that this overabundance of answers, and the lack of definitive answers, is contrary to the purpose of philosophical questioning. Yet we know of other traditions in which the lack of answers is indeed a central goal of philosophical questioning.
Zen koans are riddles Zen masters give to their students to help them achieve enlightenment. Students are expected to meditate on their koans for years, until the koan successfully works its effect on them, bringing them in an intuitive flash into the state of satori.
I dabble in the collecting of questions, and with regard to this habit, the observation Descartes makes above about the curious “who investigate rarities only to wonder at them and not to understand them” fits me well. One of my favorite sets of questions comes from Robert Graves’s The White Goddess, a book which, starting from an analysis of The Romance of Teliesin that makes Frazer’s Golden Bough seem pedestrian by comparison, attempts to unravel the true purpose of poetry and, in the process, answers the following questions:
Another set comes from Slavoj Zizek’s Enjoy Your Symptom!, in which Zizek elucidates certain gnomic pronouncements of Jacques Lacan through an analysis of mostly 50’s Hollywood movies:
Modern life provides its own imponderables:
There are no definitive right or wrong answers to these questions. Rather, how we approach these questions as well as how we respond to them contribute to shaping who we are. In his short work, Existentialism and Human Emotions, Sartre tells an anecdote about a student who once asked him for advice. The student wanted to leave home and join the French Resistance. The reasons were clear to him. The Germans were illegitimate occupiers and it was the duty of every able-bodied Frenchman to fight against them. At the same time, the student had a sickly mother who required his assistance, and leaving her would not only break her heart, but he might possibly never see her again. To leave her would entail sacrificing his filial duties, while not to leave her would entail abandoning his moral duty. To this student, caught in the grips of a mortal quandary, Sartre offered the unexpected advice: choose!
…[I]n creating the man that we want to be, there is not a single one of our acts which does not at the same time create an image of man as we think he ought to be. To choose to be this or that is to affirm at the same time the value of what we choose, because we can never choose evil. We always choose the good, and nothing can be good for us without being good for all.
But this isn’t the whole truth. There are also choices we make that appear arbitrary at the time, committed without any thought of ‘man as he ought to be,’ but which turn out to have irreversible consequences upon who we become. In the film The Goalie’s Anxiety at the Penalty Kick, Wim Wenders follows a goalie who is kicked off of his soccer team after failing to defend against a penalty kick that costs his team the game. The goalie proceeds aimlessly through a series of pointless and reprehensible acts. I once asked a soccer-playing friend if the circumstances of the penalty kick are as they were described in the movie, and he said yes. Before the penalty kick, against a skilled opponent, the goalie has no idea which way the ball will go. He stands in the middle between the two posts and must choose which in direction he will leap, without enough information to determine whether his choice is the right one or the wrong one. The only thing he knows is that if he does not leap, he will be certain to fail.
All these theoretical questions, and the survey of the theory of questioning in general, are intended to provide the background necessary for answering the very practical question: Why do software projects fail?
Fred Brooks’s succinct answer to this is: software projects fail because software projects are very hard to do. In other words, we are asking the wrong question. A better way to phrase the problem is “Why are we still surprised when software projects fail?”
This question might be extended to other fields of endeavor:
In the discipline of software development, it is often found as one item in a larger list of software koans:
Here we have an overabundance of questions that can all be answered in the same way. Kant phrased the answer in this way:
Out of the crooked timber of humanity, no straight thing was ever made.
Aristotle phrases it thus in Metaphysics II:
Perhaps, too, as difficulties are of two kinds, the cause of the present difficulty is not in the facts but in us.
As reported at Slashdot, Archaeology magazine has an online article about evidence of zombie attacks in ancient Egypt, circa 3000 B.C. According to the article, written with a light hand, one suspects, the Palette of Narmer (above), found at Hierakonpolis, depicts this early encounter with the undead:
On the other hand, in support of the earlier date, some have claimed that the famous Palette of Narmer (ca. 3000 B.C.), also from Hierakonpolis, far from recording a victory in the war of unification of Upper and Lower Egypt, is instead a celebration of the successful repulse of a zombie attack. Although we tend to focus on the verso where the king is shown smiting a kneeling enemy, it is the other side that is actually the front. It is the side with the depression for mixing the cosmetics for adorning the cult statue, and so it would seem that the scene of the king marching in procession to view a pile of decapitated bodies is the really important message.
An interview with Max Brooks, author of The Zombie Survival Guide, conducted in April of 2006, provides a more detailed reconstruction of the evidence for man’s ongoing war with the zombies:
The theory of “Undead Evolutionary Influence” has many supporters in the paleoanthropological community. Louis Leakey even mentioned it in his ground-breaking paper “Lucy Fights a Ghoul.” However, in order to test this theory, one would have to clone our pre-human ancestors, then infect them with the zombie virus.
All discussions of these various zombie related resources tend to include the phrase tongue in cheek — a term I am not familiar with — which suggests the visceral experience of a zombie devouring its own tongue in the early stages of zombification. Whatever the true origin of this term, it is clear that zombies and tongue in cheek will indelibly be linked in my mind.
Software engineering is a young discipline. The process of developing software has been compared to building houses (software patterns), craft (pragmatic programming), gardening, manufacturing on an assembly line (ask your boss about this), and no doubt other metaphors will appear as the discipline’s teeth get lengthier.
Despite the relative youth of the profession, and despite the constantly changing nature of the game, there are nevertheless certain articles about the industry, the discipline, the phenomenon of software programming that have proven to be indispensable, if only because they continue to reveal truisms about ourselves, as coders, that we continue to forget. The simplest lesson is that software projects are always harder than we think they are going to be.
What Mary Knows is a philosophical essay by Frank Jackson which poses a thought experiment in order to explore what we mean when we speak of ‘knowing’. Mary is an imaginary person locked in a darkened room and deprived of all sense impressions. At the same time, she is fed all the knowledge she needs in order to develop a scientifically descriptive and scientifically full picture of the world. Eventually she is released from her room and encounters something she has never comes across before, despite the completeness of her education. She comes across the color yellow, and finds that she doesn’t ‘know’ what it is.
Software developers often suffer from a similar condition. While we feel fully knowledgeable about the specific things, and the specific trends, we are currently working on in our cubicles, we tend to lose sight of the big picture.
To this end, I’ve been trying to build up a library of things every developer should know. The only condition on this canon is that the articles must be publicly accessible for, to my thinking, if an article is hard to get or is something that someone else is making money off (which they certainly deserve to — I don’t have a particular beef with monetizing denken) then it is not something that everyone needs to know. That’s a bit of sophistry, of course, but it serves my purposes. Any suggestions for extending this list (quite short at the moment) are most welcome.
Here’s the beginnings of the canon:
1. Alan Turing’s 1950 essay, Computing Machinery and Intelligence — software programming constantly runs into questions about artificial intelligence, if only because, in developing business solutions, we often are trying to translate human activities into software enforced rules. This article is the original source for many of these considerations.
2. Vernor Vinge’s influential 1993 article on The Singularity — if the Turing paper tells us where software developers come from, Vinge’s paper tells us where we may be headed. It combines two notions: first Moore’s Law, which has become a metaphor for the exponential rate at which technology improves, and second the observation that we are not in control of this progress.
3. Nicholas Carr’s IT Doesn’t Matter is rather hard to find, but is important in that it questions the primacy of software development, and proposes that we, as software developers, may all be working ourselves out of a job.
4. Fred Brooks’s No Silver Bullet tells us why software development on reasonably large projects is so, very hard. More importantly, he reveals some of the preconceptions that often make it harder than it needs to be, and dooms us to frequent failures. Brooks’s essay The Mythical Man-Month is also essential reading, but I cannot find an Internet source for it, alas (fortunately, the title, for the most part, tells you what you need to know).
5. In the same vein as Fred Brooks’s skeptical essays about the nature of software development is Michael Oakeshott’s Rationality in Politics. Though not really about software programming, it does examine some of the preconceptions about rationality that underpin how we view software programming, and provides some intellectual underpinnings for Fred Brooks’s work, should you like to explore it.
The Oakeshott also makes a good segue for discussing some free podcasts worth listening to. Since I spend an inordinate amount of time in Atlanta traffic, I am always on the lookout for these sorts of things. And like the software canon above, I like my listening to be free.
First, you might enjoy Learn Out Loud’s philosophy podcast. By subscribing to this podcast through iTunes, you have the opportunity to listen to dramatic readings of the major philosophers, including Plato, Aristotle, Hobbes, Hume and Wittgenstein. This may seem, at first, like a recipe for falling asleep at the wheel, but they are actually quite engaging, thanks largely to the fact that the reader used in most of these recordings is a dead-ringer for Captain Picard from TNG.
(For some reason, I get many hits for Kierkegaard’s essay The Crowd is Untruth on my blog. For those readers, in particular, I’d like to point out that Learn Out Loud has a 3.6 MB download of K’s essay.)
You might get a kick out of Open Court Publishing’s podcast of excerpts from their Popular Philosophy series. Subscribe today, and you will have the opportunity to hear:
This is my first public translation effort, other than several fruitless attempts to translate Baudelaire as an undergraduate. This translation is based on Moncrieff’s excellent work. I beg your indulgence, and wud, natchrli, appreshi8 ani correkshunz u wud lik 2 offr: