The scientific method is one of the great wonders of deliberative thought.  It isn’t just our miraculous modern world that is built upon it, but also our confidence in rationality in general.  It is for this reason that we are offended on a visceral level at all sorts of climate change deniers, creationists, birthers, conspiracy theorists and the constant string of yahoos that seem to pop up using the trappings of rationality to deny the results of the scientific method and basic common sense.

It is so much worse, however, when the challenge to the scientific method comes from within.  Dr. Yoshitaka Fujii has been unmasked as perhaps one of the greatest purveyors of made up data in scientific experimentation, and while the peer review process seems to have finally caught him out, he still had a nearly 20 year run and some 200 journal articles credited to him.  Diederik Stapel is another prominent scientific fraudster whose activities put run-of-the-mill journalistic fraudsters like Jayson Blair to shame.  Need we even bring up the demotion of Pluto or the unrecognized difficulty of predicting Italian earthquakes (seven members of the National Commission for the Forecast and Prevention of Major Risks in Italy were convicted of manslaughter for not forecasting and preventing a major seismological event)?

It’s the sort of thing that gives critics ammo when they want to discredit scientific findings like Jerry Mahlman’s hockey stick graph in climatology.  And the great tragedy isn’t that we reach a stage where we no longer believe in the scientific method, but that we now believe in any scientific method.  Everyone can choose their own scientific facts to believe in and a general opinion that incompatible scientific positions do not need to be resolved with experimentation but rather through politics prevails.

Unconscious Thought Theory is now the object of similar reconsiderations.  A Malcolm Gladwell pet theory based on the experiments of Ap Dijksterhuis, Unconscious Thought Theory posits that we simply perform certain cognitive activities better when we are not actively cognizing.  As a software programmer, I am familiar with this phenomenon in terms of “sleep coding”.  If I am working all day on a difficult problem, I will sometimes have dreams about coding in my sleep and wake up the next morning with a solution.  When I arrive back at my work, it will effectively take me a few minutes to finish typing a routine into my IDE that I’ve been working for a day or several days trying to crack. 

I am a firm believer in this phenomenon and, as they say in late night infomercials, “it really works!”  I even build a certain amount of sleep coding into my programming estimates these days.  A project may take three days of conscious effort, one night of sleep, and then an additional five minutes to code up.  Sometimes the best thing to do when a problem seems insurmountable is simply to fire up the Internets, watch some cat videos and lolcatz the unconscious.

Imagine also how salvific the notion of a powerful unconscious is following the recent series of financial crisis.  At the first level, the interpretation of financial debacles blames excessive greed for our current problems (second great depression and all that jazz).  But that’s so 1980’s Gordon Gecko.  A deeper interpretation holds that the problem comes down to falsely assuming that in economic matters we are rational actors – an observation that has given birth (or at least a second wind) to the field of behavioral economics. 

Lots of cool counter-factual papers and books about how remarkably irrational the consumer is has come out of this movement.  The coolest has got to be not only that we are much more irrational than we think, but that our irrational unconscious selves are much more capable than our conscious selves are.  It’s a bit like the end of of Isaac Asimov’s I, Robot (spoilers ahead) where after working out all the issues with Robots someone discovers that things are just going too smoothly in the world and comes to the realization that humans are not smart enough to end wars and cure diseases like this.  After some investigation, the intrepid hero discovers that our benign computer systems have taken over the running of the world and haven’t told us because they don’t want to freak us out about it.  They want us to go on thinking that we are still in charge and to feel good about ourselves.  It’s a dis-distopian ending of sorts.

As I mentioned, however, Unconscious Thought Theory is undergoing some discreditation.  One of the rules of the scientific method is that with experiments, they gots to be reproducible, and Dijksterhuis’s do not appear to be.  Multiple experiments have not been able to replicate Dijksterhuis’s “priming effect” experiments which used social priming techniques (for instance, having something think about a professor or a football hooligan before an exam) and then evaluating the exam scores correlated with the type of priming that happened.  There’s a related social priming experiment by someone else, also not reproducible, that seemed to show that exposing people to notions about aging and old people would make them walk slower.  The failure to replicate and verify the findings of Dijksterhuis’s social priming experiments lead one inevitably to conclude that Dijksterhuis’s other experiments promoting Unconscious Thought Theory are likewise questionable.

On the other had, that’s exactly what a benevolent, intelligent, all-powerful, collective supra-unconscious would want us to think.  Consider that if Dijksterhuis is correct about the unconscious being, in many circumstances, basically smarter at complex thinking activities than our conscious minds are, then the last thing this unconscious would want is for us to suddenly start being conscious of it.  It works behind the scenes, after all. 

When we find the world too difficult to understand, we are expected to give up and miraculously, after a good’s night sleep, the unconscious provides us with solutions.  How many scientific eureka moments throughout history have come about this way?  How many of our greatest technological discoveries are driven by humanity’s collective unconscious working carefully and tirelessly behind the scenes while we sleep?  Who, after all, made all those cat videos to distract us from psychological experiments on the power of the unconscious while the busy work of running the world was being handled by others?  Who created YouTube to host all of those videos?  Who invented the Internet – and why? 

 

The following is a parable about the difference between theory and practice, which Michael Oakeshott frames as the difference between technical and practical knowledge, found as a footnote in Michael Oakeshott's essay Rationalism In Politics.  I find that it has some bearing, which I will discuss in the near future, to certain Internet debates about pedagogy and software programming:

"Duke Huan of Ch'i was reading a book at the upper end of the hall; the wheelwright was making a wheel at the lower end.  Putting aside his mallet and chisel, he called to the Duke and asked him what book he was reading.  'One that records the words of the Sages,' answered the Duke.  'Are those Sages alive?' asked the wheelwright.  'Oh, no,' said the Duke, 'they are dead.'  'In that case,' said the wheelwright, 'what you are reading can be nothing but the lees and scum of bygone men.'  'How dare you, a wheelwright, find fault with the book I am reading.  If you can explain your statement, I will let it pass.  If not, you shall die.'  'Speaking as a wheelwright,' he replied, 'I look at the matter in this way; when I am making a wheel, if my stroke is too slow, then it bites deep but is not steady; if my stroke is too fast, then it is steady, but it does not go deep.  The right pace, neither slow nor fast, cannot get into the hand unless it comes from the heart.  It is a thing that cannot be put into rules; there is an art in it that I cannot explain to my son.  That is why it is impossible for me to let him take over my work, and here I am at the age of seventy still making wheels.  In my opinion it must have been the same with the men of old.  All that was worth handing on, died with them; the rest, they put in their books.  That is why I said that what you were reading was the lees and scum of bygone men.'"

-- Chuang Tzu

Science is all about making proposals that can be tested (especially after Karl Popper's formulation of the Falsifiability Criterion), and then undergoing the experience of having that proposal rejected.  This is the essence of any successful process -- not that it eliminates errors altogether, but rather that it is able to make corrections despite these errors so that the target need never shift.

Professor Alain Connes recently gave his opinion of Xin-Jing Li's proof for the Riemann Hypothesis -- a proof which relies in part on Professor Connes' work --  in a blog comment to his own blog (by way of Slashdot):

 

Self-correcting extends to other professions, as well.  Scott Hanselman recently posted to correct an opinion he discovered here which he felt required some testing.  Through his own tests, he discovered that nesting a using directive inside a  namespace declaration provides no apparent performance benefit over placing it outside the namespace.

This leads him to draw these important lesson:

 

The sentiment recalls Ralph Waldo Emerson's memorable words:

 

 

A similar sentiment is expressed in Hobbes' Leviathan, though with a wicked edge:

 

 

We find it again expressed in Descartes' Discours de la méthode. Descartes, it might be remembered, occasionally exchanged letters with Hobbes:

 

 

Both Hobbes and Descartes formulate their defense of common sense somewhat ironically.  In a recent post, Steve Yegge takes out the irony (or perhaps takes out the kernel of truth and leaves nothing but the irony) in his argument against Joel Spolsky's widely aknowledged criteria for a desirable employee: "smart, and gets things done."

According to Yegge, the crux of the problem is this:

 

 

And yet the self-correcting process continues, on the principle that we are all smart enough, collectively, to solve our problems in the aggregate, even if we can't solve them as individuals.

Presidential candidate Barack Obama recently held a news conference to correct a misunderstanding he had made a few hours earlier about his stance on the Iraq War.  According to CNN:

 

 

This is, of course, merely a blip in the history of self-correction.  A more significant one can be found in Bakhtin's attempt to interpret the works of Rabelais, and to demonstrate (convincingly) that everyone before him misunderstood the father of Gargantua. 

Bakhtin's analysis of Rabelais in turn brought to light one of the great discoveries of his career: The Carnival -- though a colleague once found an earlier reference to the concept in one of Ernst Cassirer's works.  Against the notion of a careful and steady self-correcting mechanism in history, Bakhtin introduced the metaphor of the Medieval Carnival:

 

 

The Carnival serves to correct inequalities and resentments in society and its subcultures not by setting it upon a surer footing, but rather by affording us an opportunity to air our grievances publicly in a controlled ceremony which allows society and its hierarchical institutions to continue as they are.  It is a release, rather than an adjustment.  A pot party at a rock festival rather than a general strike.

As for the Internet, it is sometimes hard to say what is actually occurring in the back-and-forth that occurs between various blogs.  Have we actually harnessed the wisdom of crowds and created a self-correcting process that responds more rapidly to intellectual propositions, nudging them over a very short time to the correct solution, or have we in fact recreated the Medieval Carnival, a massive gathering of people in one location which breaks down the normal distinctions between wisdom and folly, knowledge and error, competence and foolhardiness?