How far have we come?

Note: this is a sketchy rant and I haven’t backed up my arguments to any great detail. I don’t know why I feel the need to state that on my blog but… there it is.


The recent release of the source code for the Lunar Lander had me thinking about the relative value of code and whether programming has really advanced much in the last fifty years in terms of what people are actually achieving with software. How far have we come?

I’m probably like most “modern” programmers - when I looked at the source code for the Lunar Lander I was simultaneously astonished, horrified and embarrassed. My single adventure into the land of assembly programming left me with painful memories of hours spent writing and testing hundreds of lines of machine code for the Motorola Z80 in order to get a speaker to beep. Writing assembly code is painful. How on earth did programmers manage to write assembly code to send man to the moon?

Reading this interview from Allan Kump managed to shed some light on it for me. The Lunar Lander module code was designed by about 300 people over a 7 year period for 46 million dollars in 1967. This puts the apparent pyramid building feat of the LM into some perspective. They succeeded because they had to. It took a great deal of time, money and people to succeed.

Even then, I might question how they managed to achieve what they did without modern development techniques and engineering practices… or a drawn out beta period. How did they avoid bugs? How did they manage this task without a higher level language.

I don’t know much about their development process. I don’t think I need to and, really, I don’t find it that interesting to ponder. How about the question of bugs and no higher level language? Well….

Firstly, they did write it in a higher level language which they hand-compiled to the machine code and secondly the code was never bug free and at least one of those bugs could have been catastrophic.

However, it still disturbs me that in fifty years, we haven’t really come all that far. 1967, what else happened in 1967?

A fully mechanical ABS was in the Ferguson P99 (Mercedes had the first electronic version in their 1978 S-class).

Not that interesting. What is interesting though is this: Lisp was first specified in 1958.

You know what McCarty said about Lisp?

“Lisp will become obsolete when someone makes a more comprehensive language that dominates Lisp practically and also gives a clear mathematical semantics to a more comprehensive set of features.”

Note that it says obsolete, not untrendy.

I don’t want to get into debates around the why of Lisp not particularly been a popular success but it was first specified before work commenced on the lunar module. Would it have been a good idea to design the lunar module in Lisp?

The answer, I think, is no.

Why? The designers needed to keep touch with the input, output and with the system that they were interfacing. I don’t know how this would be achieved in Lisp at the time. I have no backing evidence for this but (this is the web and I’m writing on a blog so hey, I don’t need any) I’d say Lisp would not have been very close to the problem domain the developers were working in given that they ultimately had to hand compile it down to machine code.

What if it was 2009 instead of 1967 though? What about finding 300 developers in 2009 to write the Lunar Lander control module? What language?

For 300 developers today, chances are that Lisp still wouldn’t be chosen, or any functional language for that matter. Parentheses aside, cultural issues aside, accidents and convenience aside - the thing is that really, most developers simply do not grok the concepts at all

I’ve been looking at some legacy code recently and the same old pattern has emerged there as I see everywhere else. The list of things that many developers seem to understand sufficiently to be comfortable using is about four things: SQL, procedures, global variables, XML.

For front-end developers just replace SQL with CSS.

Either most developers only know those things or most developers simply feel they can solve any problem with that magical quartet. Well yeah, they probably can solve all their problems with that - in a completely non-flexible, non-extensible and generally kludgy manner. Nobody cares though right? It’s all abandonware anyway, somebody will come back in ten years with an even bigger budget and redo the whole thing again exactly the same in a different system but this time use thirty times the CPU as last time.

Lisp (or another functional language) is simply too hard because it’s almost impossible to write the kind of really ugly, shit code that I know - because I’ve seen it - exists in too many code bases for legacy and modern systems written, in many cases, by decent programmers. Sure, it’s possible to write another sort of rubbish code but in my experience people who are going to write rubbish code aren’t bothered to learn clever ways to do so… The most basic features of the most accessible language will do just fine.

The solution to the nobody gets it problem, according to some pundits, was going to be Object Oriented programming. I’m actually quite confident that to a large extent that was a massive success. Why? Because, despite a bit of initial confusion it seems that a greater percentage of developers than are able to grasp other composing techniques are able to grasp the concept that instead of having lots of procedures they can partition their code into… ummm… things and attach procedures (methods) to those things/objects. Not making great use of OO but it’s a start.

Even given that, too many developers will still code by creating a bunch of procedures, initiate them from a starting point and pass any shared state they need to via whatever global mechanism the language allows. Why? Because it’s easier and they don’t need to know much of the language to do it. This is a curse that has struck many languages. JavaScript, meet the curse. Curse, meet JavaScript.

Going back to the legacy system I mentioned earlier… What is the dirty, horrible barbarian language it was written in? I’d rather not say, but… it has lambda (hand-waving version at least) among some other nice language features that help to compose software. Sure it’s got some oddities but… It has lambdas!

Perhaps this is the same tired old story paraded around but really; this legacy system might as well have been written in any language if… and only if, that language was used for nothing but lots of (basically) global procedures and variables. The developers (or most of them) had a language with some expressive power and they used none of it. Again, reminds me of JavaScript. At least JavaScript stopped short of letting people ram most of their code into SQL or XML… No wait, somebody figured out how to do that in JavaScript too.

So the new, non-legacy, replacement system is written in a modern Object Oriented language. I haven’t looked at the code base but my bet is that really it’s only Object Oriented as far the developers were forced to code in a (kinda) Object Oriented language. The attention in the code to actually doing Object Oriented is probably only as much as the developers are baptised members of the church of Object Orientation and they’re really scared of being excommunicated. Oh sure, there’s probably a bit of a perverse bastardisation of MVC shoved in there somewhere and they might map objects to the db and they probably have lots and lots of middleware and instead of a custom configuration script conjured up by a developer there’s probably nine hundred XML files instead but… you get the idea.

Of course, what the new system has is libraries and API’s ready to go that reduce the amount of actual code that any developer on the current system has to write. Now, instead of learning a language and how to program effectively in that language there’s a tendency for people to learn the language basics and then refer everything to an API that serves as a tourist dictionary. Of course, there’s more than one API and these language tourists are walking around with a bag full of dictionaries trying to communicate. This has nothing to do with the merit of one language versus another or the incredible usefulness of well designed API’s… It has everything to do with tools being applied where they should or being used as a crutch for an unrelated deficiency.

How far have we really come? I wonder.

Ever noticed how C presents with some immediacy those language features that are the first to be learned? Hello World. My Name is… Count to ten.

There are many powerful mechanisms in C and many more powerful language constructs but fundamentally the simplest of programmers really only groks just enough of a programming language to get around as a tourist. Oddly enough, the popularity of most programming languages is independent of their actual merits and most closely tied to how easily a tourist is able to find the nearest public toilet.

Perhaps because of this shortcoming the general approach over time seems to have been removing the language aspect of programming as much as possible. Given that developers are not capable storytellers in the programming language they have to use a whole heap of effort goes into IDE’s and tools by which developers are able to avoid writing programs. So great was the extent of this that the era of GUI programming came about in many arenas from WYSIWYG to a misguided hope that one day developers could create UML diagrams and have a large proportion of their code written for them.

That was the dream. People would go to the nth degree to model everything and design everything up front and the handle got cranked and out popped a UML diagram. The model would be run through a model checker and then the handle would get another crank and out popped all the code nicely done. Application finished.

Then what?

Then it’s discovered that nobody can use it because the “developers” were too busy navel gazing to check at any point if the original specification provided something that anybody could use… assuming anybody even wanted anything like it in the first place.

I think it is a mistake for anybody to think that programming is purely an exercise in symbol manipulation with developers sitting in their Searle Chinese Room translating symbols from one representation to another.

Actually, I don’t think it’s just a mistake. I think it is exactly the wrong idea. The removal of emphasis from language is wrong. Humans are good at language, exceptional at language, but instead of applying that linguistic ability the trend was towards making people push coloured blocks around a page as if the extent of their creative ability was on par with a parrot such as Alex.

Alex can’t tell stories. Programming is about telling a story.

Good programmers have a concept in their head about the software they’ve written, a narrative through a story told by the code they have written. Code smell? It’s a result of a poorly constructed story, a bad plot, wooden dialogue.

Programming should be a linguistic activity, it should be a craft and it should have a design component. A good program should read well and be entertaining all by itself. It makes absolutely no sense to me that anybody would want to remove the language component for programmers unless we really want to start using parrots to do software architecture… pushing those coloured blocks around and churning out software.

As for that Lunar Lander… I bet every single person working on that project was excited to be working on a system that was involved in sending humans to the moon. That’s a pretty good incentive for making sure it works regardless of what it was coded in.

Anyway, apparently they built the pyramids without any computers at all. I sure bet they had language though.

CoffeeScript in Action

CoffeeScript in Action book cover

I'm the author. Get it from Manning.