Recently by Patrick
This is the personal blog of Patrick Lee and represents his (my) personal opinions and not those of others I may be affiliated with. There is no bio. Yes, I do regret the image-based headers regardless of any auto-generation. If you need to email me you should be able to guess the email address. These are my entries:
How far have we come?
Note: this is a sketchy rant and I haven’t backed up my arguments to any great detail. I don’t know why I feel the need to state that on my blog but… there it is.
Anyway…
The recent release of the source code for the Lunar Lander had me thinking about the relative value of code and whether programming has really advanced much in the last fifty years in terms of what people are actually achieving with software. How far have we come?
I’m probably like most “modern” programmers - when I looked at the source code for the Lunar Lander I was simultaneously astonished, horrified and embarrassed. My single adventure into the land of assembly programming left me with painful memories of hours spent writing and testing hundreds of lines of machine code for the Motorola Z80 in order to get a speaker to beep. Writing assembly code is painful. How on earth did programmers manage to write assembly code to send man to the moon?
Reading this interview from Allan Kump managed to shed some light on it for me. The Lunar Lander module code was designed by about 300 people over a 7 year period for 46 million dollars in 1967. This puts the apparent pyramid building feat of the LM into some perspective. They succeeded because they had to. It took a great deal of time, money and people to succeed.
Even then, I might question how they managed to achieve what they did without modern development techniques and engineering practices… or a drawn out beta period. How did they avoid bugs? How did they manage this task without a higher level language.
I don’t know much about their development process. I don’t think I need to and, really, I don’t find it that interesting to ponder. How about the question of bugs and no higher level language? Well….
Firstly, they did write it in a higher level language which they hand-compiled to the machine code and secondly the code was never bug free and at least one of those bugs could have been catastrophic.
However, it still disturbs me that in fifty years, we haven’t really come all that far. 1967, what else happened in 1967?
A fully mechanical ABS was in the Ferguson P99 (Mercedes had the first electronic version in their 1978 S-class).
Not that interesting. What is interesting though is this: Lisp was first specified in 1958.
You know what McCarty said about Lisp?
“Lisp will become obsolete when someone makes a more comprehensive language that dominates Lisp practically and also gives a clear mathematical semantics to a more comprehensive set of features.”
Note that it says obsolete, not untrendy.
I don’t want to get into debates around the why of Lisp not particularly been a popular success but it was first specified before work commenced on the lunar module. Would it have been a good idea to design the lunar module in Lisp?
The answer, I think, is no.
Why? The designers needed to keep touch with the input, output and with the system that they were interfacing. I don’t know how this would be achieved in Lisp at the time. I have no backing evidence for this but (this is the web and I’m writing on a blog so hey, I don’t need any) I’d say Lisp would not have been very close to the problem domain the developers were working in given that they ultimately had to hand compile it down to machine code.
What if it was 2009 instead of 1967 though? What about finding 300 developers in 2009 to write the Lunar Lander control module? What language?
For 300 developers today, chances are that Lisp still wouldn’t be chosen, or any functional language for that matter. Parentheses aside, cultural issues aside, accidents and convenience aside - the thing is that really, most developers simply do not grok the concepts at all…
I’ve been looking at some legacy code recently and the same old pattern has emerged there as I see everywhere else. The list of things that many developers seem to understand sufficiently to be comfortable using is about four things: SQL, procedures, global variables, XML.
For front-end developers just replace SQL with CSS.
Either most developers only know those things or most developers simply feel they can solve any problem with that magical quartet. Well yeah, they probably can solve all their problems with that - in a completely non-flexible, non-extensible and generally kludgy manner. Nobody cares though right? It’s all abandonware anyway, somebody will come back in ten years with an even bigger budget and redo the whole thing again exactly the same in a different system but this time use thirty times the CPU as last time.
Lisp (or another functional language) is simply too hard because it’s almost impossible to write the kind of really ugly, shit code that I know - because I’ve seen it - exists in too many code bases for legacy and modern systems written, in many cases, by decent programmers. Sure, it’s possible to write another sort of rubbish code but in my experience people who are going to write rubbish code aren’t bothered to learn clever ways to do so… The most basic features of the most accessible language will do just fine.
The solution to the nobody gets it problem, according to some pundits, was going to be Object Oriented programming. I’m actually quite confident that to a large extent that was a massive success. Why? Because, despite a bit of initial confusion it seems that a greater percentage of developers than are able to grasp other composing techniques are able to grasp the concept that instead of having lots of procedures they can partition their code into… ummm… things and attach procedures (methods) to those things/objects. Not making great use of OO but it’s a start.
Even given that, too many developers will still code by creating a bunch of procedures, initiate them from a starting point and pass any shared state they need to via whatever global mechanism the language allows. Why? Because it’s easier and they don’t need to know much of the language to do it. This is a curse that has struck many languages. JavaScript, meet the curse. Curse, meet JavaScript.
Going back to the legacy system I mentioned earlier… What is the dirty, horrible barbarian language it was written in? I’d rather not say, but… it has lambda (hand-waving version at least) among some other nice language features that help to compose software. Sure it’s got some oddities but… It has lambdas!
Perhaps this is the same tired old story paraded around but really; this legacy system might as well have been written in any language if… and only if, that language was used for nothing but lots of (basically) global procedures and variables. The developers (or most of them) had a language with some expressive power and they used none of it. Again, reminds me of JavaScript. At least JavaScript stopped short of letting people ram most of their code into SQL or XML… No wait, somebody figured out how to do that in JavaScript too.
So the new, non-legacy, replacement system is written in a modern Object Oriented language. I haven’t looked at the code base but my bet is that really it’s only Object Oriented as far the developers were forced to code in a (kinda) Object Oriented language. The attention in the code to actually doing Object Oriented is probably only as much as the developers are baptised members of the church of Object Orientation and they’re really scared of being excommunicated. Oh sure, there’s probably a bit of a perverse bastardisation of MVC shoved in there somewhere and they might map objects to the db and they probably have lots and lots of middleware and instead of a custom configuration script conjured up by a developer there’s probably nine hundred XML files instead but… you get the idea.
Of course, what the new system has is libraries and API’s ready to go that reduce the amount of actual code that any developer on the current system has to write. Now, instead of learning a language and how to program effectively in that language there’s a tendency for people to learn the language basics and then refer everything to an API that serves as a tourist dictionary. Of course, there’s more than one API and these language tourists are walking around with a bag full of dictionaries trying to communicate. This has nothing to do with the merit of one language versus another or the incredible usefulness of well designed API’s… It has everything to do with tools being applied where they should or being used as a crutch for an unrelated deficiency.
How far have we really come? I wonder.
Ever noticed how C presents with some immediacy those language features that are the first to be learned? Hello World. My Name is… Count to ten.
There are many powerful mechanisms in C and many more powerful language constructs but fundamentally the simplest of programmers really only groks just enough of a programming language to get around as a tourist. Oddly enough, the popularity of most programming languages is independent of their actual merits and most closely tied to how easily a tourist is able to find the nearest public toilet.
Perhaps because of this shortcoming the general approach over time seems to have been removing the language aspect of programming as much as possible. Given that developers are not capable storytellers in the programming language they have to use a whole heap of effort goes into IDE’s and tools by which developers are able to avoid writing programs. So great was the extent of this that the era of GUI programming came about in many arenas from WYSIWYG to a misguided hope that one day developers could create UML diagrams and have a large proportion of their code written for them.
That was the dream. People would go to the nth degree to model everything and design everything up front and the handle got cranked and out popped a UML diagram. The model would be run through a model checker and then the handle would get another crank and out popped all the code nicely done. Application finished.
Then what?
Then it’s discovered that nobody can use it because the “developers” were too busy navel gazing to check at any point if the original specification provided something that anybody could use… assuming anybody even wanted anything like it in the first place.
I think it is a mistake for anybody to think that programming is purely an exercise in symbol manipulation with developers sitting in their Searle Chinese Room translating symbols from one representation to another.
Actually, I don’t think it’s just a mistake. I think it is exactly the wrong idea. The removal of emphasis from language is wrong. Humans are good at language, exceptional at language, but instead of applying that linguistic ability the trend was towards making people push coloured blocks around a page as if the extent of their creative ability was on par with a parrot such as Alex.
Alex can’t tell stories. Programming is about telling a story.
Good programmers have a concept in their head about the software they’ve written, a narrative through a story told by the code they have written. Code smell? It’s a result of a poorly constructed story, a bad plot, wooden dialogue.
Programming should be a linguistic activity, it should be a craft and it should have a design component. A good program should read well and be entertaining all by itself. It makes absolutely no sense to me that anybody would want to remove the language component for programmers unless we really want to start using parrots to do software architecture… pushing those coloured blocks around and churning out software.
As for that Lunar Lander… I bet every single person working on that project was excited to be working on a system that was involved in sending humans to the moon. That’s a pretty good incentive for making sure it works regardless of what it was coded in.
Anyway, apparently they built the pyramids without any computers at all. I sure bet they had language though.
Dynamic inheritance?
Chris Double has some nice examples over on his blog for using dynamic inheritance in Self and Io. As JavaScript is similar to Self and Io (as far as it also has a prototypical object system) Chris mentions with regards to JavaScript:
JavaScript is also a prototype based programming language. Unlike Self or Io it only allows one object to be used as the prototype in any given object. This is stored in a hidden ‘__proto__’ member and cannot be updated once set on construction (some implementations allow changing it however). Objects are created by using the ‘new’ keyword on a constructor function that initializes the object. For now I’ll leave it as an exercise for the reader to implement the examples above in JavaScript. I’d be interested in the approaches people take.
So, interested at seeing how I might approach this I sat down to recreate the Self example in JavaScript without resorting to manipulating __proto__ which isn’t in the spec and will not work cross-browser.
Failed miserably.
Without multiple inheritance and with the prototype reference locked in for the life of the object I only ended up chasing code down rabbit holes.
Finally, in desperation I settled on what I thought was the least crappy option:
// In the absense of ES5's Object.create
function object(o) {
function F() {}
F.prototype = o;
return new F();
}
var file = (function () {
var that = {
close: function () {
console.log("closing");
this.close = null;
this.open = that.open;
},
open: function () {
console.log("opening");
this.open = null;
this.close = that.close;
}
},
stub = object(that);
stub.close();
return stub;
}());
var myFile = object(file);
myFile.open();
myFile.close();
myFile.open();
var myFile2 = object(file);
myFile2.open();
Whilst the prototype link may be fixed at construction I know that in JavaScript all the “user” object properties are dynamic and able to be set (unless it’s ES5 and the object has been frozen or the property been made non-writable) - meaning that I can dynamically turn properties “on” and “off” as needed.
This only superficially achieves the results of the example by switching functions in and out of slots on this, it does not give a flexible prototypes with traits system as outlined in Ungar’s Organizing Programs Without Classes paper. However, JavaScript is a flexible language and it wouldn’t be all that hard to implement something more sophisticated that would get the desired effect. Unfortunately, as with attempts to foist class systems onto JavaScript the end result is typically kludgy and the resulting code built on top of such an approach would not be idiomatic JavaScript.
Sure, multiple inheritance and dynamic inheritance is powerful in Self and might have gone nicely with JavaScript’s prototypical objects but they aren’t there and trying to get too clever in a scripting language can make you Br’er Rabbit stuck to a tar baby. I think that in JavaScript it’s best to stick to a shallow inheritance hierarchy that doesn’t use traits and use other compositional techniques that feel more natural to the language.
That said, somebody else may have a particularly elegant way to do this that I have completely missed. If so, I’d definitely like to see it.
Renounced magic
I started working on a quick port of blosxom to JavaScript running on v8cgi mostly because every programming language needs a blogging engine and JavaScript kinda doesn’t have one.* Despite being potentially superfluous, a post on it might have made a good read. Unfortunately I temporarily lost interest so that will have to wait for another day.
Instead, I decided to write something that’s barely related to JavaScript because, well, if I don’t write I quickly forget how to write - to point where I’ll spend ten minutes composing a two sentence email only to delete and never send it because it didn’t make sense. Anyway, here goes. I’m breaking a few rules in the following text so I can only hope the reason won’t be lost in the reflection.
In order to pay the “bills” during my years at university I used to tutor high school Maths, Physics and English. At first the syllabus for each subject was fairly close to what I’d been through myself but as each year passed I noticed bizarre things happening to each subject as they were morphed into things I barely remembered until eventually every last equation had been removed from Physics and HSC English students started telling me they were studying Blade Runner.
Truth be told, I don’t really like maths all that much (many years since leaving uni it’s still possible to give me a mild panic attack by asking me to calculate an autocorrelation coefficient) so I could probably have done without tutoring HSC Maths. I did generally enjoy tutoring English though.
The first lesson with a new student was always interesting. Generally it would begin with a fifteen minute chat with the parents - a chat that would tell me nothing about the student except that their parent had absolutely no idea what their children were doing at school. I’m not much for making light conversation but the preamble seemed important in making the parents feel comfortable and involved and they always liked it when I told them their child was talented and capable of doing well in English but had never been taught how. (I also had to teach some of the kids to spell despite the parent assuring me it wasn’t necessary because, you know, computers meant nobody needed to spell anymore.)
The student would be ready with the texts they had been assigned for the year and a stack of notes given to them by their teacher. That stack of notes would invariably contain a few sheets containing quotes that the teacher thought students were supposed to learn. For every student that I tutored though, there were only really ever two problems that needed work.
The first was that all-too-often they hadn’t actually read the text, or if they had read it they hadn’t taken the time to understand it. The solution to that was fairly easy. Tell them to read it and come back the following week promising to do nothing but sit there for an hour whilst they explained the text to you. It’s amazing what most people will do to avoid an hour of awkward silence.†
The second problem was the important one. Every single person that I tutored HSC English to had absolutely no idea how to approach composition. It might have been tempting to think that they had no idea how to write effectively because they had missed that day in school but that wasn’t the case. I don’t know if it’s been different in other times or other parts of the world but most of what these kids learnt at school was about turning up in the morning and sitting at a desk all day.‡
These kids, or teenagers, had no idea what they were supposed to write except that somebody had told them that they needed to produce something called an essay with an introduction, body and conclusion and that they should litter it with the quotes supplied in the relief package. These essays were terrible, starting with “In this essay I will talk about” and rapidly declining from there.
So, I would spend one hour telling these kids how I approached composition (not sure if that part helped) and then assigned them a truck load of writing exercises which we’d look at the following week.
The writing exercises worked.
I don’t know if any of those former students still write anything longer than an SMS but the impact of seeing smart kids with valuable ideas completely unable to put them across on paper has always stuck with me.
I think writing matters. More importantly I think good writing matters. I also think that like most of the things I grew up with it will eventually be put through the photocopier enough times that only a surgeon’s photo will remain. I think that’s a shame.
I think writing matters and that’s one of the motivations for spending the time to write (and rewrite) the posts on this site. I also think it helps me extract my mind from programming flatland… though I have little supporting evidence for that one.
Anyway, finally to the point. One of the great things about the web for me has always been that it gave more people an opportunity to explore and express the ideas they had to an engaged community.
I may be subscribing to the wrong sites but watching the declining quality of posts in my feed reader doesn’t seem encouraging. The number of link blogs, top ten lists, twitter summaries, spam, trolling and product whoring seems to be steadily increasing. I know some of these things have traditionally been considered synonymous with The Internet but for a while there it actually seemed like there was a lot of effort going to producing quality writing. Am I still looking at originals or is it all photocopies? Do people still write or is it all just out takes, bloopers and greatest hits? Does anybody care?
—
* There is actually at least one built on Helma used on reasonably high traffic production sites such as blogger.de but I am conveniently choosing to ignore them for the moment. ↑
† In hindsight there may have been some motivation in not wanting to pay me to sit there for an hour and do nothing. Hard to say. ↑
‡ I don’t mean to criticise the education system for teaching kids these two things as they are as essential for keeping a job as brushing and flossing are essential for keeping teeth. ↑
Cascading
It’s possible to criticise CSS for solving the wrong set of problems. Whilst I am talking about CSS here, I am not talking about that. CSS has been an undeniable success in terms of removing a heap of cruft from HTML. I don’t think people should be happy to stop at that though. There are problems.
Anyway, I’ve written a fair amount of CSS in my time. At least, I’ve written enough CSS that there’s a chance that I’m partly responsible for the great wad of legacy CSS that you hate working on every day.
If I might quote Toad from the claymation movie version of Wind in the Willows…
I’m not sorry so it’s no good saying I am.
At one point in time a front-end developer working on legacy code would be staring at a vast sea of nested tables created by a tool such as FrontPage - trying to figure out where in that sea they needed to add their code such that it won’t cause the layout to barf table everywhere.
These days it’s just as likely that a front-end developer will be working on legacy code and be starting at a vast sea of my CSS trying to figure out what last-minute 2am bug report had prompted the use of so much “!important”.
Actually, it’s worse than that. These days it’s just as likely that as a front-end developer you will be working on a vast sea of your own legacy CSS… written only three months ago. The thing with CSS is that most of it is legacy as soon as it’s written. This is not a criticism of people writing CSS, I’ve seen people with good ideas put a lot of thought into the best way to maintain CSS with good results. I just don’t think CSS is really helping those people solve their problems.
Let’s say that you decide to write new CSS to replace somelegacy code. What’s the usual approach?
Start with a blank file, insert standard browser reset code and off you go.
I can’t think of another coding discipline where even proficient, thoughtful coders achieve so little reuse from project to project.
Now, in the time since I last wrote any large amount of CSS that somebody else has had to work on, I’ve seen a lot of promises about the latest trend in writing or organising CSS. Indent it this way, indent it that way. Put these comments in. Group by specicifity. Group by function. Break it down this way. Break it down that way. Make it Object Oriented!
Yup… I heard that news about using component based architecture for CSS. Component based architecture makes a lot of sense. I still bet the next time you work on a completely new project you start with an empty file and CSS every single line from go to woe without reusing more than half a dozen classes from the last project you worked on.
It’d be nice if a component based approach actually resulted in front-end developers being able to compose a new site from an existing set of components. It’d be nice if there was, I dunno, meta-CSS.
Unfortunately, I can’t see that happening. Instead, it seems that some front-end developers love writing CSS selectors so much that the only thing they wish to change about that particular 4GL is the lack of variable spaghetti to add to their CSS minestone. I don’t think variables would help.
It looks like developers love those selectors so much that they’ve transported them and the development technique wholesale to other places such as JavaScript where instead of ever writing an abstraction or component themselves they start with a blank file, a selector engine and… well, you know… potato programming all the way. It looks like that because it is like that. The reason? CSS encourages it.
Just like SQL… Just like XML. CSS - and the attached mindset - seems to be one of those things that’s frequently over-applied and relied on too heavily in places where it either doesn’t belong at all or should only be a small component. There’s nothing wrong with supporting XML in a programming language but there’s definitely something wrong when XML becomes the programming language. Similarly, there’s nothing wrong with supporting CSS selectors in Javascript but there’s definitely something wrong when CSS selectors become the programming language.
Some may disagree, but I think that as a developer, I shouldn’t be trying to make everything else look like what I’m familiar with… instead I should be trying to make what I’m familiar with better. I think CSS could be better and I don’t think much more ground can be gained whilst the language remains as it is.
Space Invaders
A while ago I started experimenting with Dmitry’s handy Raphaël SVG/VML library for doing some charting.
The charting work has not yet seen the light-of-day but during some spare moments I had a look at implementing an old arcade game using Raphaël. A very rough, incomplete and buggy approximation of the original Space Invaders.
Given that this is an “alpha release” of something on my blog and also that I don’t really like IE, despite Raphaël supporting IE I never tested this game in IE and hence, in that browser you probably can’t currently get beyond the intro screen because I didn’t wire up the keyboard controls. I may get it to “work” in that browser if somebody eventually guilts me into it. I probably should get it working in IE because, well, otherwise I’m partly wasting the benefits of Raphaël over pure SVG? Anyway…
It does work in Firefox, Chrome and Opera… possiby Safari, though again… untested so far.
I used Raphaël for the graphics, SoundManager2 for the sounds and implememented the top scores over on AppJet which unfortunately seems to be having occasional problems so I might have to move that to an install of couchdb I have.
Some observations
JavaScript in the browser is probably fast enough right now for games of this type though you would have to do some optimisation. Apart from some time shift corrections my game code got no speed optimisations. That said, setTimeout and setInterval seem fairly inadequate and difficult to work with. I didn’t use a global/world clock but that is probably going to be necessary for decent synchronisation.
I didn’t spend much time on polishing the game and it shows.To make a highly playable game a huge amount time is going to be tweaking the UI, playability, difficulty progression and generally getting it polished. If web developers consider the amount of effort put into the final polishing stages for a website and multiply it by… ten… they might get an idea of how much time would go into polishing something as interactive as a game like Space Invaders.
I used a boatload of closures and prototypal inheritence via object cloning and didn’t care about doing any memory cleanup. I don’t typically use Chrome but V8 really handled that unoptimised code really well. I would expect Tracemonkey to be as good whereas the release of FF3 I’m using seems to struggle a little.
Raphaël worked nicely but in order to get the two animation states for the invaders I ended up doing them as sprites and constantly changing the sprite source for all the invaders. Without spending more than thirty seconds looking at Raphaël’s internals I suspect my code would perform better to do them as actual SVG and just change the path for each state instead of constantly adding and removing sprites.
I don’t really think it matter but the code for this Space Invaders version is not MVC. All the objects are responsible for calling the draw and sound API’s directly based on their own state. I used what is probably more of a morphic style which seemed much more natural for what I was writing.
For the game, no guarantees given as to whether or not it will crash any browser at any time. There’s a bunch of timeouts still running when you die so try to not leave your browser sitting on the high score page for any length of time just yet.
Anyway, here it is: Space Invaders.
Be warned that it has sound, the sound is a bit crap… and you can’t (yet) disable it. Also, I’m well aware that the second level is basically impossible at the moment. I’ll look at some improvements in the coming days and weeks.
If anybody wants to see the game code, without having to view source the game itself you can see that over here.
Objects, messages, users, people
One of the highlights for me at JAOO Sydney 2009 was Dave Thomas’ talk on Cloud computing. Among other things, he talked about the vast dinosaur swamp of OO middleware that Java developers will be having fun maintaining for the next thirty years.
As somebody whose programming time is mostly either UI related in JavaScript or hobby coding in whatever takes my fancy in a given week (this week it’s factor) I was in a good position to laugh in a “it’s funny because it’s not happening to me” way at the humorous insights from Dave, the self confessed former Pied Piper of Objects. Actually, I found it especially amusing because my recollection of computer science at university in the late 90’s was that almost every student at the time was begging, demanding, wailing and gnashing their teeth… to be trained as a member of this army of average Java middleware programmers.
Of course, opting to code UI in JavaScript long ago revoked my membership to the fraternity of real programmers so my hyperbole on these things can freely be dismissed as nonsense by that part of the world shepherding business objects for a living.
I was raised on objects. I like objects. I do not like what we did with them.
In particular, I don’t like that I can sit in a meeting of non-programmers who once walked past a Computer Science 101 lecture and hear somebody saying that “we should take an object-oriented approach” to the problem.
I don’t like that anybody doing architecture ever talked in terms of UML class diagrams where all the business objects were drawn and detailed and then all the messages, you know the actual stuff going on was reduced to zero. It’s all interface, all objects and nothing happening. That’s not designing a living system for code, it’s designing a pyramid to bury people in.
All objects with no messages, no configurability, no data. Everything else got moved or obliterated until finally, finally when somebody turned around and realised “Hey, we need ways to do a bunch of this other stuff” the answer that came forth was XML. So now there’s this whole universe of people programming all the important parts of their software in XML. I don’t care to take a guess at how that is going to work out.
Of course, that was all too hard for many people and instead of following the musical pipe they went off to use PHP, broke every single principle of good, bad and ugly software design - including some that weren’t invented yet, and still managed to get things done. That said, I’ve done enough work with things such as the Wordpress template system to be reasonably confident that something has gone horribly awry and that there’s a good chance that a whole community of programmers will also be keeping themselves in jobs maintaining the mess they’ve created in PHP.
Back to JAOO though, it was good to see a strong representation for dynamic languages, DSL’s, functional programming and on top of all that, it was good to hear all types of people with a good sense of history and architects talking about service oriented architectures and composition.
Now, at this point I would assume that any lispers still paying attention to current trends can only be wondering why it took so many dances around the prickly-pear tree before more people started to “get it.” The lispers may have questions such as “Did it really take fifty years for lambda to even start entering mainstream consciousness?” and “Is XML really what you made of s-expressions?” and those would be just warm up questions… what happened?
The shift to DSL’s is encouraging… I can only hope that this time we can actually get them right as almost everything that has come before has been an abomination. That goes double for most of the 4GL’s around, I’m looking at you CSS. The problem with most 4GL’s is that they were designed to suit the problem, not the people who had to solve the problem. I think one of the big challenges going forward will be how to design languages, or design the space around which languages can develop, that work better for users… while still maintaining conceptual purity and elegance.
More or less
I seem to remember once sitting in a software project meeting where the following discussion took place…
‘What day of the month is it?’ he said, turning to Alice: he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.
Alice considered a little, and then said ‘The fourth.’
‘Two days wrong!’ sighed the Hatter. ‘I told you butter wouldn’t suit the works!’ he added looking angrily at the March Hare.
‘It was the best butter,’ the March Hare meekly replied.
‘Yes, but some crumbs must have got in as well,’ the Hatter grumbled: ‘you shouldn’t have put it in with the bread-knife.’ †
Then again, perhaps it was a dream because I can’t remember if everybody was standing up or not.
Besides… in this new hyperreality it all would have been tweets and there’d be no watch and nobody watching - just a handful of autonomous iPhones tweeting away whilst their owners slept. Roomba style.
In this new hyperreality there are nine thousand eminently usable ways to do useless things and no usable ways to do anything useful. I’m not sure whether that matters though because at some point “usable” will become heterological.
If nobody knows how to write or design software there are at least two options:
Stare at our navels and argue over a neologism for “poison the language.”
Repurpose some old software. That may be all we could do regardless because really, there are probably only five different possible software applications, just like there are only seven plots and eskimos have… what was it at last count?… nine thousand words for snow.
Going back to that first scene though. If it were, in fact, a dream then in the next sequence I begin to describe the latest poster child of this trend before quickly being interrupted by a network execubot:
Silence hack! We’ve been monitoring our Nielsen families carefully and during the 12 seconds … was on screen, viewer eyeball focus was up 90%.‡
I find it encouraging that everybody is getting better at inventing value from these things. Perhaps the web really is ready to supplant all aspects of television.
If questioned, I will claim that this post may be either the Eraserhead of posts on this blog or the Plan 9.
—
† Of course, the quote is actually from Alice in Wonderland
‡ This one from Futurama Episode 5-15.
decorate-sort-undecorate
Most Perl programmers would be familiar with the Schwartzian transform which derives from the lisp decorate-sort-undecorate idiom. If you’re not familiar with it then reading the wikipedia article should be a sufficient introduction. Essentially it avoids recalculating the comparisons for each call to sort.
In 1994, Randal Schwartz replied to a question in comp.unix.shell asking how to sort on the last name for a list like this:
adjn:Joshua Ng adktk:KaLap Timothy Kwong admg:Mahalingam Gobieramanan admln:Martha L. Nangalama admrp:Michael R. Pearson adshl:Siu Lung Henry Lee adsyh:So Yi Ho adtkk:Tariq K. Khan adtkl:Terence K. H. Leung advld:Vickie L. Dwyer alaen:Alfred E. Neuman anedp:Erin D. Picard
The answer:
#!/usr/bin/perl
require 5; # new features, new bugs!
print
map { $_->[0] }
sort { $a->[1] cmp $b->[1] }
map { [$_, /(\S+)$/] }
<>;
Map, sort, map.
Anyway, here’s a version in JavaScript 1.6†. If we have the following array:
var myArray = ["adjn:Joshua Ng", "adktk:KaLap Timothy Kwong", "admg:Mahalingam Gobieramanan", "admln:Martha L. Nangalama", "admrp:Michael R. Pearson", "adshl:Siu Lung Henry Lee", "adsyh:So Yi Ho", "adtkk:Tariq K. Khan", "adtkl:Terence K. H. Leung", "advld:Vickie L. Dwyer", "alaen:Alfred E. Neuman", "anedp:Erin D. Picard"];
We can sort on the last name like so:
var sorted = myArray.map( function (e) {
return {o: e, s: e.split(/[\s]/gi).pop()};
}).sort( function (a, b) {
return a.s> b.s;
}).map( function (e) {
return e.o;
});
Note that the value used to sort is calculated in the map().
I’d be interested in further examination into how well this works in current JavaScript implementations (and any map provided for older implementations); and whether there are more appropriate ways to code it up in JavaScript.
—
† Map isn’t in until 1.6.
Plugins... and JavaFx
So I’ve complained about JavaFX before but I’m going to mention it again because, well, every time I see these examples I feel like a 2009 web user trapped in a 1995 world.
There’s an example of a Carousel implemented in JavaFX over on JavaFX.com. Here’s what you get when you try to view it:
First up, it’s loading…
Now I have to verify that I want to run org.jdesktop.applet.util.JNLPAppletLauncher. Ok. Why not? I always trust things with names like that when they ask.
Which takes me back to the loading page for about five seconds whilst it continues loading.
Until finally I see a carousel.
To adapt a quote from Bill Moggridge… kind to Java developers, cruel to users.
Varnishing JavaScript
Optimising delivery to the client is an important step in improving user experience on a website. Unfortunately, it generally falls under the domain of technical people tasked with fixing either their own problems or server problems, not normally user problems in any direct way. As such, despite tools such as YSlow it’s still all too common for people to be delivering web sites in a sub-optimal fashion in terms of delivery to users.
I’m going to be exploring some cacheability optimisations for JavaScript files using a reverse proxy called Varnish, though you could extend these to other resources as well. As a primer for what I’m talking about here, it may be a good idea to bone up on web caches and what they do.
Anyway… to the point at hand. I’m using Varnish to handle versioned URL’s for JavaScript files such that I only ever need a single copy of those files on the server but am able to get a far-future expires version out on the web purely by changing the path I use to refer to the resource. For example, let’s say I have a JavaScript resource that normally lives at http://sand.boundvariable.com.au/js/core.js. Using this Varnish setup I can get long lived versions of this file onto the web by referencing it at a URL like http://sand.boundvariable.com.au/js/v.1.core.js. Note that I change the file path instead of adding a query parameter as query parameters tend to prevent caches from keeping a copy.
The point here is to maximise the time for which particular resources are kept in client-side caches. Instead of trying to get URL’s to expire as often as I expect them to change I would instead start referencing the same resource on a new URL.
I’ve got an example setup on a Gentoo box running Varnish in front of Cherokee lighttpd. Apart from the Varnish config subs listed below, both Varnish and Cherokee lighttpd are in default setup with Varnish on port 80 proxying to Cherokee lighttpd on port 8080.
So, the Varnish config…
Here’s the receive:
sub vcl_recv {
set req.backend = default;
if (req.url ~ "\.v\.[0-9]*") {
set req.url = regsub(req.url, ".v.[0-9]*", "");
set req.http.magicmarker = "1";
remove req.http.cookie;
}
}
This tests for the presence of a version in a file path, sets a marker for the backend and removes the version.
Here’s the fetch:
sub vcl_fetch {
if (req.http.magicmarker == "1") {
unset obj.http.magicmarker;
set obj.http.expires = "Mon, 1 Jan 2018 20:00:00 GMT";
unset obj.http.cache-control;
unset obj.http.etag;
}
}
This checks for the marker and on finding it removes it and sets some header optimisations with a far future Expires.
You can see an example at http://sand.boundvariable.com/js/core.v.106022009.js. Open it up and check out the headers. For comparison, have a look at the headers on http://sand.boundvariable.com/js/core.js which is doing a straight pass to Cherokee.
You can also check out the cacheability of the file via a report on ircache.
The danger in this approach served as-is, is that somebody could fill the cache and downstream proxies with stale copies of files. Also, this setup in the current state is not appropriate for making sure that every user getting a resource at a particular URL is getting the same result.
Determining if those two points are problems for particular cases and the solutions for them is left as an exercise for the reader.
Thoughts on
tagged with
- event handling
- observations
- language
- opinion
- agile
- baudrillard
- cacheability
- correctness
- dependencies
- focus
- forms
- game
- gentoo
- growth
- idiom
- intentionality
- interface
- language design
- lighttpd
- nietzsche
written in
- August 2009
- July 2009
- June 2009
- May 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- August 2008
- July 2008
- June 2008
