A+ for Failure

This week I spent a considerable amount of time working on a piece of code for an upcoming Processing.js workshop.  I wanted to re-purpose a jQuery plugin for accessing and displaying a live Twitter feed so that it could be used in Processing code.  I got it working pretty quickly, and then Jon suggested I redo it as a more stand-alone library.  "No problem, shouldn't be hard."

One of the things you do most often as a programmer of real-world code is take something written by someone else and make it do something new, hack it into a different system, surgically remove a portion of it to transplant somewhere else, etc.  You spend most of your time working with code written by other people and try to get it working with things you're written.  My favourite kind of coding is getting components to work in systems that they weren't designed for in the first place.  It's a sickness.

When doing this kind of work, layers of abstraction are your friend.  If you can find the right level of abstraction, you can often do a magic trick where you levitate the bulk of the program and carefully slip a new layer underneath.  The code is mostly unchanged, except for some portion of the foundation, which now sits on new ground.  If you've never done work like this, seek some out.  It's incredibly rewarding to see the mass of some code suddenly snap into action while sitting atop a new operating system or framework.

But it isn't always that clean.  Sometimes you have to change things from the middle out.  What's so much fun about the case I listed above is that you need to understand almost none of the code; your only concern are the bits that touch the new foundation.  But in the middle-outward case, you have to do the opposite.  Suddenly, you have to understand a web of complexity, and be able to change it without bringing the whole thing down around you.

This second type of coding is where I found myself with the twitter library.  The code I was porting was nothing like what I wanted.  It worked perfectly, but it wasn't structured in a way that allowed me to isolate a small bit and have it work.  I spent more time than I'd care to admit making it work.  When I was done I hit a strange error.  The browser was locked in an infinite loop of some kind, continually loading a script over and over.  I debugged it for quite a while before giving up and thinking that my approach was wrong.  "I'll rewrite it," I thought.  So I threw away all my code and started again.  And even though I used some different algorithms and a modified approach, the second attempt was the same as the first.

I tell my students that it's OK to fail.  I tell them to write about their failures, instead of hiding them.  And even though I don't enjoy it any more than they do, I try to do the same.  When you fail like this--and it's not 'if' but 'when'--you need some strategies.  My first strategy was to distrust my own code, and I rewrote it.  My second strategy was to distrust the code around me.  I debugged things related to my code for hours.  My third strategy was to talk to some friends about it.  At a certain point you can't see your own errors anymore.  It's hard to get enough distance from your own code to really see what's there, and what isn't.

Mike is one of the most meticulous debuggers I know, and I was pleased when he answered my call for help.  I pushed my code to a new branch on github called 'crap' and walked away from my computer.  When I came back later that night, Mike had found the problem.  The bug was very subtle, and introduced an edge case to a piece of code we know works already (when I say know I mean "know").  My problem was that I wasn't prepared to widen my circle of distrust in this code far enough.  The issue was out beyond the boundary of my work, and it took imagination to get that far removed from what was clearly being caused by something I had done.

Programming is hard, and it needs tremendous patience, time, and a cadre of programmers you can rely on for good advice.  Don't be afraid to fail, just do it ways that are likely to succeed.