The Common Cold, Common Stock, and TDD; Or: the ‘Cargo-Culting’ of Automated Testing

Nearly everyone in software engineering, whether they use automated testing and/or TDD (test-driven development) or not, should read Test Double’s post on “The Failures of ‘Intro to TDD'”. It encapsulates quite nicely the best intentions of TDD, and demonstrates that Test Double are smart people working hard at the Craft of Software.

But… here’s my problem with it: saying that “TDD is about solving problems incrementally with a side effect of total regression safety” is a lot like saying that “Airplanes are about having 5 hours of distraction-free work time with a side-effect of getting you from one place to another.” Both statements are true independently, but it confuses the purpose with the side-effect, even if the side-effect is overall more important to the success of the system. (There are days when having 5 hours of distraction-free work time does indeed feel like it changed my life more than did merely getting from New York to San Francisco. But it’s still merely a side-effect of my trip, not its purpose.)

Solving problems incrementally (and sustainably, and extensibly) is what software engineering, as a discipline, is all about. It is the beginning, the middle, and the end. We can easily argue that the “software” trade is actually about getting products out the door and solving business problems. But that’s not “engineering”, any more than a homeowner slapping together a shed with two-by-fours and particle board is engaging in the discipline of “architecture” or “structural engineering.” There’s a lot of overlap, but they’re not the same thing. (The concept of “technical debt” could be almost comprehensively defined as one of the two places in the Venn diagram where the domains of “software” and “engineering” do not fully overlap.)

But we don’t have a good, pithy name for what I’m talking about. “Sound Engineering Practices” is so… yawn. But “TDD”…  now there’s a handy little moniker. It’s a thing we can tie up with a bow and hand around our office or community, bestowing our magnanimous gift of knowledge upon everyone we see, and having them think: he’s so clever. Why don’t we have anywhere near as handy a brand or abstraction for the idea that “software engineering as a whole is about solving problems incrementally and sustainably and extensibly”?

And yet, that’s exactly what the Test Double article proves: we need a brand for the whole “incrementally and sustainably and extensibly” thing. Because automated testing is NOT actually about that. Testing is, in fact, about preventing regressions. We have accidentally discovered that we can (not always, but often) write better code when we do testing, because testing leads us into a way of thinking about our units that is conducive to better code. And because that is a Good Thing, we created a slightly dogmatic sub-discipline to reinforce its value and effects. And the engineer thus blessed ‘TDD’ and called it holy.

But this is like the idea in some cultures that letting your feet get too chilly causes the common cold. Rhinovirus causes the common cold. Period. But there is some evidence that chilling of the extremities somehow lowers immune defenses against viral infection, thus increasing the likelihood of cold symptoms. And there is lots of evidence that dedication to automated testing increases the likelihood of good code. But bad code and rhinovirus are both the pathogens we should actually be concerned about. Keeping your feet warm, and maintaining good test coverage are both very handy things to do. But letting your hooves get frigid doesn’t cause colds any more than a lack of testing causes bad code. You don’t actually have to have warm feet or testing to get to your objectives. They’re just helpful tools that often contribute to getting to a desired end-state.

Which is why the post on Test Double is missing the forest for the trees. On the one hand, the author has noticed that merely teaching people how to test isn’t succeeding in getting students to his real objective. What they’ve has failed to notice is that their real objective isn’t actually about testing. It’s about writing good software. They come very close to noticing it, too:

If this is where we’re bound to end up, why not head in that direction from the outset? My approach to TDD considers this, and could be described as an exercise in reductionism.

Try swapping the word “software design” for “TDD”, there and elsewhere in the post, and the whole thing suddenly becomes much more general, and thus much more useful. Now you’re talking about the whole domain, not merely something that overlaps with it.

Yet, we have all kinds of training courses about testing and TDD. Books, blogs (not only the author’s blog but their company is called “Test Double” for chrissakes), seminars, scholarly articles, Meetups, plenary sessions, etc., etc. Which is not a Bad Thing at all! But when we start thinking that the reason we do Testing (or even TDD, which is a more strategic kind of deal) is that it makes us write Good Code, we’re confusing the side-effect for the actual objective. This writer understands that they’re not getting where they want to be by teaching TDD alone, but they’re missing that “TDD” is the wrong objective – and the wrong class name – in the first place.

Test Double even points out, astutely, that you can have all the TDD you want, and it won’t necessarily make your code better or cleaner and easier, unless you’ve also focused on teaching the discipline of Good Code. They call it “reductionism”, but that’s really what they’re after. But why do that if TDD and Getting to Good Code are really just the same thing?

Because they aren’t. TDD isn’t making it easier for people to write Good Code. Good Code is making it easier for people to do TDD. The one can exist by itself, without the other. But the reverse is impossible. You can write bad code that’s almost entirely un-testable; whereas you’ll have a very, very hard time writing a truly excellent, large-scale codebase that is maintainable and extensible without it naturally lending itself to being relatively-well mocked and tested.

In this way, TDD is a bit like playing a game of hot/cold. The person who’s telling you whether you’re getting nearer to – or farther from – your objective in the hot/cold game isn’t telling you how to get nearer to it. Similarly, if you’re the CEO of a publicly-traded company, the stock market doesn’t tell you how to appease the demons and make your company more profitable and more valuable. In both situations, you just have a heuristic for knowing (when they function as intended) whether you’re overall succeeding or failing, but not what the right path to success is. TDD is a bit like those heuristics. It allows us to infer whether we’re getting nearer to, or further from, our actual objective: Good Code. But in and of itself, it teaches us nothing at all about how to get there. That’s a discipline you have to learn separately. (There is no “invisible hand” that magically brings quality to code when you run tests on it, any more than there’s one that magically removes inefficiencies from a market corrupted by power and collusion.)

We don’t teach classes on fertilizer and then get around to mentioning that fertilizer happens to cause healthier crops and more beautiful gardens. We start from those obvious objectives and teach fertilizer as one of the many techniques that gets you there. We should be teaching classes and writing books about what good code is and what it looks like (the way Venkat Subramaniam does, for instance), and then discussing TDD as just one of the tools in your belt that can help you notice where you’ve gone wrong.



  • Let me be explicit: Test Double’s article actually has some really great things to say about how to improve your approach to building software. The whole outside-in concept is really, really good. Just substitute “good software design” for “TDD” as the reason why you’re doing it, and you’re all set.
  • I have a theory about why TDD is so successful (probably seen elsewhere, but I can’t remember where). It’s like the gamification of software, or tiny little pellets of serotonin to reinforce positive behaviors. Write a test, then make it pass: ding! Write lots of tests and make them all pass: ding! Ding! Ding! I think it’s basically the same reason why people come back and write for StackOverflow more than once. It’s all about coming back each day and seeing a few new reputation points and medals in your account. Ones that relate to something you actually are more highly invested in than being the ‘Mayor’ of your local pub. <Disappears for a moment to write post recommending obscure but occasionally useful design pattern>: Ding! And lord knows we mightn’t ever bother to blog if not for WordPress metrics indicating someone occasionally reads the damn things…
  • For anybody unfamiliar with the idea of “Cargo-Culting” in software, check out Wikipedia. Or perhaps just this more visual, narrative introduction to the original concept.


Tagged , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: