Skip to content

Tests That Earn Their Keep

Overview

Section titled “Overview”

Almost everyone agrees that automated tests matter, and TDD is often presented as the way to get them. In practice, testing well is harder than it seems. A good test suite takes sustained effort and attention. Left to grow unchecked, poor tests make a system harder to change rather than easier, and they give a false sense of security, especially when the goal becomes a coverage number.

The challenge isn’t whether to test. It’s keeping the test suite an asset rather than a burden: tests that check the code does what’s intended, without locking in how it does it.

Context

Section titled “Context”

This surfaces most on long-running projects. There’s always some delivery pressure, but on a project that runs for years, it keeps adding up, and most codebases are around for longer than anyone expects.

Tests are where that pressure accumulates most quietly. Refactoring production code to pay down technical debt is relatively easy to justify. Refactoring and improving tests is harder to justify, even though it’s the same investment. In John Ousterhout’s terms, it’s strategic rather than tactical programming: continuing to invest in code quality, including the quality of the tests.

It also surfaces wherever testing is applied dogmatically, whether that’s mandated TDD, strict rules about what counts as a unit test, or a coverage target.

The Tensions

Section titled “The Tensions”

  • Isolation vs. confidence: the point of developer tests is to check that the code does what’s intended. Stubs and fake objects have their place for isolating the subject under test, but the overuse of mocks is an obvious problem, and being dogmatic about isolation misses the point. If a meaningful test needs to cover several “units” together, so be it. The measure of a good test isn’t whether it tests a strict unit, but whether it adds value and confidence to the delivery of the project.

  • Speed vs. meaning: fast tests matter, because a slow suite stops developers running it often. But a fast unit test that doesn’t really test anything is worse than a slower, larger test that tests something meaningful. Michael Feathers’ well-known rules say a test isn’t a unit test if it talks to a database or touches the file system. I think that distinction is irrelevant, and can even be counterproductive, if it pushes teams to mock away the parts where the interesting failures happen.

  • Avoiding duplication vs. explicit tests: the design principles for test code are slightly different from production code. Production code benefits from higher abstraction, avoiding duplicate logic and centralising business rules. In tests, being explicit about the fixture and the expected outcome matters more than avoiding duplication. I’ve seen suites where so much fixture setup was shared in utilities that the tests became unreadable. The opposite problem is just as real: setting up data that’s irrelevant to the test obscures what it’s actually testing.

  • Test-first discipline vs. finding the best design: TDD is often described as a design tool, with testability as a barometer of good design. For me, that’s only true for a subset of development work, where there’s real business or rich domain logic. John Ousterhout puts the counter-argument strongly in A Philosophy of Software Design: “The problem with test-driven development is that it focuses attention on getting specific features working, rather than finding the best design”, which he calls “tactical programming pure and simple, with all of its disadvantages”. I agree with the danger, but I don’t hold the view quite as strongly. He makes a similar point about agile (“one of the risks of agile development is that it can lead to tactical programming”), and there too I’d accept the risk but still advocate an agile approach. To be fair, the approach he recommends is itself incremental, adding and refining a few abstractions with each step.

Where I’ve Landed

Section titled “Where I’ve Landed”

Tests have to earn their keep. They have a cost, and if they don’t justify it, they should be deleted. Deleting old tests isn’t a bad thing, if they’re replaced with better ones or never really tested anything in the first place.

Test at the appropriate level. The traditional test pyramid, with lots of unit tests at the base, is overused. I agree much more with Kent C. Dodds’ “testing trophy”, which puts the emphasis on tests in the middle, but the real principle is simply to write each test at the level where it gives the most confidence. I’m not a fan of the term “integration tests” either, because of its connotations. I prefer “developer tests”, which I picked up from Ian Cooper and which describes them more accurately.

Unit test genuine logic. Where a class or function has real business logic, it makes sense to test that logic in isolation, in the spirit of unit testing. But only where there’s genuine logic to test.

Use TDD where it helps. I don’t always practise it. TDD has its place when the functionality is more complex. For basic CRUD or reporting-style features, where the behaviour is simple and well understood, I don’t think strict TDD is needed. There should always be some level of automated test coverage, but it doesn’t have to follow a TDD workflow.

Treat tests as an asset. That means organising them well, naming them clearly, keeping each at the right level, and giving each clear assertions and outcomes. Test data builders with sensible defaults, customised per test, keep fixtures readable (see Builder). None of this is a one-off: the test code needs the same ongoing attention as the production code throughout the project.

Keep names readable, without a mandated taxonomy. A test name should describe the test, typically along the lines of the “When…” part of Given-When-Then. I’ve experimented with Gherkin-style naming for fixtures and methods, but stopped short of structuring tests around Given-When-Then, and I wouldn’t mandate a detailed naming convention. Clear and explicit is the goal.

Readable tests over executable specifications. ATDD and BDD haven’t earned their place for me. I’ve looked at SpecFlow, but I don’t buy into the value of tests written in English and pattern-matched onto code. In practice, I haven’t found business stakeholders who would read or review them. We also tried Gherkin syntax for user story acceptance criteria, and even that was cumbersome for stakeholders. Well-written tests that developers can understand are more valuable.

AI-generated tests still need good test design. Tests seem a natural place to use AI, but the temptation is simply to let it loose. If “slop” is a concern in AI-generated production code, it’s an even easier trap in test code, which never runs in production. Every reason above for investing in the quality of tests still applies, so any AI approach to testing has to follow good test design. It’s an area I’m actively exploring, and the test that matters is the same as for hand-written tests: does each one earn its keep?

What I’ve Seen Go Wrong

Section titled “What I’ve Seen Go Wrong”
  • Test suites that grow unchecked until they become a burden, making the system harder to change rather than easier.
  • Heavy use of mocks that locks tests to the implementation, so they break on every refactor but not when the behaviour changes.
  • Chasing a coverage number, which produces tests that don’t really test anything and a false sense of security.
  • Shared fixture utilities that grow until no one can read what a test is setting up.
  • Improving the tests never quite getting prioritised against production work.

References

Section titled “References”