Did I get your attention with that title? I hope so.
Let me clarify it: most people use code coverage for the wrong reason, making it worthless. I know I did that for a while.
Back when I first learned about writing tests, it didn’t take long until I heard about code coverage, and then the search for the magic code coverage percentage started:
“100% code coverage?”. Nope, that’s impractical
“50%, then?”. Nope, too low.
“92.35?”. Yeah, that’s more like it! Well… not!
Seriously, I’ve seen some crazy numbers as the required code coverage policy out there.
Writing tests for the sake of bringing up code coverage will NOT:
- make the code quality get better
- delivery better value to the business
- make refactoring easier
I have seen tests out there that have no assertions.
Those tests have hundreds of lines of code (usually involving some crazy, unreadable mock setups), and no assertions. Why? Simple: because developers had to satisfy the policy of XX% code coverage! The only thing those tests do is make sure no exceptions get thrown when those lines of code run. It’s pretty much a smoke test.
Such tests do NOT bring value. In many cases, the tests may be exercising lines of code for features that aren’t even used!
Think of new developers joining the project and having to go through all of that code trying to learn things, figuring out how things are done. Even existing developers after a while will have a hard time remembering why certain code is there.
So, when is code coverage worthwhile?
When writing tests for existing code!
Once a conscious decision has been made about what we should write tests for, start by writing a test that does a clean pass through the code (meaning, without it throwing exceptions). We’re likely to uncover dependencies we didn’t even know the code had. This will be a big integration test. I wouldn’t even fret about putting assertions in that test. Why? It’s very likely I don’t even know what the expect outcome of that code is at that moment.
With the first clean pass in place, look at the code coverage number. If we have about 30%, that’s too low, so we need to look into writing more tests that go through different branches of the code. Once we get to a number we feel comfortable with (that could be 70, 80, 90%… it really depends on the risks and costs of breaking changes), then we can start capturing the current outcome of that code, by writing assertions for it, bearing in mind that the outcome may not even be accurate, but it is what the code produces without any changes.
Now we can go ahead and start refactoring the code, making it more readable, without fear of breaking whatever it currently does. As we split it into smaller chunks of code, we identify opportunities to write new unit tests for those smaller pieces of logic.
Eventually, we’ll get to a point where that initially big integration test may either end up not being relevant anymore (and can be removed, replaced by the new unit tests), or, it can be refactored to something that more accurately describes the reason the code exists; the big picture.
Once the team starts using code coverage for the right reasons, then the metrics can be changed over from “Code Coverage” to “Feature Coverage”. Knowing what features are covered by test is a far more valuable practice.
If you choose to get one thing out of this post, may it be this: read Working Effective with Legacy Code, by Michael Feathers. It still one of my all-time favorite books.
Claudio Lassala's Blog 