Claudio Lassala's Blog

We normally hear that we should only be writing tests for our code, not someone else’s (external libraries or APIs). For example, if we are writing tests for code that calls out to an API, the test should mock the dependency on the API and only verify that the calls are made as expected and that the results are handled as expected. The test should NOT verify that the API produces the appropriate results for the given the input; that would be testing someone else’s code.

I agree with all of that; for unit tests.

However, I’d still consider writing integration tests against that API, NOT to test someone else’s code, but to document our assumptions about the API.

Why? Because our code relies on those assumptions. That’s a dependency.

What happens if the API implementors decide to make changes that break our assumptions? Without integration tests validating those assumptions, all our unit tests would still pass, but we could end up with either defects or inaccurate results in production (which could go unnoticed for a long time).

Another added benefit from the practice of writing such tests is that, should a new version of the API come out, evaluating risk levels of consuming the new version becomes much simpler: just run the tests against the new version.

Last but not least, say an API offers a large variety of features that could be used; having tests that describe how we use that API makes it much easier for developers to learn the details of how we depend on it. Such understanding, again, helps with both assessing risks when consuming different versions of the API, as well as assessing a potential replacement of the API.

Dependency management is very important!

Did I get your attention with that title? I hope so.

Let me clarify it: most people use code coverage for the wrong reason, making it worthless. I know I did that for a while.

Back when I first learned about writing tests, it didn’t take long until I heard about code coverage, and then the search for the magic code coverage percentage started:

“100% code coverage?”. Nope, that’s impractical
“50%, then?”. Nope, too low.
“92.35?”. Yeah, that’s more like it! Well… not!

Seriously, I’ve seen some crazy numbers as the required code coverage policy out there.

Writing tests for the sake of bringing up code coverage will NOT:

• make the code quality get better
• delivery better value to the business
• make refactoring easier

I have seen tests out there that have no assertions.

Those tests have hundreds of lines of code (usually involving some crazy, unreadable mock setups), and no assertions. Why? Simple: because developers had to satisfy the policy of XX% code coverage! The only thing those tests do is make sure no exceptions get thrown when those lines of code run. It’s pretty much a smoke test.

Such tests do NOT bring value. In many cases, the tests may be exercising lines of code for features that aren’t even used!

Think of new developers joining the project and having to go through all of that code trying to learn things, figuring out how things are done. Even existing developers after a while will have a hard time remembering why certain code is there.

So, when is code coverage worthwhile?

When writing tests for existing code!

Once a conscious decision has been made about what we should write tests for, start by writing a test that does a clean pass through the code (meaning, without it throwing exceptions). We’re likely to uncover dependencies we didn’t even know the code had. This will be a big integration test. I wouldn’t even fret about putting assertions in that test. Why? It’s very likely I don’t even know what the expect outcome of that code is at that moment.

With the first clean pass in place, look at the code coverage number. If we have about 30%, that’s too low, so we need to look into writing more tests that go through different branches of the code. Once we get to a number we feel comfortable with (that could be 70, 80, 90%… it really depends on the risks and costs of breaking changes), then we can start capturing the current outcome of that code, by writing assertions for it, bearing in mind that the outcome may not even be accurate, but it is what the code produces without any changes.

Now we can go ahead and start refactoring the code, making it more readable, without fear of breaking whatever it currently does. As we split it into smaller chunks of code, we identify opportunities to write new unit tests for those smaller pieces of logic.

Eventually, we’ll get to a point where that initially big integration test may either end up not being relevant anymore (and can be removed, replaced by the new unit tests), or, it can be refactored to something that more accurately describes the reason the code exists; the big picture.

Once the team starts using code coverage for the right reasons, then the metrics can be changed over from “Code Coverage” to “Feature Coverage”. Knowing what features are covered by test is a far more valuable practice.

If you choose to get one thing out of this post, may it be this: read Working Effective with Legacy Code, by Michael Feathers. It still one of my all-time favorite books.

Another common question I get from developers who are starting to get into testing (or even from devs who have been doing it for a while): “how do you decide what to write tests for?”.

This question normally applies to brownfield cases (existing codebase). There’s already a lot of code there. Where do we even start? Yes, maybe we write tests for the new code, but what about the existing one?!

Here’s my personal technique for it. When working with an existing codebase, I’ll ask the business:

What is the single most important feature of this product?
Think of the feature that, if broken, will either cause the business to lose money or not make money.

THAT is where we start. Those are our must-have tests.

Once the most important features have been covered by tests, the next question is:

What is the feature or area of the system that when you tell developers they need to make changes to it, they feel like running away?

That’ll usually surface areas where the code is a mess, complex, convoluted. Hence, it needs tests, so developers can feel safe making changes to it, refactoring it. Now, we only move on to this one when the features that came out of the first question above have been covered by tests.

A very common question I hear from developers is “How do I write tests for private methods?”. My immediate answer is “You don’t!”. Technically, if you’re in C# Land, you can instantiate the class and then use Reflection to call the private method. But please don’t!

Say you have some class like this one:

Of course, instead of comments, you’d have the actual code. You get the point.

You then decide to clean things up a bit and extract the “make sure all the ingredients are in” code into a separate, private ValidateIngredients method, like so:

That’s usually the moment when developers ask “how do I test that private method?”. If we have tests for the main method (DoTheMagic, in this example), then ValidateIngredients already get test coverage.

Quite often, when developers feel strong about having separate tests for a private method, there’s a clear indication that the private method should really be a public method on a separate class. Think Single Responsibility Principle.

Following the example above, we create a Validator class and move our validation logic in there:

And then we use that validator in the previously shown class:

You’ve probably noticed that we also introduced an IValidatePotion interface. Think Dependency Inversion Principle. One of the benefits here is being able to isolate tests for the AwesomenessPotion and AswesomenessPotionValidator classes.