Posts Tagged testing
At the last Virtual Brown Bag we talked about Test-Driven Development (TDD), Behavior-Driven Development (BDD), and related subjects (you can watch it here).
I’ve been very fortunate in the last 5 years working on projects where I simply write the tests as part of the work. It’s just something I do.
Is it a new feature?
If I need to work on a new feature, I write the tests to spec out my understanding of the feature. If I can’t write the test, most likely I don’t understand what the feature is, and if that’s the case, I shouldn’t write any code to begin with. Instead, I should pair up with the QA guys and/or the business analysts (or work directly with my client), and make sure I understand not only what the feature is, but also why it is important to the business.
Is it a defect?
If I need to work on fixing a defect, I’ll do my best to write a test that reproduces the issue. Granted, some issues (such as multi-threading issues) are really difficult to reproduce in test, but hopefully those types of issues aren’t the core of my projects. Whenever there is an area that’s hard to test, I’ll try to isolate as much of the code as possible, decreasing the surface that’s hard to be tested.
The biggest lessons I’ve learned from testing
While I think I’ve learned quite a bit about the technical aspects of writing tests, and have been harvesting so many positive outcomes, such as writing better code, I don’t think that alone makes me a better developer.
I believe that the act of taking a step back to think about how I’m going to write tests has made me take an extra step back to understand what it is that I’m building. This has made me collaborate a lot more with people who actually understand the business. I don’t really care much anymore about using this or that framework/language/etc just because it’s cool; I care much more about building features that will bring value to my client. Seeing a client smile because you gave him or her the feature needed is very rewarding.
A recent experience
In the last 3 months I’ve been working part time on a project where I’m the only developer, and I have no QA, no business analyst, no project manager. Here are some bullets from my personal project notes:
- The first phase of this project has gone into production two weeks ago
- The client is beyond happy
- Very few defects were found in production (and corrected immediately)
- There are close to 400 tests
- I do NOT have everything under test. I’m keeping Controllers very thin, calling out to Commands, Services, etc. Things the controllers call ARE under test.
The whole testing thing…
As I finished writing this post, I just remember I actually wrote another post about the whole testing thing 8 years ago! Time sure flies by…
Three years ago I blogged about my thoughts at time on the whole testing thing. I have stuck with the practices and really believe on it. I’ve tried different tools and approaches, and I keep trying different things as I go. Behavior-Driven Development (BDD) seemed like my next step, but it didn’t take me long to realize MS-Test didn’t give me what I needed in order to start to get into that practice. There wasn’t much I could do as I was stuck with it.
When C# 3.0 came out, I’ve learned how I could improve my tests using extension methods, building some sort of fluent DSL to test certain things. Eventually, I was able to migrate to xUnit, along with SubSpec, and then I was able to write tests a lot better, following the “given-when-then” style. Writing new features and certain components following that style has been great; I believe everything I wrote following that approach has a much better design and quality.
I had heard Rails developers gave a lot more importance to BDD and testability than .NET developers, so that was one of the driving factors that encouraged me to try Rails.
After getting a little more comfortable with Rails and Ruby, I started to devote more time to RSpec and Cucumber, without really understanding much what they were exactly. So far, I’m enjoying using both. I’m almost done reading the RSpec book (I’ll post a quick review once I’m done) and writing as many tests as I can, but I still have lots to learn; I’m still at that stage where I do know there must be a better way to write something, but I’m not to a point where I know how to do it.
I am writing lots of “features” formatted so that I can drop them into Cucumber .feature files, and then implementing them. I’m finally realizing some benefits of BDD that were apparent to me before. Up until recently, I was seeing BDD mostly as a way to write tests where the scenarios were described such as “given whatever, when something happens, then I should get x”. But those were described as strings surrounded by C# code, so the business value wasn’t immediately apparent.
Now, however, I’ve been sticking to this format (right off the official Cucumber site):
This format makes it a lot easier to see the business value out of an atomic feature. It shows a short title for the feature, the ultimate value out of it (“in order to…”), who’s getting benefit from it (“as a…”), and what the person wants (“I want to…”). Then, it describes the different scenarios that exist for the given feature. I like the fact that there’s no specific programming language clutter mixed in with the description of the feature.
If I’m the one writing those stories, as a developer, it forces me to be as clear and succinct with the English language as possible, so that it will make it easier for me to validate these with my clients and users; and as they get familiar with the format, they can start providing me the stories themselves.
Another advantage I’ve been noticing doing these is that by analyzing the number of steps within a given scenario I can identify whether the user experience is going to be good or not. The scenario unveils how hard it is for the user to access the feature (“user has to click this, select that, fill out the other, click this other thing, and only then get to what she needs…”), and how the user interacts with the system (what the user does, and how the application responds).
Breaking scenarios into small, concise units… doesn’t that sound similar to “breaking classes into smaller, concise classes, methods, etc.”? Putting some good thought into writing these stories can help out the developer later when actually implementing the feature.
And here’s another benefit of keeping scenarios small and concise: planning. When discussing the feature with the clients, I can get them a feel for the complexity on each scenario, and they can tell me which scenarios are must-to-have, and which ones could come at a later time. This way we can plan on which features and which specific scenarios are going to be implemented on a given iteration or release.
I’m also following the idea that “Cucumber tests cover the application’s behavior, and RSpec tests cover class’ behavior”. In other words, I write Cucumber tests to check a feature end-to-end, from the user’s point of view, and I use RSpec to test a class’ behavior.
Some people say one of the main advantages of Cucumber is that non-technical people can easily read, or even write, the tests, but some developers say “the non-technical people will never read anything anyways, so why bother?”. So far, I’m seeing the Cucumber tests as being very beneficial for myself, for the reasons listed above (like forcing me to think of the features in a cleaner and more concise way, etc.).
At the moment, I have the following workflow:
- Write the features and store it in Evernote (tagging everything so I can group features based on specific user roles and things like that);
- Review stories with the client, rewrite pieces, split features up, move scenarios around, etc;
- Once it feels like a given feature is a little more stable, I put it on the Kanban board in AgileZen;
- Pick features that are on the board, and plan them accordingly;
- Once I start working on a given feature, it’s just a matter of turning them into a Cucumber file, and go about implementing it.
This is not written in stone, and it’s just something I’m trying out. I have no idea whether that’d scale for large teams or whatever, but I’m being pragmatic and doing whatever it takes to be productive given what I have. It seems to be working. I’ll keep you posted.
I keep hearing how developers say it’s kind of hard to get their minds around this testing thing, so I was then thinking to myself: “how did I get into this, by the way?”
It was at DevTeach 2003 when I first heard about unit testing. Scott Bellware was presenting a session about it. I decided to sit in and see what that was all about. Scott showed NUnit, and how to write some unit tests (sorry, that’s as much I can remember so many years later…).
Right after seeing that session, I decided to start writing some tests for some new code I was writing. At the time, I was writing the test after I wrote the production code, still trying to figure that NUnit thing. I was pretty pleased by having automated tests for some of my code.
Late in 2005, the same Scott came to Houston to present a full-day TDD workshop. Again, that was kind of long ago, and my memory is fuzzy. I remember Scott touched on dependencies, mock objects, how Test-Driven Development (TDD) helps keeping track whether the implementation is going on or off track, etc. I guess that was when I’ve heard of using TDD more as a design methodology than as a testing methodology, but I’m sure I didn’t quite *get* the idea back then.
Around that same time, I was working on a project here where we’ve decided we wanted developers to write more tests. We were using NUnit, and writing “integration” tests, not “unit” tests (as I came to realize later). We were writing tests to make sure the data put into our business entities were being saved and retrieved correctly to and from the database.
Right before starting off a new project (early in 2006), I looked back and noticed that a lot of redundant code was being written for those tests (instantiate entity, stick test values to its properties, save it, reload it, check to make sure the values saved are the ones expected, delete it, and have assertions all around to make sure things were working).
Decided to make that easier, I wrote a testing framework that’d do all that boring and repetitive work for the developers. For each business entity, the developer would create a test fixture class, make it inherit from a baseclass from my framework, override a method to provide test data to be pushed onto the entity, and give the baseclass the type of entity under test. The framework would then take care of doing all the heavy lifting and report any problems to the developer by means of assertions with as much detailed information possible (which property has failed, in which process, why, what could be causing it, etc.).
We had also noticed how much of a pain it was to create some specific test data that a given entity needed in order for it to be properly tested. I’ve then added some helper methods to my test framework that allowed the developer to pretty much ask for “5 entities of type Customer for tests”, or something like that, and the framework would take care of creating those, giving it back to the developer (who could also tweak with the test entities, of course), and then cleaning it up afterwards.
The framework worked well, and the great thing about it was that it was pretty easy to get some good code coverage with those tests. However, at the time I *thought* we were doing unit tests, but we were actually doing integration tests. It finally downed on me that we should also be able to test things in isolation. That is, if what we need to test is a business rule, we shouldn’t need a live connection to the database, or even a data access layer for that matter. “A-ha, so that’s what those mock objects are all about, then”, I thought to myself.
With that in mind, I decided to improve our testing framework so to make it easier for our developers to write unit tests for our business rules, so that they didn’t have to worry about the dependencies we have in our middle-tier framework in order to do so. After I’ve done some research on mock frameworks, I’ve decided to go with TypeMock, which I wrap up in my own special framework for testing our middle-tier components.
I’m currently working on rewriting some pieces of our testing framework in order to make use of some of the new features in C# 3.0 (such as object and collection initializers, lambda expressions, extension methods, etc.). This allowed me to clean up my API’s a lot (or so I think).
Everything I’ve mentioned so far has definitely improved my skills with doing TDD as a “testing” methodology, trying to make sure our code has some tests around it. That is something that most developers have a better time understanding and trying to do it, so that’s certainly valid.
Doing TDD as a “design” methodology is something eventually I got the idea and decided to start practicing it. This is definitely much harder for developers to get used to, because it really requires a different approach to writing software; different from what they’ve been doing for so many years now. However, like it’s been said over and over again by other people, when you really stick with TDD, then you start feeling bad about any code you write that you didn’t design beforehand, test-first.
By no means am I saying that I have mastered TDD (far from that), but I’m just certain that the code I wrote test-first is of much higher quality. Not only I believe my design is more solid (both because of the thought I’ve put through the design, but also because of the side-effect of getting the implementation under test), but I find it’s much easier to explain something to another developer by showing her the tests first, and then digging into the implementation.
I have been reading a lot of books, articles, websites, blog posts, other people’s code, etc., trying to improve my approach to TDD (which always entertains me by getting blown away by some code other people write). This has been great, and I plan to continue doing so.
I’ll probably have some more posts coming this way on the subject, as I get my mind around this thing. 🙂