Charlotte Mays
Unit testing is something that deeply divides programmer communities. Nearly everyone agrees that it’s good to have unit tests in place, but some developers question whether the time invested in writing unit tests would be better spent writing “real” code, doing manual QA, or debugging.
In practice, it's a good use of time and should be standard in any company which takes pride in its end product.
Real-world examples
On one project, we self-enforced a requirement that at least 90% of our code is covered by unit tests at any given time. We automated this so that, if our code drops below that level, it won’t be merged into the main codebase until enough tests have been written to bring it back up. This ensures that tests are written as code is written, avoiding the monstrous task of writing tests for an already-massive codebase which has no tests yet.
There have been times when we have been on the verge of not finishing a task within the time we had planned and tests haven’t been written for that code yet. It’s extremely tempting in that situation to skip test-writing. In a company that values deadlines over quality, such tests would likely be skipped, but we’ve made a different choice at Caktus. I think it’s the right one.
At least a couple times a month I find myself writing tests for the code I’ve just written and realizing that I had omitted a check for an edge case. These usually take little time to fix. Writing tests can also help me think about how the code should be structured, particularly encouraging me to make it more modular. Not only does that increase readability, but it also can make it easier to update later as requirements change.
When I think about tests, I automatically go straight to the edge cases. A manual QA process may or may not catch problems with rare or unusual inputs and it can take a lot of time to manually test numerous edge cases. But having written an automated test, I ensure that the edge case continues to be handled according to the client’s specifications.
These same unit tests made a large refactoring process much easier. Going into the process, I knew that the change I was making would require compensatory changes in dozens of other places in the code, and I’m sure I would have eventually located all of the places it needed to change anyway. But, since we already had thorough test coverage, I was able to make the initial change, run the test suite, and use the test failures to know where I needed to make changes in the existing code. I also knew when I was done because all the tests were passing again. One final scan through the code confirmed that I hadn’t missed anything, and subsequent real-world tests have confirmed that everything seems to be working fine. Because of the attention to tests throughout the process, the client could be assured of a consistently high-quality product with very few bugs in less time than it would take without the tests.
Establishing a culture of testing
The first step in establishing testing as a standard part of the coding process is simply to measure it. Plenty of tools are available to measure your testing and get reports on what’s being covered and what isn’t. The best starting point is to use coverage, which will tell you how much of your code is being executed by your existing tests. As Caktus chose to do in the above example, a minimum coverage level can be set which must always be maintained, which works great when implemented at the start of a project and adhered to consistently.
If trying to add testing to existing code, the same principle can be applied with some minor tweaks. Unless you have the luxury of putting a hold on new code while tests are written (unlikely!), you will probably need to gradually add tests. The most reasonable way to do this is to either set goals for coverage or to impose a requirement that the coverage must always go up (until it reaches a reasonably high level).
Regardless of the application, if unit tests are consistently expected, the team will get faster and better at implementing them.
Code maintenance
It’s often asserted that a test suite is simply more code to maintain. While technically true, tests, once written, should only need to change if the requirements also change. This means that the tests should not need to be tweaked constantly. When they do need to be tweaked, that also helps streamline the process of finding the code that needs to change. Most of the time, the tests will sit untouched and do their job, asserting that all of the code is working as expected, with no maintenance required. When a test needs to be changed, it is again doing its job, pointing to code that is involved in changing requirements. No test should be changed just because it fails. A failing test tells you that either the requirements (and therefore code) changed, or that the test was not written correctly in the first place.
False sense of security?
One drawback of unit tests is that they can make you feel like everything is working great, and reduce motivation to do real-world testing. While unit tests make a great first pass over the code, there is no substitute for genuine QA. The tests should make the QA process go faster, as some of the more obvious bugs will be found before any manual testing happens, but QA will always still be needed. Even if a codebase has 100% coverage, there’s no guarantee that something hasn’t been missed. A bug in a test can easily disguise a bug in the code.
Reflections on testing
It took a not-insignificant amount of time for me to get the hang of writing unit tests when I was new to the concept, but my learning time has been more than made up for by the time those same tests have saved me. Testing is now second-nature to me, and I can write unit tests in no time when I am testing code I’ve just written. It only takes a few extra minutes and it so often catches errors or assists in later coding that I can’t imagine not taking the time to write tests from the beginning.
Certainly, tests need to be fairly comprehensive in order to gain all these benefits, but even a small test suite can be helpful and test coverage can be increased bit by bit if tests are written with every new pull request. We have made concerted efforts to establish test coverage on existing, untested code before, and that’s great if you have the time. If not, though, just remember that some is better than none, and increasing is better than stagnating.
Next steps
If you want to work on increasing emphasis on tests in your own projects, here are some strategies to think about:
- Practice writing tests for every bug fix or new feature (better yet, before starting on them!)
- Get in the habit of running test suites frequently
- Implement a policy that every pull request should include a test for the feature or bug being worked on
- Implement a policy that code coverage should not go down on any pull request
- Run mutation testing to find places where coverage is fine, but results of the executed code are not actually being tested
Ready to get started? Read more about testing and code quality.
