Categories
computer

If A Unit Test Fails In The Woods, Does It Make A Sound?

No, it doesn’t. Unit tests that execute a large amount of code but fail to make assertions along the way give you a false sense of confidence in the code. They pass when they should fail. These problems, formally known as type 2 errors, are a huge liability for a development team because the tests are believed to be verifying the intended behavior of the software, but are really doing nothing in a really lengthy way.

For a new person maintaining the code under test, the problems worsen. The new maintainer will not understand what the code is supposed to be doing: what it’s currently written to do or what is implied by the possibly out-dated and incomplete unit tests. Good luck finding API documentation if the unit tests suck, and have fun with those future API changes when you must attend to the “unit tests” that need to be updated to successfully add no value to the project, just as they were originally written. No thanks.

Units test exist to prove that software is behaving as intended, not simply “mock” user actions. This means being particular about states of things during a process, and doing mean negative testing by passing nil into that function that clearly requires a non-nil value. The rule of thumb is this: if, for whatever reason, you cannot write, fix, or otherwise finish work for a correct and complete unit test, assert false. You have not proven the software works correctly, so it doesn’t work. Period.