Test Driven Development (TDD) have many benefits. For start it’s a design methodology that help avoiding “Analysis paralysis” and make sure that you only have the needed code to solve a problem.
Yesterday I found another benefit of writing the tests before the code – you get to see them fail!
A while back I wrote about another shortcoming of MSTest – how it does not have any built in facility to test against expected exception message.
Unknown to me at the time there was a bug hidden in the code I’ve published – see if you can spot it:
public static class MyAssert
{
public static void BadThrows<T>(Action action) where T : Exception
{
try
{
action();
Assert.Fail("Exception of type {0} should be thrown.", typeof(T));
}
catch (T exception)
{
// All is well!
}
}
}
BTW this example was simplified for your viewing pleasure - I want to make a point here, not stick to the facts. have you found the bug yet?
Consider the following test:
[TestMethod]
public void ThisTestWouldNeverFail()
{
MyAssert.BadThrows<System.Exception>(() => Dummy.SomeMethod());
}
Have you guessed it? (hint: its in the test name)
This test would never fail. If we take a closer look at BadThrows we’ll see the problem.
The problem is that the unit testing frameworks throws an exception when assertions fails. And since we’re expecting an exception of type Exception it means that all exceptions thrown (including the AssertFailedException) would be caught – causing the test to pass even if it actually "failed".
That’s right – the test is completely useless – if you want to see the working code just head to this post.
I did not find this bug mainly because I usually don’t throw System.Exception in my code.
The guy that did find this bug found it because he wrote a test and wanted to see it fail. It wasn't enough for him to know that his code made the test pass – he had to make sure that it failed first.
A quick search in our code found multiple tests that did exactly that which means two things:
- These test were not written using TDD
- The authors did not even bother to see if their tests actually work by making the tests fail
After fixing the bug, one test that started failing – which means that not only was the test wrong – there was a bug in the code as well.
After fixing the bug the whole team has a talk about why throwing “Exception” is not a good idea and how tests need to be “tested” by seeing them fail. Following a quick bug & exception hunt in which we've fixed the tests and the code.
I think I learnt quite a lot from this experience, I know now who does TDD and who doesn't and now I know another issue to look for when developers write their tests after the code.
Happy coding…Labels: TDD, Thoughts, Unit tests