Checking expected exception message when using MS Test

Thursday, August 20, 2009

In Typemock we use both NUnit and MSTest to run our unit tests. This practice enables us to check that Isolator works fine on both unit testing framework. Most of NUnit attributes can be translated fully into MSTest attributes (and vise-versa) there is one attribute we tend to use that works differently in MSTest - ExpectedException.
ExpectedException attribute is used to specify that a test should throw an exception to pass, both framework let the user define the type of the exception thrown (i.e. typeof(ApplicationException)) the big difference between the framework lies in the 2nd (optional)  parameter of that attribute - the Message.
While NUnit compares the massage to the exception's message MSTest does not. At first I though it was a bug but further investigation revealed that this was done intentionally, Microsoft decision was that the message should be only descriptive and used to associate a message to the exception and not check it.
So what could a TDD developer do if he wants to write a test that checks the exception message as well?
One solution is to add code to the test to check the exception's message:
[TestMethod]
[ExpectedException(typeof(ApplicationException))]
public void AddNewUser_UserEmailIsEmpty_ThrowException()
{
    try
    {
        // This line should throw an exception
        UserConnector.AddNewUser("user1", "");
    }
    catch (ApplicationException exc)
    {
        Assert.AreEqual("Error: user's email address missing", exc.Message);
        throw;
    }
}
You can read more about this method at IMistaken Blog.
The problem with this solution is that whenever I need to check an exception message I need to write ~4 more lines of code that makes my test less readable and somewhat error prone.

After digging a bit on the net I found a better solution, it seems that both XUnit and MBUnit uses Assert.Throws method instead of an attribute to check for expected exceptions.
First I've created a new class MyAssert that handles the exception verification logic:
public class MyAssert
{
    public static void Throws<T>(Action action, string expectedMessage) where T : Exception
    {
        try
        {
            action.Invoke();
        }
        catch (T exc)
        {
            Assert.AreEqual(expectedMessage, exc.Message);

            return;
        }

        Assert.Fail("Exception of type {0} should be thrown.", typeof(T));
    }
}
Then all I needed to do is use the new class whenever I needed to test for an exception and verify its message:
[TestMethod]
public void AddNewUser_UserEmailIsEmpty_ThrowException()
{
    MyAssert.Throws<ApplicationException>(
        () => UserConnector.AddNewUser("user1", ""), "Error: user's email address missing");
}
Although this solution seems more complicated at first it does have two advantages:
  1. I don't need to write the verification logic anymore
  2. I can specify the exact line in which I expect the exception to be thrown from.
Some would claim that using Throws is an over specification of the test but I don't think so. If using Assert calls during unit tests to check that specific line of test returned specific value is fine so does testing that a specific line throw a specific exception is acceptable as well.
After using MyAssert.Throws for a while I have to say I prefer it to using ExpectedException attribute because of the specific meaning it bring to my unit tests.

1 comment

  1. I really like the possibility to specify the exact line the exception should be thrown from. I've been in a situation where a test failed by throwing an exception at one of the first lines of the test method. The expected outcome was an exception in the last line. Since the early, unexpected, exception was of the same type as the expected exception everything looked fine when running the tests.

    With your Assert.Throws method that error would have been caught.

    ReplyDelete

Newer Older
Related Posts Plugin for WordPress, Blogger...