How to run your unit tests as part of TFS build

Writing unit tests is good, having a build server that run the unit test on each commit/check-in is great!

In the past I’ve used TeamCity and FinalBuilder to administer my builds and run my tests, it was easy and painless and it worked. Unfortunately we cannot always decide our organization build strategy and if you’re writing .NET code you might need to use Team Foundation Server (TFS).

Faced with this new challenges you have two options either complain and try to convince everyone that it would be better to change how things work or you can learn how to harness the new tool to do your biddings – I choose the latter.

Running MSTest unit tests in TFS

Running Microsoft’s testing framework tests is easier of the two – because it’s fully supported by TFS (2008). In order to run the tests you need to open the TFS project file and change the RunTest element

<RunTest>true</RunTest>

Congratulations you’ve just told TFS to run your unit tests - now let’s tell it what tests to actually run. Find the related ItemGroup you’ll recognize it by the remark:

<ItemGroup>
   <!--  TEST ARGUMENTS
  If the RunTest property is set to true then the following test arguments will be used to run 
  tests. Tests can be run by specifying one or more test lists and/or one or more test containers.

Defining what tests to run can be done in three different ways by adding the following under the ItemGroup :

1. Running test assembly/assemblies:

<TestContainer Include="$(OutDir)\MyProject1.Tests.dll" />
<TestContainer Include="$(OutDir)\MyProject2.Tests.dll" />
<TestContainer Include="$(OutDir)\MyProject3.Tests.dll" />

2. Running all assemblies in path:

<SearchPathRoot>$(OutDir)\Tests\</SearchPathRoot>

3. Running according to the vsmdi file

<MetaDataFile Include="$(SolutionRoot)\MyProject\MyProject.vsmdi">
     <TestList>CI_Tests</TestList>
     <TestRunConfig>$(SolutionRoot)\<MyProject>\MyProject.testrunconfig</TestRunConfig>
</MetaDataFile>

Note that the TestList parameter is optional – if you do not choose test list to run it will run all of the tests in your project.

That’s it, your unit tests should run as part of the build and their results will be reported to-whom-it-might-concern.

Running NUnit unit tests in TFS

Running NUnit tests is a bit trickier, the problem is not to run the test (which can be done by using Exec) but how to publish the results to TFS.

Luckily for us this problem was already solved by the NUnit for Team Build project.

What you need to do is run the tests and then use NUnit for Team Build to convert the result xml file into MSTest result (trx) and publish the result to TFS server.

Another handy component you want ot have (although not a must) is MSBuild.Community.Tasks with it’s NUnit task that believe it or not runs NUnit tests.

If you cannot install both tools on your server or you prefer a self contained build process you can copy both locally and add the following at the beginning of the .proj file:

<PropertyGroup>
      <MSBuildCommunityTasksPath>$(SolutionRoot)\<MyBuildTools>\MSBuildCommunityTasks\</MSBuildCommunityTasksPath>
</PropertyGroup>
<Import Project="$(MSBuildCommunityTasksPath)\MSBuild.Community.Tasks.targets" />

Instead of \MyBuildTools just put the path to where you’ve installed the community tasks

The following was more or less copied from the example that came with NUnit for Team Build:

<Target Name="AfterCompile">
    <PropertyGroup>
        <NUnitBinaries>$(SolutionRoot)\<MyProjectTools>\NUnit\</NUnitBinaries>
        <NUnitTFSDirectory>$(SolutionRoot)\<MyBuildTools>\NUnitTfs</NUnitTFSDirectory>
    </PropertyGroup>
    <Message Text="Running NUnit test with custom task" />

    <!-- Create a Custom Build Step -->
    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Name="NUnitTestStep" Message="Running Nunit Tests">
        <Output TaskParameter="Id" PropertyName="NUnitStepId" />
    </BuildStep>

    <ItemGroup>
        <TestAssemblies Include="$(OutDir)\MyProject.Tests.NUnit.dll" />
    </ItemGroup>

    <!-- Run NUnit and check the result -->
      
    <NUnit ContinueOnError="true" 
             Assemblies="@(TestAssemblies)" 
             OutputXmlFile="$(OutDir)nunit_results.xml" 
             ToolPath="$(NUnitBinaries)">
        <Output TaskParameter="ExitCode" PropertyName="NUnitResult" />
    </NUnit>

    <BuildStep Condition="'$(NUnitResult)'=='0'" TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Id="$(NUnitStepId)" Status="Succeeded" />
    <BuildStep Condition="'$(NUnitResult)'!='0'" TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Id="$(NUnitStepId)" Status="Failed" />

    <!-- Regardless of NUnit success/failure merge results into the build -->
    <Exec Command="&quot;$(NUnitTFSDirectory)\NUnitTFS.exe&quot; –n &quot;$(OutDir)nunit_results.xml&quot; -t &quot;$(TeamProject)&quot;
                   -b &quot;$(BuildNumber)&quot; -f &quot;%(ConfigurationToBuild.FlavorToBuild)&quot; -p &quot;%(ConfigurationToBuild.PlatformToBuild)&quot;
                   -x &quot;$(NUnitTFSDirectory)\NUnitToMSTest.xslt&quot;" />

    <!-- If NUnit failed it's time to error out -->
    <Error Condition="'$(NUnitResult)'!='0'" Text="Unit Tests Failed" />
</Target>

And that’s it… hopefully.