A while ago, I attended a user group meeting on the subject of unit testing using VSTS in the real world. It was an interesting talk and opened me up to some of the novel techniques that can be used in the realm of unit testing. One of these techniques was data driven unit tests and in this post, I'll walk you through creating such a test.
Introduction to data driven unit tests
A data driven unit test is very similar to a normal unit test, except that it is run a number of times, each time with a different set of inputs. The inputs to the unit test usually take the form of some kind of data source (such as a CSV file or database). For each row in the data source, the test host system will invoke the unit test.
Data driven tests are useful when you need to test with a range of input values (e.g. boundary conditions or a specific set of values that are known to cause trouble). Instead of writing a number of unit tests where each one tests a single value, you write the test once and provide a list of different inputs to test.
At a high level, in Visual Studio 2005 Team System, you can create unit tests by taking the following steps:
- Create the data source for the data driven unit test. This takes the form of a table of values where the columns represent inputs to the unit test and the rows represent separate unit test runs.
- Add the [DataSource()] attribute to the unit test, specifying the location of the data.
- Write your unit test. Instead of hardcoding input values (such as the proverbial expected value), read them from the current test data row instance using TestContext.DataRow["ColumnName"] where "ColumnName" is the name of the column you wish to get the current value for.
- Run your test. You should notice that the total test count reflects the number of rows in the data source, not the number of tests you have written.
How to write a simple data driven unit test
This tutorial will show you how to write a simple data driven unit test to test a simple mathematical addition function.
Step 1: Create the Visual Studio solution
We need to have something to test. For these purposes, let's start with a simple class library called "Math".
Once you have created the blank class library, create a class called "Adder" and add the following method body.
/// <summary>
/// Adds two integer values together
/// </summary>
/// <param name="x">The first value</param>
/// <param name="y">The value to add to the first value</param>
/// <returns>The result of x+y</returns>
public int Add(int x, int y)
{
return 0;
}
Notice that we have left the method body empty (except for the return statement to allow it to compile). This will hopefully cause our tests to fail to allow us to demonstrate that our test is indeed working.
Step 2: Create the unit test project
Right click on the "Add" method signature and select "Create Unit Tests..." you should be presented with the dialog shown that will allow you to automatically create a unit test stub.
When you click "OK", you should be presented with a unit test stub similar to the following.
/// <summary>
///A test for Add (int, int)
///</summary>
[TestMethod()]
public void AddTest()
{
Adder target = new Adder();
int x = 0; // TODO: Initialize to an appropriate value
int y = 0; // TODO: Initialize to an appropriate value
int expected = 0;
int actual;
actual = target.Add(x, y);
Assert.AreEqual(expected, actual, "Math.Adder.Add did not return the expected value.");
Assert.Inconclusive("Verify the correctness of this test method.");
}
Leave the code as is for now... we'll modify it later.
Step 3: Create your data source
For the purposes of this test, we'll define a data source to hold a set of values to test with. The container for the data will be a comma separated values text file (.csv) and it will contain 4 columns and a number of rows as shown (the first row contains the column names).
X,Y,Expected,Comment
1,1,2
2,3,5
-1,2,1
1,-1,0
To add this data to the unit test project, select the project in the solution explorer, right click and select "Add -> New Item...". Select "Text File" from the list of items, enter a filename (e.g. "AddTestData.csv") and click "OK". Enter the data into the CSV file and save it.
You'll notice that we've left the last column as a free text comment column. Although not used directly in this demo, it can come in useful when trying to identify rows that cause the test to fail as we can append the comment to any messages that we print out to the unit test host system.
Step 4. Configure the data source
In order to use the CSV file as a data source you need to give Visual Studio some information about it (e.g. where to find it and the data source provider to use).
First of all, we need to ensure that the file gets deployed with our tests. To do this, double-click on the "localtestrun.testrunconfig" file that was automatically created by Visual Studio. You should be presented with a dialog like the one illustrated below.
Select the "Deployment" configuration section and click "Add File...". This will allow you to select files that will get distributed along with the unit tests.
In our case, we want to distribute the file that contains the data for our data-driven test ("AddTestData.csv"), so add it to the list.
Each time our tests are run, this file will get copied into the same folder as the binaries for the unit test.
In order to tell Visual Studio how to read the data contained in our file, we need to add a configuration file and specify the data provider implementation. It's possible to do this with attributes on our unit test itself, but doing it in an application config file is cleaner and allows you to easily share data sources between different unit tests.
Once you have added a blank application config file to your unit test project (Add -> New Item -> Application Configuration File) you can add the following XML (see the comments for an explanation of the different sections.
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="microsoft.visualstudio.testtools"
type="Microsoft.VisualStudio.TestTools.UnitTesting.TestConfigurationSection,
Microsoft.VisualStudio.QualityTools.UnitTestFramework,
Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"/>
</configSections>
<!-- Defines connection strings for the data-->
<connectionStrings>
<!-- The connection string to get CSV data from the current test folder.
Each CSV file will represent a different table in the data source-->
<add name="CSVData"
connectionString="Driver={Microsoft Text Driver (*.txt; *.csv)};Dbq=;Extensions=csv"
providerName="System.Data.Odbc" />
</connectionStrings>
<!-- Data sources for data driven tests. Note that "#" instead of "." is used for the file extension-->
<microsoft.visualstudio.testtools>
<dataSources>
<!-- The data table we'll use for running the Add() metod tests-->
<add name="AddTestDataSource"
connectionString="CSVData"
dataTableName="AddTestData#csv"
dataAccessMethod="Sequential"/>
</dataSources>
</microsoft.visualstudio.testtools>
</configuration>
Step 5. Implement the data driven unit test
Once you have saved the application file, go back and edit the code for your unit test. It should look like the following. Notice how we have specified a [DataSource()] attribute and we get the input and expected values now come from the DataRow property on the TestContext object.
/// <summary>
///A test for Add (int, int)
///</summary>
[TestMethod(), DataSource("AddTestDataSource")]
public void AddTest()
{
Adder target = new Adder();
int x = (int)TestContext.DataRow["X"];
int y = (int)TestContext.DataRow["Y"];
int expected = (int)TestContext.DataRow["Expected"];
int actual;
actual = target.Add(x, y);
Assert.AreEqual(expected, actual, "Math.Adder.Add did not return the expected value.");
}
Step 6: Run the unit tests and watch them fail
If you build the solution and run the unit tests, they should fail as we have not provided a valid implementation for the Add() method. You should notice that the test count reflects the number of rows in your data source - showing that your test was invoked once for each row. (The one successful result shown below was because one row in my input data contained an expected result of 0 - exactly what we return in our method stub)
If you double-click on the test, you will be presented with a listing of each test invocation, detailing which rows passed and failed:
Step 7. Implement the method and re-run the tests.
I'll leave this step up to you... I'm sure you can figure it out. :-)