Results-Based Testing

A while ago I gave a talk on unit testing at a Philly ALT.NET meeting.  On the way home from that meeting, Brian Donahue and I had a discussion that went something like this (somewhat paraphrased (ok, a lot parapharased)):

Brian: Dude, you have to stop testing expectations and start testing the results!
Me: Damn, you’re right!  I used to do that but kinda fell away from it.  Guess it’s time to go back.

What do I mean by results-based testing?

If you’re familiar with most any testing framework (like NUnit) and Rhino.Mocks, you’ll be familiar with two ways of testing objects.  One way is asserting something was called (ex: foo.AssertWasCalled(x => x.MyMethod()), another way is just comparing objects and ensuring the value returned was expected (ex: Assert.AreEqual(foo, bar)).

Results-based testing means testing more of the latter, and less of the former.  As I’ve gone through the learning curve of TDD, I’ve noticed that focusing on using the AssertWasCalled style is often less than useful.  Let’s face it, usually you’re aware you called that method, what you want to know is what was returned from it.  So results-based testing focuses more on comparing results to ensure they’re what you expect.

An Example

Let’s say we have a simple class with one method on it, Execute().  Let’s also say that it has one dependency, a repository whose job it is to get all users.  The Execute method may look something like this:

   1: public IList<User> Execute()

   2: {

   3:     return userRepository.GetAll();

   4: }

Now, if you’re just testing expectations, you might write a test that says something like this:

   1: userRepository.AssertWasCalled(x => x.GetAll())

As mentioned above, this doesn’t feel very useful.  My class is only doing one thing, so I’m pretty confident that the repository was called.  What I want to test is that this method is actually returning all users.

To do this, first you have to set up your expectations, then you can do a simple Assert.AreEqual to ensure that the results are equivalent.  For the first part, Rhino.Mocks lets you stub out expectations on a class, so we’ll take advantage of that to set up the results that we expect to be returned.  Then we’ll get the actual result of the method call, and finally we’ll assert that the result we set up is equal to the one that was actually returned:

   1: var myListOfusers = new List<User> { new User() };

   2: userRepository.Stub(x => x.GetAll()).Return(myListOfUsers);

   3: _result = myClass.Execute();

   4: Assert.AreEqual(myListOfusers, _result);

That’s all there really is to it.  Now your tests are really going to start to become valuable, because as soon as the logic in your class changes, and affects the users that are returned, your test will break in a meaningful way.

Expectation Tests Are Not Useless

Far from it, actually.  They have their place.  Typically I use them in scenarios where different conditions mean that different methods will be called, for example, an if/else statement where the method called on an object changes depending on what branch of the statement you are in:

   1: if (criteria != null)

   2: {

   3:     userRepository.GetByCriteria(criteria);

   4: }

   5: else

   6: {

   7:     userRepository.GetAll();

   8: }

In this case, if you don’t have access to the results returned by these calls, it might be worthwhile to write 2 tests, one for each condition, and each test will simply assert that the appropriate method was called.

In either case, find the test type that’s right for you – the one that really tests the business logic, because that’s where you get your real value.

Advertisement