Mocking frameworks are a path to insanity! They start out great. They're a great way to mimic methods. They initially save time. But they're a gateway drug that will ultimately break your encapsulation, break your builds, break your tests, and break your mental capacity to understand what's going on!

This post describes why I prefer using in-memory test doubles over mocking frameworks for all but the simplest of scenarios in unit tests. The examples are in C# and use NSubstitute as the mocking framework, and FluentAssertions for the asserts, but the idea applies to other languages and mocking frameworks.

For a bit of background: I used to love mocking frameworks! I marvelled at their use of generics. It was a personal challenge to see how far I could push the syntax of these frameworks, and I revelled in the satisfaction of controlling types that didn't actually exist. But gradually, the novelty wore off. The syntax became jarring. Other contributors had difficulty reading the tests. And changing signatures of methods in the code being tested would either cause compilation errors in the tests, or cause the tests to fail. I eventually concluded that creating small, simple test doubles was a better approach.

First, a quick recap of the differences and what they give us:

• Mocking frameworks, like Moq or NSubstitute in .NET, create proxy instances of classes or interfaces at runtime. They offer the ability to define the behavior of these proxies, such as specifying the return values of methods or properties, and can also record and verify interactions such as method calls.
• In-memory test doubles, are types that we write that have in-memory behavior of something that would otherwise have side effects, for example, writing to a database, file system, or any other external system or service. I'll refer to these as just test doubles going forward.

And next, just a quick recap of why we use either of them: when testing the behaviour of something (often called the System Under Test, or SUT), we often want to swap out certain dependencies that the SUT has. For example, dependencies that write to a database. We may not want to write to a real database in our unit tests, so we need to replace it with a test double or a fake.

What follows is why I think manually created test doubles are preferable to mocking frameworks.

Test doubles focus on behaviour, not implementation #

When you write a test double, you focus on mimicking the behaviour of the real production code.
When you use a mocking framework, you focus on which methods are called with which parameters.

Having such intimate knowledge breaks encapsulation and is a burden. It tightly couples your test setup code to actual method signatures in the production code. And when a method signature changes, you will likely need to update all the places in your test setup code that have set up the call to that method. But with a test double, you only need to update the test double itself, usually in just one place.

To demonstrate, consider this interface and type that uses it:

public interface IProductRepository
{
    void Store(Product product);
    Product Get(int id);
}

public class ProductService(IProductRepository _productRepository)
{
    public void OnboardNewProduct(int id, string name) =>
      _productRepository.Store(new Product(id, name));
}

Here's a test using NSubstitute:

[Fact]
public void Using_mocks()
{
    var repo = Substitute.For<IProductRepository>();
    var sut = new ProductService(mock);

    sut.OnboardNewProduct(123, "Product 123");

    repo.Received().Store(Arg.Is<Product>(p => p.Id == 123));
}

There are a couple of things wrong with this approach. Aside from needing to understand the mocking framework syntax to read and modify the test, the test has explicit knowledge of the implementation of the OnboardNewProduct method. It knows that somewhere in the implementation, it calls a method named Store and it knows the parameters that are provided to that method.

Your test doesn't need to know this, it only needs to know that the product is stored in the repository. To explain further, here's the test double and revised test:

public class InMemoryProductRepository : IProductRepository
{
    private readonly List<Product> _products = new();

    public void Store(Product product) => _products.Add(product);
    public Product Get(int id) =>  _products.FirstOrDefault(p => p.Id == id);

    // This is not part of the interface, but is useful for testing
    public bool DidStore(int id) => Get(id) is not null;
}

And here's the test that uses the test double:

[Fact]
public void Using_test_doubles()
{
    var repo = new InMemoryProductRepository();

    var sut = new ProductService(repo);

    sut.OnboardNewProduct(123, "Product 123");

    repo.DidStore(123).Should().BeTrue();
}

The test no longer needs to have the intimate knowledge of specific methods being called. The only thing it needs to know is that the observable behaviour is correct. It does this with the call to repo.DidStore(...).

The main point to note here: if the signature of the Store method changes, then it only needs to be updated in one place, in InMemoryProductRepository. And if you use a refactoring tool, it will likely change the signature in both places.

One of the biggest objections I hear when suggesting test doubles is that it is more code and more typing. But generally, you type this stuff once, and it is read thousands of times. So if you optimise for reading, you're saving yourself time in the long run. And the test double evolves with the real code; changes happen in a single place, rather than being spread out across all the test setup code (more on that in the next item).

Another scenario that I often use is to tell the test double to always throw an exception. Let's see how that looks using both approaches. Here's the test using a mocking framework:

[Fact]
public void Could_not_store_new_product_using_mocks()
{
    var repo = Substitute.For<IProductRepository>();

    repo.When(x => x.Store(Arg.Any<Product>()))                     👈
        .Do(x => throw new InvalidOperationException("oh no!"));    👈

    var sut = new ProductService(repo);

    Action a = () => sut.OnboardNewProduct(123, "Product 123");

    a.Should().ThrowExactly<InvalidOperationException>()
        .WithMessage("oh no!");
}

Again, the test setup needs to know explicit implementation details, namely that the Store method is called with a particular parameter.

Here's how to do the same thing with a test double. The test double from above needs modifying:

public class InMemoryProductRepository : IProductRepository
{
    private readonly List<Product> _products = new();
    private Exception? _alwaysThrowsWhenStoring;    👈

    public void Store(Product product)
    {
        if (_alwaysThrowsWhenStoring is not null)   👈
          throw _alwaysThrowsWhenStoring;           👈

        _products.Add(product);
    }

    public Product Get(int id) => _products.FirstOrDefault(p => p.Id == id);

    public bool DidStore(int id) => Get(id) is not null;

    public InMemoryProductRepository AlwaysThrowsWhenStoring(Exception e) 👈
    {                                                                     👈
        _alwaysThrowsWhenStoring = e;                                     👈
        return this;                                                      👈
    }
}

The test setup has a simple one-liner to say that the behaviour associated with storing (no method name is needed), now throws an exception:

[Fact]
public void Could_not_store_new_product_using_test_doubles()
{
    var repo = new InMemoryProductRepository()
        .AlwaysThrowsWhenStoring(new InvalidOperationException("oh no!")); 👈

    var sut = new ProductService(repo);

    Action a = () => sut.OnboardNewProduct(123, "Product 123");

    a.Should().ThrowExactly<InvalidOperationException>()
        .WithMessage("oh no!");
}

The differences between the two are:

// mocking
var repo = Substitute.For<IProductRepository>();
repo.When(x => x.Store(Arg.Any<Product>()))
    .Do(x => throw new InvalidOperationException("oh no!"));

// test double
var repo = new InMemoryProductRepository()
    .AlwaysThrowsWhenStoring(new InvalidOperationException("oh no!"));

I think you'll agree that the test double version is more readable, which brings us onto the next point...

Readability and maintainability #

Test doubles promote cleaner, more understandable code because the logic is in a separate class, not intertwined with test setup code.

In the example above, if a reader wanted to know how the repository is imitated, they'd just take a look at the test double to see what it does. That knowledge is in one place, as opposed to dozens or hundreds of mocked method calls spread throughout the tests. Also, it is clear where to add convenience methods such as DidStore.

DRY: these small examples are tiny, but in a real codebase, there is likely hundreds or even thousands of calls setting up particular methods. A lot of this is duplication (and 'wordy' duplication at that), which usually ends up obscuring the real intent of the test.

And speaking of the DidStore method: such methods make the tests easier to read as you're asking questions of the behaviour, not individual methods. Which of the following would you prefer to read?

// mocking framework
repo.Received().Store(Arg.Is<Product>(p => p.Id == 123));

// or, using a test double
repo.DidStore(123).Should().BeTrue();

Better simulation of real scenarios #

Test doubles can mimic real-life scenarios more effectively than mocking frameworks. They can preserve state across multiple method calls, which can be challenging to do with mocking frameworks. Mocking frameworks tend to be very specific and focused on individual calls, making them less suited to complex or state-dependent behavior.

Performance #

Test doubles can be more performant than mocking frameworks. With test doubles, there is no need for runtime generation of proxies or for methods to be intercepted and recorded. Performance may or may not be a problem for you, but I've seen it make a huge difference with thousands of tests. I've also seen this make a big difference when using automated test runners, such as the ReSharper/Rider continuous tests or NCrunch.

More expressiveness #

In your test doubles, you can add a fluent interface.

In software engineering, a fluent interface is an object-oriented API whose design relies extensively on method chaining. Its goal is to increase code legibility by creating a domain-specific language (DSL). The term was coined in 2005 by Eric Evans and Martin Fowler.

For instance, you may want to just set up a test double to always throw an exception, or always return a specific value. Here's an example showing how that looks in your test:

[Test]
public void TestGetProduct()
{
    var repo = new InMemoryProductRepository()
      .AlwaysReturns(new Product(123, "Test Product"));

    var sut = new MyService(repo);

    // any call to Get will return the product

    ...
}

AlwaysReturns is a fluent method that returns the type itself so that method calls can be chained. Here's what it looks like in the test double:

public InMemoryProductRepository AlwaysReturns(Product p)
{
    _alwaysReturns = p;
    return this;
}

Note that we've changed the behaviour of the test double to always return a specific product. We didn't change the test setup code to say that "when the Get method is called, with an ID parameter of any value, then return this particular product". We merely said "This repository always returns this product". This is an important consideration; we don't care how this is done (e.g. someone calls the Get method), we leave that to the in-memory abstraction (more on abstractions next).

More focused abstractions #

If your interfaces are small (and they should be), then writing test doubles should be trivial. But if your interfaces are big, then writing test doubles for them is not going to be trivial. The test doubles will end up with reams of code that will be difficult to follow and difficult to evolve along with the real production code. In this case, you are no better off than using a mocking framework.

If you find it difficult or onerous writing a simple imitation of an interface, then that is likely a sign that the interface is too large and needs to be broken down into smaller, more focused abstractions.

Earlier, I described that a common objection from fellow developers to using test doubles is "But it's more typing". This objection is compounded and affirmed when you have overly large interfaces. The correlation between a large interface and complexity isn't always obvious to them, but what is obvious to them is that mocking frameworks make their tests easier to write, and they therefore regard mocking frameworks as good.

🤔 If your imitation (test double) is difficult to write and understand, just imagine how difficult the real implementation will be to write and understand!

What have we seen? #

While mocking frameworks have their uses, in-memory test doubles provide a robust, clear, and efficient approach to testing. They isolate tests from specific implementation details, make the code more readable, provide resilience to refactoring, and more effectively simulate real-world scenarios.

Remember, the goal of tests is not just to ensure that the system works as expected but also make the codebase more maintainable. By using in-memory test doubles, you're taking a step towards writing clean, maintainable, and robust tests.

The overall theme here has been the benefits of focusing on behaviour and not implementation.

Please leave a comment with your thoughts

FAQ #

Here's a couple of things that I've heard asked:

How can I test that my SUT has called the right method with the correct parameters? #

This is testing implementation and not behaviour. Your SUT called something and there is likely an observable side-effect of that. Test the side-effect and not that a particular method was called. If the code is refactored (e.g. you change the implementation but not the behaviour), then your test that checked that a method was called will likely break, but your test that tested the behaviour should remain unchanged and should still pass.

How is DRY violated? Can't I just move duplicated setup code into a shared method in a 'TestHelper' class? #

You could, but the problem remains of needing explicit knowledge of what methods need to be called. The method that you just created in your TestHelper class to avoid duplication will likely take some parameters that relate to the method that needs to be set-up. So, instead of changing hundreds of lines of code that used to set-up a method, you're now changing hundreds of lines of code that call your TestHelper method.

By focusing on behaviour and driving that behaviour with descriptively named methods in the test double itself (e.g. AlwaysThrows), then you both remove duplication and the reduce the chances that you'll need to change parameters.

🙏🙏🙏

Since you've made it this far, sharing this article on your favorite social media network would be highly appreciated 💖! For feedback, please ping me on Twitter.

Leave a comment

Comments are moderated, so there may be a short delays before you see it.

Published 17 Feb 2024