Tag: Mocks

To mock or not to mock

That is the question… debated in a recent civilized exchange of comments on this blog.

Like Ringo Star, I’m not afraid to position myself as pro-mock.

Of course mocking is not the goal, just a means of quickly writing tests which are focused on a small piece of logic. The goal is maintainable, thoroughly-tested code which functions as expected.

I’m not religious about mocking. If the method you’re testing has a collaborating class which you can just instantiate without any hassles and it works perfectly in the test from the start, then go for it. No need to mock. But that’s not something I see very often, apart from data structures or purely mathematical functions with no side effects. Most collaborating classes will use the network or disk or exhibit some non-repeatable behavior. At a minimum, there may be behavior that’s dependent on its state and which could evolve as the product evolves. You will likely need an integration test to show that your classes work together, but when you’re unit testing, you should be focused on the method you’re testing and not its collaborators.

For me (and many others), an interface is like a contract. If an interface is injected into the class I’m testing, I assume the object that implements it will do what’s expected, and I don’t care how it’s done. I would rather mock (or stub) the interface than write a test which gets involved with the details of its implementation. The contract doesn’t say the implementation won’t change, it just promises a certain behavior (a result and/ or side effects). If requirements evolve and the contract changes (different behavior, different return type, and/or different arguments), there will be some pain whether or not you use mocks, but it’s often more painful with mocks because many tests have to change (though the changes tend to be simple to implement). On the flip side, if, with no contract change, a bug slips into the implementation of the collaborating class, none of the tests which mock it will fail. If the creator of the collaborating implemented good unit tests, the bug will be spotted instantly in that case. On the other hand, without mocks, all the tests using the real collaborating class could fail at once, in which case it can take a little digging to find the bug.

There’s also the perhaps more frequent case of implementation details changing without a change to the interface. In that case, if the tests use mocks, there’s no issue at all. If, on the other hand, the tests call a constructor of the collaborating class which changes to add a new dependency, or if suddenly the collaborating class requires database access, all the tests without mocks are broken, often with no easy fix other than mocking.

For me, mocks are part of a modular approach to development. You develop little components, and testing quickly and simply with mocks you are confident that each component works exactly as specified. A few integration tests prove that the components can work together. This is the same sort of philosophy used with physical products such as cars. If you change the tire of your car, you don’t lose sleep over the fact that the tire has never been tested on the wheel in question. You assume that the tire has been tested, and you know that the wheel worked with a different tire with the same standards, and that’s that.

Does TDD “damage” your design?

I recently came across a couple articles that challenged some of my beliefs about best practices.

In this article, Simon Brown makes the case for components tightly coupling a service with its data access implementation and for testing each component as a unit rather than testing the service with mocked-out data access. Brown also cites David Heinemeir Hansson, the creator of Rails, who has written a couple of incendiary articles discouraging isolated tests and even TDD in general. Heinemeir Hansson goes so far as to suggest that TDD results in “code that is warped out of shape solely to accomodate testing objectives.Ouch.

These are thought-provoking articles written by smart, accomplished engineers, but I disagree with them.

For those unfamiliar with the (volatile and sometimes confusing and controversial) terminology, isolated tests are tests which mock out dependencies of the unit under test. This is done both for performance reasons (which Heinemeir Hansson calls into question) and for focus on the unit (if a service calls the database and the test fails, is the problem in the service or the SQL or the database tables or the network connection?). There’s also a question of the difficulty of setting up and maintaining tests with database dependencies. There are tools for that, but there’s a learning curve and some set-up required (which hopefully can be Dockerized to make life easier). And there’s one more very important reason which I’ll get to later…

Both Brown and Heinemeir Hansson argue against adding what they consider unnecessary layers of indirection. If your design is test-driven, the need for unit tests will nudge you to de-couple things that Brown and Heinemeir Hansson think should remain coupled. The real dilemma is where should we put the inevitable complexity in any design? As an extreme example, to avoid all sorts of “unnecessary” code you could just put all your business logic into stored procedures in the database.

“Gang of Four” member Ralph Johnson described a paradox:

There is no theoretical reason that anything is hard to change about software. If you pick any one aspect of software then you can make it easy to change, but we don’t know how to make everything easy to change. Making something easy to change makes the overall system a little more complex, and making everything easy to change makes the entire system very complex. Complexity is what makes software hard to change. That, and duplication.

TDD, especially the “mockist” variety, nudges us to add layers of indirection to separate responsibilities cleanly. Johnson seems to be implying that doing this systematically can add unnecessary complexity to the system, making it harder to change, paradoxically undermining one of TDD’s goals.

I do not think that lots of loose coupling makes things harder to change. It does increase the number of interfaces, but it makes it easier to swap out implementations or to limit behavior changes to a single class.

And what about the complexity of the test code? Brown and Heinemeir Hansson seem to act as if reducing the complexity of the test code does not matter. Or rather, that you don’t need to write tests for code that’s hard to test because you should just expand the scope of the tests to do verification at the level of whole components.

Here’s where I get back to that other important reason why “isolated” tests are necessary: math. J.B. Rainsberger simply destroys the arguments of the kind that Brown and Heinemeir Hansson make and their emphasis on component-level tests. He points out that there’s an explosive multiplicative effect on the number of tests needed when you test classes in combination. For an oversimplified example, if your service class has 10 execution paths and its calls to your storage class have 10 execution paths on average, testing them as a component, you may need to write as may as 100 tests to get full coverage of the component. Testing them as separate units, you only need 20 tests to get the same coverage. Imagine your component has 10 interdependent classes like that… Do you have the developer bandwidth to write all those tests? If you do write them all, how easy is it to change something in your component? How many of those tests will break if you make one simple change?

So I reject the idea that TDD “damages” the design. If you think TDD would damage your design, maybe you just don’t know how bad your design is, because most of your code is not really tested.

As for Heinemeir Hansson’s contention that it’s outdated thinking to isolate tests from database access, he may be right about performance issues (not everyone has expensive development machines with fancy SSD drives, but there should be a way to run a modest number of database tests quickly). If a class’s single responsibility is closely linked to the database, I’m in favor of unit-testing it against a real database, but any other test that hits a real database should be considered an integration test. Brown proposes a re-shaped, “architecturally-aligned” testing pyramid with fewer unit tests and more integrated component tests. Because of the aforementioned combinatorial effect of coupling classes in the component, that approach would seem to require either writing (and frequently running) a lot more tests or releasing components which are not exhaustively tested.

A Testing Toolbox for Java 8

A few months ago, I presented interface-it, a java-8 tool to generate mixin interfaces. In case you don’t know what that means or why mixins could be useful to you, I wrote a short article which explains it.

One of the key motivations for the interface-it tool was to be able to generate mixins for the latest versions of unit-testing libraries like Mockito and AssertJ. Now you no longer have to worry about that, because I’m doing it for you. And more.

I now have several more projects to present to you.

Presenting tdd-mixins-junit4

Working backwards – if you want to have a great test fixture by adding only one dependency in your build configuration (your Maven Pom, Ivy xml, Gradle, or just a fat jar added to your classpath), use tdd-mixins-junit4. It gives you all the basics you need to do mocking and assertions with fluidity, simplicity and power.

Normally, that’s all you should need for your tests. Mockito allows you to to set up collaborating objects and verify behavior, and the extensions I added make it even easier to handle cases where, for example, you want to mock behavior based on arguments passed to the mock which are generated by your unit under test. As for verifying returned results, AssertJ and JUnit assertions allow you to verify any data returned by the unit under test.

Presenting tdd-mixins-core

If you do not want to use JUnit 4 (maybe you want to use TestNG or an early version of JUnit 5), then you can use tdd-mixins-core, which has everything that tdd-mixins-junit4 has, except the mixin for JUnit assertions and JUnit itself.

Presenting extended-mockito

So these tdd-mixins libraries notably give you mixins for the aforementioned libraries Mockito and AssertJ. As for Mockito, they use my extended-mockito (TODO: link to project) library, which not only provides mixins for classes like Mockito and BDDMockito, but it also provides extra matcher methods to simplify specifying matching arguments for mocked methods. For example:

when(myMockSpamFilter.isSpam(containsOneOrMoreOf("Viagra", "Cialis", "Payday loan")))
            .thenReturn(Boolean.TRUE);

See the project’s home page or the unit tests for more details.

Presenting template-example

As for AssertJ, it’s already quite extended for general purpose use, so there is no extended-assertj project, but if you want to take things farther, I did create a project called template-example, which shows how, with a little tweaking, you can use a Maven plugin to auto-generate custom assertions for your own JavaBeans which are combined with the AssertJ mixin from tdd-mixins-core. These custom assertions allow you to do smooth, fluent assertions for your own data types, allowing this sort of validation call:

assertThat(employee).employer().address().postalCode().startsWith("11");

With these tools, you can more productively write unit tests with powerful assertions and mocking. They give you a fixture that you can set up in any test class by implementing an interface or two – for example:

public class MyTest implements ExtendedMockito, AllAssertions {

Not Included

What’s missing from these tools? I wanted to keep the toolset light, so there are some excellent but more specialized tools which are not included. For example, I have generated a mixin for Jsoup, which is very useful if you need to validate generated HTML, but unless I hear a clamoring for it, I will leave it out of tdd-mixins-core because it adds a dependency that lots of people may not need. Same for extensions to AssertJ – I generated mixins for AssertJ-DB  and for AssertJGuava (UPDATE: also added one for Awaitility), but did not include them in tdd-mixins (you can copy and paste the generated mixins’ source files if you want to use them).

Another library which is useful but which does not lend itself to mixins (because it uses annotations rather than static calls) is Zohhak it simplifies testing methods which return results that depend on a wide variety of possible input values (such as mathematical calculations or business rules).

Gentle Introduction: Mockito

From time to time I will write an article aimed at introducing a tool or a concept to newbies, under the category “Gentle Introduction”. Since development technologies evolve quickly, you’re surely a newbie at something even if you’re a brilliant, accomplished developer.

Here I present the mock object framework Mockito.

Why do you need it?

For most of your unit tests (unit tests which are not integration tests – I’m still struggling to find exactly the right term to make this clear), you need to isolate the unit under test from everything external (so, for example, your suite of 500 tests can verify the logic of your code in 1 minute instead of wasting 20 minutes on calls to the database which are unnecessary for testing the logic).  You could do this by writing a stub class for every collaborator, but that’s an unwieldy approach, and you may find yourself spending more and more time maintaining a bunch of stubs as API’s evolve. A test double library like Mockito saves time and helps you write better, clearer, focused tests which run fast and are easier to maintain.

Why I chose Mockito

I used to use JMock as a mock objects framework, but I prefer Mockito because of its clearer syntax, its clear separation of stubbing and verification functionalities, and the fact that by default it ignores interactions which are not explicitly stubbed or verified (whereas with JMock adding a new interaction will almost always break your test).

What you can do with Mockito

The Mockito javadoc provides a lot of good introductory examples, but here’s a basic example to show what Mockito can do. It’s a piece of a TODO list service.  The full example code (which runs with JUnit 4) is available in a pastebin file. Below we see a test to verify a service method that delays by 1 day the deadline for a TODO item.  Using Mockito.when(), we set up the call which retrieves the item from storage to return a known value (this is known as stubbing), and we do verification (using Mockito.verify()) that the new deadline is correctly passed to storage:

	
@Test
public void can_delay_by_one_day() throws UnknownIdException {
	// Given:
	LocalDateTime originalDeadline = LocalDateTime.of(2016, Month.MARCH,
			20, 8, 00);
	// Stub the call to getById to return a known value
	Mockito.when(storage.getById(ITEM_ID)).thenReturn(
			new TodoItem(ITEM_ID, "Blog article", originalDeadline));

	// When:
	this.underTest.delayByOneDay(ITEM_ID);

	// Then:
	// Verify that the storage interface is called with the correct
	// arguments
	Mockito.verify(this.storage).updateDeadline(ITEM_ID,
			LocalDateTime.of(2016, Month.MARCH, 21, 8, 00));
}

If you use static imports, you’ll get code that looks more like:

verify(storage).updateDeadline(ITEM_ID, of(2016, MARCH, 21, 8, 00));

Here’s an example where we stub the storage retrieval method to throw an exception to verify that it’s not caught and swallowed by the service method:

	
@Test(expected = UnknownIdException.class)
public void should_pass_on_unknown_id_exception() throws UnknownIdException {
	// Given:
	Mockito.when(storage.getById(ITEM_ID)).thenThrow(
			new UnknownIdException(3L));
	
	// When:
	this.underTest.delayByOneDay(ITEM_ID);

	// Then: expect exception
}

More stuff you should know

If you understood everything up to here (stubbing to return a value, stubbing to throw an exception, and verifying a call with specific arguments), you know more than half of what you need to understand about Mockito to get a lot of value from it. It’s also important to know about matchers. If you have to verify a call but either do not know (or care about) or do not have access to an argument which will be passed, you can use matchers to cover the case you need to cover.  If you use a matcher for one argument of a call, you need to use it for all the arguments, so the Matchers class (as well as its subclass Mockito) contains a bunch of standard matchers like eq() and anyObject() for simple cases, and argThat() which allows you to use a more complex custom matcher.

You should know that stubbing void methods is a bit different.  Instead of calling

Mockito.when(myMock.voidMethod(arg0))...

you need to call:

Mockito.doAnswer(...).when(myMock).voidMethod(arg0);

There’s a nice explanation of this on StackOverflow.

You also need to know about Mockito.verifyNoMoreInteractions() in case you want the test to fail when there’s an unforeseen interaction with a particular mocked collaborator. There are also argument capturing (extracting arguments passed to mocked calls for later comparison or other analysis) and spies (partial mocking of real objects), which could be useful in certain cases.

Also, if you prefer a BDD-style approach to tests, and it bothers you to call when() in your “given” section, the class BDDMockito is for you.  It renames the “when” and “verify” methods to “given” and “then”.

Caveats and Limits

One caveat: I do not recommend using the auto-wiring annotations available with Mockito.  There are known limitations to the @InjectMocks annotation (as described here and here), which create some confusion and encourage questionable design practices.  Better to explicitly create and pass around mock objects. It makes your dependencies more visible, which is good design. Using @Mock (which is a factory annotation for a field of a class which is a mock) might save you a few keystrokes if you have a lot of mocks in one class, but you must either use Mockito’s test runner (JUnit allows only 1 test runner per class, and I think some are more useful than Mockito’s) or call MockitoAnnotations.initMocks(this) in your setup(), so unless you have a lot of mocks in one test class (which might be a code smell) I don’t think it’s worth it.

Mockito is based on a low-level proxy library (cglib or ByteBuddy, depending on the version of Mockito), which means that there are some limits to what it can do, but those limits are generally conducive to good design.  For example, Mockito, unlike PowerMock or JMockit, can not mock static methods.  If you’re developing new code, this is a good thing.  For legacy code, my goal would be to get to the point where the test code works with Mockito only, even if I have to use PowerMock to test things before refactoring.  Mockito can’t mock final classes or methods, either, but that’s another limitation which can be overcome by improving your design.

Mockito can mock concrete classes, but normally you should be mocking mostly interfaces.  If you follow the dependency inversion principle religiously, you can test the logic in your classes mocking only interfaces and maybe some abstract classes.  For concrete classes you don’t own, you can mock them, but it’s generally encouraged to wrap them in an abstraction with an interface that you do own and can mock instead.

I tested the example code above with Mockito versions 2.0.2-beta and 2.0.36-beta. Note that 2.0.36-beta has some breaking changes vis-à-vis earlier versions where matchers are concerned, though the workarounds for these are pretty easy. The example code here runs on both versions without any changes.

How to get Mockito

To try Mockito, you can download the 2.0.2-beta mockito-all fat jar from bintray.com, though you’d be better off using a tool like maven (or gradle or ivy) to automatically download. For the latest beta (currently 2.0.40), there is no fat jar – you will need mockito-core and the separate dependency jars – so a dependency manager tool like maven is even more useful.