Category: Gentle Introduction

A gentle introduction to Domain Driven Design

Domain Driven Design (DDD), first described in Eric Evans’s so-called “Blue Book” is a diverse set of practices. Because these practices are loosely associated and not tied to any specific tool, it can sometimes even be unclear whether you are in fact doing DDD or something else.

There are, however, some core practices in DDD which are worth understanding and doing. The goal of DDD is to put the focus of developers on the domain by removing so-called “accidental” complexity (a key goal for any software craftsperson). Some of the DDD ideas and practices can also be viewed as bringing some of the benefits of functional programming to the object-oriented world.

Key ideas:

Ubiquitous language

The idea of ubiquitous language is simple: The code is developed to solve a problem for some people, so developers and their code should use the same terms as the people for whom they are solving the problem.
This seems like common sense, except that it’s rare in practice for institutional/organizational reasons. Developers are off in their cloister building stuff using the language of developers and architects, and they end up having to translate between what they are building and what the domain experts and end users are expecting. This is accidental complexity which increases the likelihood of and the difficulty of correcting specification problems.

Object types:

DDD divides objects into several categories. Objects which fall clearly into one of these categories are easier to understand and work with in the context of a business domain…

  • Value objects

A value object represents a piece of information in the domain which does not change but which could be replaced. For example, a bank account could contain $55. The value object is a currency object with a numeric value of 55 and a currency type value of “USD”. It could be replaced by a value of $85 if a deposit of $30 is received.
A value object can normally be immutable (which makes it easier to do functional-style processing with no confusing side effects). If a value object is replaced by another value object of the same type with identical values, no one notices or cares.

  • Entities

An entity has identity and state. A customer is a good example of an entity. The customer might have a customer id. Unlike value objects, entities are not interchangeable. An entity will often have “invariants” which must be enforced (do not expose any public methods, including constructors, which could allow violation of an invariant, unless it’s absolutely required by a framework you need to use). An invariant is a logical condition which applies to an entity (for example, an order always has a product ID and a customer ID and an order date, or a Flight always has a departure airport code and an arrival airport code).

  • Aggregates

An aggregate is a cluster of associated objects with a “root”, which acts as a façade – objects outside the aggregate interact only with the root. The aggregate root is responsible for assuring in each of its methods that by the end of the method call all invariants are satisfied within the aggregate (within a single transaction, generally). An example of an aggregate root can be found in this article by Julie Lerman.

  • Domain Events

A domain event encapsulates an event which could potentially trigger actions in various components of your system.  The use of domain events is a kind of decoupled publish/subscribe design pattern (similar to Observer) which allows code to emit events which are handled without being responsible for how and by which components they are handled.

Modules

You already know what a module is in general. In DDD, according to Eric Evans, modules should “tell the story of the system and contain a cohesive set of concepts”, and their names should be part of the ubiquitous language and “reflect insight into the domain”.

Repositories

Repositories are objects which allow retrieval of entities, aggregate roots. In DDD a repository serves as an adapter between the domain object model and the persistence model.

Bounded contexts

A complex application may have to solve different problems for different people. Sometimes the problems are similar enough to each other that you’re tempted to solve both using the same objects and paradigms. Don’t. It’s been drilled into most of us that you need to reuse objects (and database tables) as much as possible to reduce the amount of code (DRY, which is a useful practice in general), but DDD optimizes above all to minimize confusion and cognitive load rather than minimizing lines of code. A bounded context means that you solve problem A with one set of classes using one set of terms and approaches and you solve problem B with a separate set of classes using distinct terms and approaches. Even if subsystems A and B have to communicate with each other, it’s generally easier in the long run to translate between A objects and B objects for communication than to mix A objects into the B implementation or vice-versa.

Shared Kernel

Speaking of our bounded contexts A and B, there can be a core domain which is used in both context A and context B. This is called a shared kernel.
In practice, a shared kernel is a set of interfaces and objects (typically in its own library, such as a jar file in Java) which allow code outside a bound context to interact with the context. This could be service interfaces, aggregate interfaces or DTO’s (data transfer objects – generally behavior-free objects used to pass entity or aggregate data to and from a service implementation). Since the shared kernel can be used in many different places by different teams and products, it should be as stable as possible.

Supple Design

Supple design is a set of patterns/practices, some borrowed from functional programming, which make it easier to reason about behavior of a part of the system without worrying about what the rest of the system is doing.

This includes:

  • Intention-revealing interfaces
    The idea is that an interface (via method names, signatures, etc.) reveals what it does, not how it’s done.
  • Side-effect-free functions
    If a method never changes the state of anything, other than perhaps creating one or more completely new objects, then it won’t cause deadlock or change the behavior of other methods being executed simultaneously. And if called again with the same inputs, you get the same result. It’s easy to verify with unit tests that there are no bugs.
  • Standalone classes
    Reduce coupling between different classes to reduce unnecessary complexity.
  • Assertions
    The idea is to enforce invariants and post-conditions of operations either in the implementation code or with unit tests (or both).

Associated design practices/architectures:

  • A Hexagonal Architecture makes DDD easier because it further separates the technical infrastructure from the domain.
  • CQRS with or without Event Sourcing is a complementary architecture/practice which simplifies the handling of data.  It’s a big topic beyond the scope of this article, but CQRS is a useful choice for reducing hidden complexity in the spirit of DDD by separating things that do not have to go together (queries and data modification).  Event sourcing is a useful technique for dealing with data that may change over time if you might ever need to know what the state was at a given moment in the past.
  • Behavior-driven development (BDD) is complementary to DDD because it encapsulates the domain in tests based on the ubiquitous language.
  • TDD is a good practice for pretty much any development project, and ensures that DDD assertions are validated early and often.
  • Functional programming (lambdas and streams in Java, for example) works well where supple design is applied.
  • Simple CRUD can coexist with DDD.  Some persisted objects are simple enough that there’s no need for assertions and other domain logic.

Getting started

Well, that’s a lot of interesting ideas, but how to get started using them in development projects? It’s not like a framework where there’s a tutorial that will give you a “Hello DDD” in half an hour.

Some early steps toward DDD are possible, though :

  • Start using value objects.  For example, you can replace numeric dollar amounts by immutable Money objects which have an amount and a currency code.
  • Start using a ubiquitous language.  Align the terminology used in the code and project structure with the domain language.
  • Identify and segregate modules and bounded contexts. You may already have some notion of modules, but maybe you can find that you’re sharing some code between them which shouldn’t be shared.
  • Get started using domain events.  There are some examples on the internet that help you get started, including this C#-oriented article by Udi Dahan.
  • Separate out shared kernels. The flip side of enforcing bounded contexts is sharing precisely what your module needs to share with other modules and the outside world.
  • Use repositories. You may already be doing this.  The key idea is to have domain code using domain storage logic (CRUD and search for entities and aggregates) rather than database logic.

 

Advertisements

To mock or not to mock

That is the question… debated in a recent civilized exchange of comments on this blog.

Like Ringo Star, I’m not afraid to position myself as pro-mock.

Of course mocking is not the goal, just a means of quickly writing tests which are focused on a small piece of logic. The goal is maintainable, thoroughly-tested code which functions as expected.

I’m not religious about mocking. If the method you’re testing has a collaborating class which you can just instantiate without any hassles and it works perfectly in the test from the start, then go for it. No need to mock. But that’s not something I see very often, apart from data structures or purely mathematical functions with no side effects. Most collaborating classes will use the network or disk or exhibit some non-repeatable behavior. At a minimum, there may be behavior that’s dependent on its state and which could evolve as the product evolves. You will likely need an integration test to show that your classes work together, but when you’re unit testing, you should be focused on the method you’re testing and not its collaborators.

For me (and many others), an interface is like a contract. If an interface is injected into the class I’m testing, I assume the object that implements it will do what’s expected, and I don’t care how it’s done. I would rather mock (or stub) the interface than write a test which gets involved with the details of its implementation. The contract doesn’t say the implementation won’t change, it just promises a certain behavior (a result and/ or side effects). If requirements evolve and the contract changes (different behavior, different return type, and/or different arguments), there will be some pain whether or not you use mocks, but it’s often more painful with mocks because many tests have to change (though the changes tend to be simple to implement). On the flip side, if, with no contract change, a bug slips into the implementation of the collaborating class, none of the tests which mock it will fail. If the creator of the collaborating implemented good unit tests, the bug will be spotted instantly in that case. On the other hand, without mocks, all the tests using the real collaborating class could fail at once, in which case it can take a little digging to find the bug.

There’s also the perhaps more frequent case of implementation details changing without a change to the interface. In that case, if the tests use mocks, there’s no issue at all. If, on the other hand, the tests call a constructor of the collaborating class which changes to add a new dependency, or if suddenly the collaborating class requires database access, all the tests without mocks are broken, often with no easy fix other than mocking.

For me, mocks are part of a modular approach to development. You develop little components, and testing quickly and simply with mocks you are confident that each component works exactly as specified. A few integration tests prove that the components can work together. This is the same sort of philosophy used with physical products such as cars. If you change the tire of your car, you don’t lose sleep over the fact that the tire has never been tested on the wheel in question. You assume that the tire has been tested, and you know that the wheel worked with a different tire with the same standards, and that’s that.

Unit Test Culture

Nobody gets paid proportionally to the number of unit tests they produce. That’s a good thing.

But how, then, do we motivate the many developers who think that they should do unit tests and/or TDD but don’t? It’s not easy to do serious unit testing and TDD when there’s legacy code (i.e. code without unit tests), and it’s never the most important or urgent thing.

The short answer is “culture”. The whole team should understand that you write unit tests and do TDD because it’s what a professional developer does. A professional surgeon washes hands before surgery. A professional chef makes sure the kitchen is clean at closing time. You get the idea.

It also helps to have a software factory set up where the coverage rate is tracked. Requiring a specific coverage rate is a questionable practice because 100% is sometimes too much (for example, if you need to implement a 3rd-party interface with 50 methods just so you can use one of them, do you have to write bogus unit tests for the 49 empty method implementations which are required by the compiler?), and less than 100% can lead to developers avoiding testing the most difficult-to-test but most important code. However, it’s motivating to see the effect that one’s commit has on the coverage number, be it the overall coverage or the coverage for the specific classes or packages which were modified.

API is Legacy

Now that I’ve released version 1.1 of interface-it, I understand better why so many projects decide to do a big 2.0 release which includes breaking changes to API’s.

Once you’ve decided to support backwards compatibility for all the exposed API (all public and protected methods in the public classes, normally), minor releases hit a lot of technical debt because you can’t refactor everything you want to. TDD doesn’t save you from this .

With interface-it, I made a few assumptions about the API which changed very quickly when I discovered new requirements. For example, before the 1.0 release, I thought about encapsulating some of the arguments to key methods, but I didn’t see clear affiliations or behavior related to these arguments. It was just data needed for generating a mixin. For version 1.1, where you can generate a hierearchy of mixin types (just parent-child via the command-line interface, but the code API is more flexible), it was no longer data but behavior that I needed, because I had to manage multiple classes in the same call. Despite having excellent code coverage thanks to TDD, the change was a bit painful to implement, and I had to leave in place some methods which I would have preferred to delete.

Gentle Introduction: Docker

The tool I present in this article is the container technology phenomenon known as docker.  If you know a lot about docker already, this article is not for you (unless you have suggestions to improve the article).

What is it?

Briefly, docker containers are like virtual machines, except that instead of emulating a whole system, each container runs as an isolated process on a shared kernel. This is much more efficient. Each container is specified by a file (DockerFile – more on this later, if you get all the way to the end of this post), which is basically a scripting and configuration file for setting up everything that differs from the default system.

What’s the big deal?

I do not enjoy system administration or configuration, so docker does not come naturally to me as an interest, but it’s probably going to change the way all developers work (it’s already changed things for some).

What’s so revolutionary about docker and containers? Surely it’s an infrastructure thing best left to the operations people, right? Surely it’s only popular because it lets cloud infrastructure providers save money on hardware because containers use fewer resources than full VM’s?

No. Basically, because of docker, execution environment can be treated as code.

infrastructure == code

In the not-so-far future, because of containers, your build tool will not only build, package and deploy the software you develop, but it will also build, package and deploy the execution environment on which your software will run, including installing database(s), application server, etc.  Some people are already doing this using custom gradle code. In the long run, containers might even make virtual machine environments like the JVM and the CLR obsolete, because those environments were originally created to shield programs from differences between execution environments.

Docker has a central repository, Docker hub, which allows docker users to pull execution environment configurations (known as “images”) from the internet using the command line in the way that maven’s central repository allows maven to pull jar files.

Hello, Postgres

So let’s go through some steps to do something rather basic with docker: installing and running a postgresql database server in a docker container. It’s easy, because postgresql is freely available on docker hub.

Running a database in a container could be useful for integration tests – each developer would have an identical local database installation.

First you need to install docker.  You’ll need to follow the instructions for your machine on the docker installation docs page.
I installed it on Windows. First I had to fiddle with my BIOS menu to enable virtualization.  Then I downloaded and launched the Docker Toolbox binary. Pretty straightforward stuff.  The ToolBox installed an icon on my desktop called Docker Quickstart Terminal. Double-clicking it launches docker in a bash shell command-line window (in a VM). It takes a while to warm up. The VM is assigned a fairly standard IP address which is shown in the terminal when finishes launching. For me it was 192.168.99.100.

dockerIP

Once you get a text prompt, you can pull the latest stable version of the postgresql image using the command:

docker pull postgres

This may take some time to finish all the downloads.

Once done, you can create a container called my-postgres to run the database:

docker run -p 5432:5432 --name my-postgres -e POSTGRES_PASSWORD=password -d postgres

Note that the -p option maps a port of the container to a port on the host. So for my-postgres the default postgres odbc port will be accessible from outside the container.
The -e option sets environment variables (in this case it sets the password for the default user “postgres” to be “password” (OWASP would not approve, but this is a tutorial).
The -d option runs the container as a background (a.k.a. “detached” or “daemon”) process and this option is not followed by an argument (the “postgres” at the end of the command is the name of the image to use).
On linux, you may need to prefix docker commands with sudo.

You can verify that the container is running with the command docker ps -a.

If you need to reboot your machine or if for some reason you find that your container is not running, you can restart it with the command `docker restart my-postgres`.

So how do you access that database now?  You need to know the IP address to connect to.  If you installed on Windows using the Docker Toolbox, and if you can’t find the IP in your Docker Quickstart Terminal window, the IP should appear in the output of the command

docker-machine ls

dockerMachine2

To get the IP on linux (don’t bother doing this on Windows, the result is only useful within the linux environment running docker), it seems you need to call

docker inspect --format '{{.NetworkSettings.IPAddress}}' my-postgres

Once you have the IP you can access via ODBC on port 5432 – using a tool like PGAdmin, for example. And you can write unit tests or create data sources in your web app using the IP and the port 5432.

And then?

This is already pretty cool, but you can go a lot farther with docker, linking data containers to application containers so that they can communicate with each other, extending the postgres container image to call SQL scripts on startup, mapping file directories in the container to file directories on the physical machine, etc. It can get a bit complicated, but the docker user guide is a good place to look to find out how to do all that.

A closer look at the DockerFile

I said earlier that infrastructure is now code, with Docker.  For the PostgreSQL container we used in our example, let’s take a quick, hand-wavy look at the DockerFile code under the hood.  You don’t need to do this to use the PostgreSQL image, but this is a techie blog, so let’s have a look under the hood.  The DockerFile for postgres 9.5 is at this link on docker hub.

The first line is:
FROM debian:jessie

Inheritance! A DockerFile starts with a FROM to indicate the parent image upon which it builds. By the way, the parent DockerFile debian:jessie is here, and it builds upon what is effectively the root image.

In the postgres DockerFile, there is a variety of commands. We see RUN used to launch linux commands like mkdir (to create file directories) and apt-get (to download and install software packages). We see ENV commands that set environment variables, like “PATH” and “PGDATA”. The EXPOSE command in the file tells Docker that the port in question is used by the container but does not make it visible from the outside (we did that with our docker run command above using the -p option). There’s also the VOLUME command which creates a mount point for a disk, a COPY command which copies into the container a shell script file which is part of the image (we see here that the postgres image consists of a DockerFile and a shell script file). There’s also an ENTRYPOINT command which launches the script file when the container is launched (ENTRYPOINT is different from RUN in that ENTRYPOINT associates the life cycle and output of the application launched with the life cycle and output of the container, allowing the use of a container to run a one-shot command and output the result, for example).

Gentle Introduction: Mockito

From time to time I will write an article aimed at introducing a tool or a concept to newbies, under the category “Gentle Introduction”. Since development technologies evolve quickly, you’re surely a newbie at something even if you’re a brilliant, accomplished developer.

Here I present the mock object framework Mockito.

Why do you need it?

For most of your unit tests (unit tests which are not integration tests – I’m still struggling to find exactly the right term to make this clear), you need to isolate the unit under test from everything external (so, for example, your suite of 500 tests can verify the logic of your code in 1 minute instead of wasting 20 minutes on calls to the database which are unnecessary for testing the logic).  You could do this by writing a stub class for every collaborator, but that’s an unwieldy approach, and you may find yourself spending more and more time maintaining a bunch of stubs as API’s evolve. A test double library like Mockito saves time and helps you write better, clearer, focused tests which run fast and are easier to maintain.

Why I chose Mockito

I used to use JMock as a mock objects framework, but I prefer Mockito because of its clearer syntax, its clear separation of stubbing and verification functionalities, and the fact that by default it ignores interactions which are not explicitly stubbed or verified (whereas with JMock adding a new interaction will almost always break your test).

What you can do with Mockito

The Mockito javadoc provides a lot of good introductory examples, but here’s a basic example to show what Mockito can do. It’s a piece of a TODO list service.  The full example code (which runs with JUnit 4) is available in a pastebin file. Below we see a test to verify a service method that delays by 1 day the deadline for a TODO item.  Using Mockito.when(), we set up the call which retrieves the item from storage to return a known value (this is known as stubbing), and we do verification (using Mockito.verify()) that the new deadline is correctly passed to storage:

	
@Test
public void can_delay_by_one_day() throws UnknownIdException {
	// Given:
	LocalDateTime originalDeadline = LocalDateTime.of(2016, Month.MARCH,
			20, 8, 00);
	// Stub the call to getById to return a known value
	Mockito.when(storage.getById(ITEM_ID)).thenReturn(
			new TodoItem(ITEM_ID, "Blog article", originalDeadline));

	// When:
	this.underTest.delayByOneDay(ITEM_ID);

	// Then:
	// Verify that the storage interface is called with the correct
	// arguments
	Mockito.verify(this.storage).updateDeadline(ITEM_ID,
			LocalDateTime.of(2016, Month.MARCH, 21, 8, 00));
}

If you use static imports, you’ll get code that looks more like:

verify(storage).updateDeadline(ITEM_ID, of(2016, MARCH, 21, 8, 00));

Here’s an example where we stub the storage retrieval method to throw an exception to verify that it’s not caught and swallowed by the service method:

	
@Test(expected = UnknownIdException.class)
public void should_pass_on_unknown_id_exception() throws UnknownIdException {
	// Given:
	Mockito.when(storage.getById(ITEM_ID)).thenThrow(
			new UnknownIdException(3L));
	
	// When:
	this.underTest.delayByOneDay(ITEM_ID);

	// Then: expect exception
}

More stuff you should know

If you understood everything up to here (stubbing to return a value, stubbing to throw an exception, and verifying a call with specific arguments), you know more than half of what you need to understand about Mockito to get a lot of value from it. It’s also important to know about matchers. If you have to verify a call but either do not know (or care about) or do not have access to an argument which will be passed, you can use matchers to cover the case you need to cover.  If you use a matcher for one argument of a call, you need to use it for all the arguments, so the Matchers class (as well as its subclass Mockito) contains a bunch of standard matchers like eq() and anyObject() for simple cases, and argThat() which allows you to use a more complex custom matcher.

You should know that stubbing void methods is a bit different.  Instead of calling

Mockito.when(myMock.voidMethod(arg0))...

you need to call:

Mockito.doAnswer(...).when(myMock).voidMethod(arg0);

There’s a nice explanation of this on StackOverflow.

You also need to know about Mockito.verifyNoMoreInteractions() in case you want the test to fail when there’s an unforeseen interaction with a particular mocked collaborator. There are also argument capturing (extracting arguments passed to mocked calls for later comparison or other analysis) and spies (partial mocking of real objects), which could be useful in certain cases.

Also, if you prefer a BDD-style approach to tests, and it bothers you to call when() in your “given” section, the class BDDMockito is for you.  It renames the “when” and “verify” methods to “given” and “then”.

Caveats and Limits

One caveat: I do not recommend using the auto-wiring annotations available with Mockito.  There are known limitations to the @InjectMocks annotation (as described here and here), which create some confusion and encourage questionable design practices.  Better to explicitly create and pass around mock objects. It makes your dependencies more visible, which is good design. Using @Mock (which is a factory annotation for a field of a class which is a mock) might save you a few keystrokes if you have a lot of mocks in one class, but you must either use Mockito’s test runner (JUnit allows only 1 test runner per class, and I think some are more useful than Mockito’s) or call MockitoAnnotations.initMocks(this) in your setup(), so unless you have a lot of mocks in one test class (which might be a code smell) I don’t think it’s worth it.

Mockito is based on a low-level proxy library (cglib or ByteBuddy, depending on the version of Mockito), which means that there are some limits to what it can do, but those limits are generally conducive to good design.  For example, Mockito, unlike PowerMock or JMockit, can not mock static methods.  If you’re developing new code, this is a good thing.  For legacy code, my goal would be to get to the point where the test code works with Mockito only, even if I have to use PowerMock to test things before refactoring.  Mockito can’t mock final classes or methods, either, but that’s another limitation which can be overcome by improving your design.

Mockito can mock concrete classes, but normally you should be mocking mostly interfaces.  If you follow the dependency inversion principle religiously, you can test the logic in your classes mocking only interfaces and maybe some abstract classes.  For concrete classes you don’t own, you can mock them, but it’s generally encouraged to wrap them in an abstraction with an interface that you do own and can mock instead.

I tested the example code above with Mockito versions 2.0.2-beta and 2.0.36-beta. Note that 2.0.36-beta has some breaking changes vis-à-vis earlier versions where matchers are concerned, though the workarounds for these are pretty easy. The example code here runs on both versions without any changes.

How to get Mockito

To try Mockito, you can download the 2.0.2-beta mockito-all fat jar from bintray.com, though you’d be better off using a tool like maven (or gradle or ivy) to automatically download. For the latest beta (currently 2.0.40), there is no fat jar – you will need mockito-core and the separate dependency jars – so a dependency manager tool like maven is even more useful.