Tag: Gentle Introduction

Gentle Introduction: Docker

The tool I present in this article is the container technology phenomenon known as docker.  If you know a lot about docker already, this article is not for you (unless you have suggestions to improve the article).

What is it?

Briefly, docker containers are like virtual machines, except that instead of emulating a whole system, each container runs as an isolated process on a shared kernel. This is much more efficient. Each container is specified by a file (DockerFile – more on this later, if you get all the way to the end of this post), which is basically a scripting and configuration file for setting up everything that differs from the default system.

What’s the big deal?

I do not enjoy system administration or configuration, so docker does not come naturally to me as an interest, but it’s probably going to change the way all developers work (it’s already changed things for some).

What’s so revolutionary about docker and containers? Surely it’s an infrastructure thing best left to the operations people, right? Surely it’s only popular because it lets cloud infrastructure providers save money on hardware because containers use fewer resources than full VM’s?

No. Basically, because of docker, execution environment can be treated as code.

infrastructure == code

In the not-so-far future, because of containers, your build tool will not only build, package and deploy the software you develop, but it will also build, package and deploy the execution environment on which your software will run, including installing database(s), application server, etc.  Some people are already doing this using custom gradle code. In the long run, containers might even make virtual machine environments like the JVM and the CLR obsolete, because those environments were originally created to shield programs from differences between execution environments.

Docker has a central repository, Docker hub, which allows docker users to pull execution environment configurations (known as “images”) from the internet using the command line in the way that maven’s central repository allows maven to pull jar files.

Hello, Postgres

So let’s go through some steps to do something rather basic with docker: installing and running a postgresql database server in a docker container. It’s easy, because postgresql is freely available on docker hub.

Running a database in a container could be useful for integration tests – each developer would have an identical local database installation.

First you need to install docker.  You’ll need to follow the instructions for your machine on the docker installation docs page.
I installed it on Windows. First I had to fiddle with my BIOS menu to enable virtualization.  Then I downloaded and launched the Docker Toolbox binary. Pretty straightforward stuff.  The ToolBox installed an icon on my desktop called Docker Quickstart Terminal. Double-clicking it launches docker in a bash shell command-line window (in a VM). It takes a while to warm up. The VM is assigned a fairly standard IP address which is shown in the terminal when finishes launching. For me it was 192.168.99.100.

dockerIP

Once you get a text prompt, you can pull the latest stable version of the postgresql image using the command:

docker pull postgres

This may take some time to finish all the downloads.

Once done, you can create a container called my-postgres to run the database:

docker run -p 5432:5432 --name my-postgres -e POSTGRES_PASSWORD=password -d postgres

Note that the -p option maps a port of the container to a port on the host. So for my-postgres the default postgres odbc port will be accessible from outside the container.
The -e option sets environment variables (in this case it sets the password for the default user “postgres” to be “password” (OWASP would not approve, but this is a tutorial).
The -d option runs the container as a background (a.k.a. “detached” or “daemon”) process and this option is not followed by an argument (the “postgres” at the end of the command is the name of the image to use).
On linux, you may need to prefix docker commands with sudo.

You can verify that the container is running with the command docker ps -a.

If you need to reboot your machine or if for some reason you find that your container is not running, you can restart it with the command `docker restart my-postgres`.

So how do you access that database now?  You need to know the IP address to connect to.  If you installed on Windows using the Docker Toolbox, and if you can’t find the IP in your Docker Quickstart Terminal window, the IP should appear in the output of the command

docker-machine ls

dockerMachine2

To get the IP on linux (don’t bother doing this on Windows, the result is only useful within the linux environment running docker), it seems you need to call

docker inspect --format '{{.NetworkSettings.IPAddress}}' my-postgres

Once you have the IP you can access via ODBC on port 5432 – using a tool like PGAdmin, for example. And you can write unit tests or create data sources in your web app using the IP and the port 5432.

And then?

This is already pretty cool, but you can go a lot farther with docker, linking data containers to application containers so that they can communicate with each other, extending the postgres container image to call SQL scripts on startup, mapping file directories in the container to file directories on the physical machine, etc. It can get a bit complicated, but the docker user guide is a good place to look to find out how to do all that.

A closer look at the DockerFile

I said earlier that infrastructure is now code, with Docker.  For the PostgreSQL container we used in our example, let’s take a quick, hand-wavy look at the DockerFile code under the hood.  You don’t need to do this to use the PostgreSQL image, but this is a techie blog, so let’s have a look under the hood.  The DockerFile for postgres 9.5 is at this link on docker hub.

The first line is:
FROM debian:jessie

Inheritance! A DockerFile starts with a FROM to indicate the parent image upon which it builds. By the way, the parent DockerFile debian:jessie is here, and it builds upon what is effectively the root image.

In the postgres DockerFile, there is a variety of commands. We see RUN used to launch linux commands like mkdir (to create file directories) and apt-get (to download and install software packages). We see ENV commands that set environment variables, like “PATH” and “PGDATA”. The EXPOSE command in the file tells Docker that the port in question is used by the container but does not make it visible from the outside (we did that with our docker run command above using the -p option). There’s also the VOLUME command which creates a mount point for a disk, a COPY command which copies into the container a shell script file which is part of the image (we see here that the postgres image consists of a DockerFile and a shell script file). There’s also an ENTRYPOINT command which launches the script file when the container is launched (ENTRYPOINT is different from RUN in that ENTRYPOINT associates the life cycle and output of the application launched with the life cycle and output of the container, allowing the use of a container to run a one-shot command and output the result, for example).

Advertisements

Gentle Introduction: Mockito

From time to time I will write an article aimed at introducing a tool or a concept to newbies, under the category “Gentle Introduction”. Since development technologies evolve quickly, you’re surely a newbie at something even if you’re a brilliant, accomplished developer.

Here I present the mock object framework Mockito.

Why do you need it?

For most of your unit tests (unit tests which are not integration tests – I’m still struggling to find exactly the right term to make this clear), you need to isolate the unit under test from everything external (so, for example, your suite of 500 tests can verify the logic of your code in 1 minute instead of wasting 20 minutes on calls to the database which are unnecessary for testing the logic).  You could do this by writing a stub class for every collaborator, but that’s an unwieldy approach, and you may find yourself spending more and more time maintaining a bunch of stubs as API’s evolve. A test double library like Mockito saves time and helps you write better, clearer, focused tests which run fast and are easier to maintain.

Why I chose Mockito

I used to use JMock as a mock objects framework, but I prefer Mockito because of its clearer syntax, its clear separation of stubbing and verification functionalities, and the fact that by default it ignores interactions which are not explicitly stubbed or verified (whereas with JMock adding a new interaction will almost always break your test).

What you can do with Mockito

The Mockito javadoc provides a lot of good introductory examples, but here’s a basic example to show what Mockito can do. It’s a piece of a TODO list service.  The full example code (which runs with JUnit 4) is available in a pastebin file. Below we see a test to verify a service method that delays by 1 day the deadline for a TODO item.  Using Mockito.when(), we set up the call which retrieves the item from storage to return a known value (this is known as stubbing), and we do verification (using Mockito.verify()) that the new deadline is correctly passed to storage:

	
@Test
public void can_delay_by_one_day() throws UnknownIdException {
	// Given:
	LocalDateTime originalDeadline = LocalDateTime.of(2016, Month.MARCH,
			20, 8, 00);
	// Stub the call to getById to return a known value
	Mockito.when(storage.getById(ITEM_ID)).thenReturn(
			new TodoItem(ITEM_ID, "Blog article", originalDeadline));

	// When:
	this.underTest.delayByOneDay(ITEM_ID);

	// Then:
	// Verify that the storage interface is called with the correct
	// arguments
	Mockito.verify(this.storage).updateDeadline(ITEM_ID,
			LocalDateTime.of(2016, Month.MARCH, 21, 8, 00));
}

If you use static imports, you’ll get code that looks more like:

verify(storage).updateDeadline(ITEM_ID, of(2016, MARCH, 21, 8, 00));

Here’s an example where we stub the storage retrieval method to throw an exception to verify that it’s not caught and swallowed by the service method:

	
@Test(expected = UnknownIdException.class)
public void should_pass_on_unknown_id_exception() throws UnknownIdException {
	// Given:
	Mockito.when(storage.getById(ITEM_ID)).thenThrow(
			new UnknownIdException(3L));
	
	// When:
	this.underTest.delayByOneDay(ITEM_ID);

	// Then: expect exception
}

More stuff you should know

If you understood everything up to here (stubbing to return a value, stubbing to throw an exception, and verifying a call with specific arguments), you know more than half of what you need to understand about Mockito to get a lot of value from it. It’s also important to know about matchers. If you have to verify a call but either do not know (or care about) or do not have access to an argument which will be passed, you can use matchers to cover the case you need to cover.  If you use a matcher for one argument of a call, you need to use it for all the arguments, so the Matchers class (as well as its subclass Mockito) contains a bunch of standard matchers like eq() and anyObject() for simple cases, and argThat() which allows you to use a more complex custom matcher.

You should know that stubbing void methods is a bit different.  Instead of calling

Mockito.when(myMock.voidMethod(arg0))...

you need to call:

Mockito.doAnswer(...).when(myMock).voidMethod(arg0);

There’s a nice explanation of this on StackOverflow.

You also need to know about Mockito.verifyNoMoreInteractions() in case you want the test to fail when there’s an unforeseen interaction with a particular mocked collaborator. There are also argument capturing (extracting arguments passed to mocked calls for later comparison or other analysis) and spies (partial mocking of real objects), which could be useful in certain cases.

Also, if you prefer a BDD-style approach to tests, and it bothers you to call when() in your “given” section, the class BDDMockito is for you.  It renames the “when” and “verify” methods to “given” and “then”.

Caveats and Limits

One caveat: I do not recommend using the auto-wiring annotations available with Mockito.  There are known limitations to the @InjectMocks annotation (as described here and here), which create some confusion and encourage questionable design practices.  Better to explicitly create and pass around mock objects. It makes your dependencies more visible, which is good design. Using @Mock (which is a factory annotation for a field of a class which is a mock) might save you a few keystrokes if you have a lot of mocks in one class, but you must either use Mockito’s test runner (JUnit allows only 1 test runner per class, and I think some are more useful than Mockito’s) or call MockitoAnnotations.initMocks(this) in your setup(), so unless you have a lot of mocks in one test class (which might be a code smell) I don’t think it’s worth it.

Mockito is based on a low-level proxy library (cglib or ByteBuddy, depending on the version of Mockito), which means that there are some limits to what it can do, but those limits are generally conducive to good design.  For example, Mockito, unlike PowerMock or JMockit, can not mock static methods.  If you’re developing new code, this is a good thing.  For legacy code, my goal would be to get to the point where the test code works with Mockito only, even if I have to use PowerMock to test things before refactoring.  Mockito can’t mock final classes or methods, either, but that’s another limitation which can be overcome by improving your design.

Mockito can mock concrete classes, but normally you should be mocking mostly interfaces.  If you follow the dependency inversion principle religiously, you can test the logic in your classes mocking only interfaces and maybe some abstract classes.  For concrete classes you don’t own, you can mock them, but it’s generally encouraged to wrap them in an abstraction with an interface that you do own and can mock instead.

I tested the example code above with Mockito versions 2.0.2-beta and 2.0.36-beta. Note that 2.0.36-beta has some breaking changes vis-à-vis earlier versions where matchers are concerned, though the workarounds for these are pretty easy. The example code here runs on both versions without any changes.

How to get Mockito

To try Mockito, you can download the 2.0.2-beta mockito-all fat jar from bintray.com, though you’d be better off using a tool like maven (or gradle or ivy) to automatically download. For the latest beta (currently 2.0.40), there is no fat jar – you will need mockito-core and the separate dependency jars – so a dependency manager tool like maven is even more useful.