Tag: JUnit

Key Software Factory Tools in the Java World

Here are some popular free and/or freemium tools for creating a software factory for a Java project :

I’ve written about these build tools before.

Here’s a chart showing the relative popularity of these 3 tools.

There are many more unit test facilitators available for free, and I’ve developed some lesser-known open-source unit test facilitators myself.

  • Integration test facilitators
    • Arquillian – allows you to run a test on a Java EE server (either a remote server or one configured and launched by your test code)
    • OpenEJB – a lightweight EJB framework implementation that can be launched in a test
    • DBUnit – set up and clean up after tests that use real database connections
    • Docker – actually, Docker is a facilitator for just about any sort of automation you want to do, but I put it in the integration test category because it allows you to spin up a database or other server from a test.

Note: do not mix the Arquillian and OpenEJB together in the same test suite – jar conflicts can ensue when tests are run manually in Eclipse

There are surely other tools I’ve left out.  Feel free to mention your favorites in the comments.

Dan North’s JGoTesting library

Dan North, who is a pioneer in the field of Behavior-Driven Development (or rather “Behaviour-Driven” – he’s British) and a very entertaining public speaker on agile/extreme development practices, has created a new Java library which brings some ideas from the Go language to JUnit 4. His library JGoTesting is in the early stages, and there are some aspects which only seem useful for retrofitting existing tests. There’s also some competition from JUnit 5 and AssertJ for accomplishing some of the same goals, but it’s worth trying out.

The major goal of JGoTesting is to allow a single test to log multiple failures. In other words, the test goes on after the first failure, and at the end the tests fails with a list of errors. This is also possible with AssertJ SoftAssertions, and it’s standard functionality in JUnit 5 with assertAll(). One nice bonus in JGoTesting is that you can also log messages which are ignored if the test succeeded. So for integration tests or tests involving random generated values, you can get extra info about the cause of a failure without log pollution in the case of tests which succeed.

JGoTesting offers a testing DSL which distinguishes between failed checks and more serious errors. Syntactically, validations and logging can be called from the unique rule object for the test class. Calls can be chained together, and the calls are basically lambda-friendly for tests in Java 8 (JGoTesting is compiled in Java 7 for wider compatibility). JGoTesting also offers static delegate calls to replace calls to JUnit 4 assertions. These only seem useful for converting existing tests. As someone who auto-generates non-static mixin delegate calls for such API’s (TODO : LINK to tdd-mixins-junit4), I can’t really be a fan of replacing one static call by another, especially when there’s already a ready-made object with all the necessary semantics in non-static methods.

I took JGoTesting for a test drive (code available on github) to see what it can do and to see if it plays well with other API’s – I tried it with Zohhak, AssertJ and Mockito (the latter two via my tdd-mixins-junit4 library). Because JGoTesting uses a rule instead of a runner, there’s no conflict between it and Zohhak’s runner.

I discovered that JGoTest can be used in conjunction with Mockito verify() and with AssertJ assumptions and SoftAssumptions. JGoTest failed checks and logs which are executed before the failing error (whether it’s a Mockito failure, an AssertJ failure or a JGoTest failure, do indeed appear in the failure logs whether the test is run from Maven or from Eclipse. Maven gets a bit confused about the number of actual tests when there are JGoTest check failures.

My Maven output is available online if you don’t have time to run the test yourself, and here are some screen captures of results when run from Eclipse:

jgomockitowithlog

jgosoft1

Conclusion:

I don’t write a lot of tests which need a tool like JGoTest. I prefer tests which are focused on one case. For tests like that, there’s no need to log extra information or multiple failures– knowing the name of the test and the fact that it failed is enough to detect and quickly resolve the problem. For tests which are more integrated (with longer scenarios), or which test a number of possible parameters in a loop (as opposed to Zohhak-style parameterized tests), JGoTest’s log feature could be helpful to indicate where the test went wrong. As far as checking multiple failures in a test goes, while I like the simple syntax of JgoTest, I prefer the output generated by AssertJ SoftAssumptions. These are very early days for JGoTest, so I will keep an eye on it to see how it improves over time.

The TestNest Pattern

I discovered the TestNest pattern – the idea of using a suite of nested test classes in order to create a hierarchical organization of tests – in this blog post by Robert C. Martin, but apart from one other article I haven’t found a lot of information online about this technique. So I thought I’d try it out in a sort of hello-world to get a feel for it. As a bonus, I added in some tests using the excellent Zohhak test library which generates automatically nested tests for sets of values via an annotation.

My test class can be found here.

Here are some of the highlights…

The outer class declaration with its JUnit Suite annotations:

@RunWith(Suite.class)

@SuiteClasses({ MyExampleClassTest.Method1.class, MyExampleClassTest.Method2.class,

MyExampleClassTest.Calculation1.class, MyExampleClassTest.Calculation2.class})

public class MyExampleClassTest {

One of the nested classes in MyExampleClassTest is called SharedState. It contains the object under test (called underTest) and is the parent of all the other nested test classes (and it also uses a mixin from tdd-mixins-junit4 which gives all its subclasses the ability to call assertions non-statically).

There are 4 nested classes which contain tests – one for each method in the class under test. This seemed like a logical organization, though it’s not what Uncle Bob did in his aforementioned blog post. It might make sense to further subdivide between happy path tests and weird corner case tests (what Uncle Bob called “DegenerateTests” in his example), and there may be better ways to divide tests than along class and method lines (though I would hope that my classes and methods under test are cohesive units of organization).

Using the Suite runner in the parent class doesn’t prevent you from using other runners in the nested classes. Two of the nested classes use the Zohhak test runner:

@RunWith(ZohhakRunner.class)

I intentionally put 2 failures in the tests (one fail() in a normal test and one error in a Zohhak value set) to see what the failures would look like.

Here’s part of what Maven had to say about these failures:

Failed tests: should_fail(org.example.MyExampleClassTest$Method2): intentional failure to illustrate nested test failure messages

should_calculate_cube [-1, 1](org.example.MyExampleClassTest$Calculation2): expected:<[]1> but was:<[-]1>

Here’s what I see in Eclipse’s JUnit sub-window:

Eclipse JUnit run results showing 2 nested errors

This seems like a minor improvement over long, descriptive test method names for quickly getting a feel for where the problem isespecially when there are multiple simultaneous failures. I didn’t have to put the name of the method under test in the test method name, and in the eclipse JUnit runner UI the tests for each method are nicely grouped together.  Zohhak works well with this approach, as well (and seems like a pleasure to use in general for testing calculation results).

 

A Testing Toolbox for Java 8

A few months ago, I presented interface-it, a java-8 tool to generate mixin interfaces. In case you don’t know what that means or why mixins could be useful to you, I wrote a short article which explains it.

One of the key motivations for the interface-it tool was to be able to generate mixins for the latest versions of unit-testing libraries like Mockito and AssertJ. Now you no longer have to worry about that, because I’m doing it for you. And more.

I now have several more projects to present to you.

Presenting tdd-mixins-junit4

Working backwards – if you want to have a great test fixture by adding only one dependency in your build configuration (your Maven Pom, Ivy xml, Gradle, or just a fat jar added to your classpath), use tdd-mixins-junit4. It gives you all the basics you need to do mocking and assertions with fluidity, simplicity and power.

Normally, that’s all you should need for your tests. Mockito allows you to to set up collaborating objects and verify behavior, and the extensions I added make it even easier to handle cases where, for example, you want to mock behavior based on arguments passed to the mock which are generated by your unit under test. As for verifying returned results, AssertJ and JUnit assertions allow you to verify any data returned by the unit under test.

Presenting tdd-mixins-core

If you do not want to use JUnit 4 (maybe you want to use TestNG or an early version of JUnit 5), then you can use tdd-mixins-core, which has everything that tdd-mixins-junit4 has, except the mixin for JUnit assertions and JUnit itself.

Presenting extended-mockito

So these tdd-mixins libraries notably give you mixins for the aforementioned libraries Mockito and AssertJ. As for Mockito, they use my extended-mockito (TODO: link to project) library, which not only provides mixins for classes like Mockito and BDDMockito, but it also provides extra matcher methods to simplify specifying matching arguments for mocked methods. For example:

when(myMockSpamFilter.isSpam(containsOneOrMoreOf("Viagra", "Cialis", "Payday loan")))
            .thenReturn(Boolean.TRUE);

See the project’s home page or the unit tests for more details.

Presenting template-example

As for AssertJ, it’s already quite extended for general purpose use, so there is no extended-assertj project, but if you want to take things farther, I did create a project called template-example, which shows how, with a little tweaking, you can use a Maven plugin to auto-generate custom assertions for your own JavaBeans which are combined with the AssertJ mixin from tdd-mixins-core. These custom assertions allow you to do smooth, fluent assertions for your own data types, allowing this sort of validation call:

assertThat(employee).employer().address().postalCode().startsWith("11");

With these tools, you can more productively write unit tests with powerful assertions and mocking. They give you a fixture that you can set up in any test class by implementing an interface or two – for example:

public class MyTest implements ExtendedMockito, AllAssertions {

Not Included

What’s missing from these tools? I wanted to keep the toolset light, so there are some excellent but more specialized tools which are not included. For example, I have generated a mixin for Jsoup, which is very useful if you need to validate generated HTML, but unless I hear a clamoring for it, I will leave it out of tdd-mixins-core because it adds a dependency that lots of people may not need. Same for extensions to AssertJ – I generated mixins for AssertJ-DB  and for AssertJGuava (UPDATE: also added one for Awaitility), but did not include them in tdd-mixins (you can copy and paste the generated mixins’ source files if you want to use them).

Another library which is useful but which does not lend itself to mixins (because it uses annotations rather than static calls) is Zohhak it simplifies testing methods which return results that depend on a wide variety of possible input values (such as mathematical calculations or business rules).