Author: redgreenwp

Is StackWalker Useful?

In Java 9, Oracle plans to introduce the StackWalker class and API.

The old way (since Java 1.4) of getting stack trace info was to call:

StackTraceElement[] stackTrace = new Throwable().getStackTrace();

The new API provides more choices with a stream-based API. To do roughly the same thing as the old example, you’d call:

List<StackFrame> stack = StackWalker.getInstance().walk(s -> s.collect(Collectors.toList()));

It’s cleaner to be able to get stack trace information without having to create an otherwise superfluous Throwable object, and the new API facilitates recovering the Class object, rather than just the qualified class name.

However, there seems to have been no effort to retrofit Throwable for the new API. Most of the time, it’s when we’re dealing with exceptions that we’re interested in stack traces. It would be nice to have a slick stream-based or builder API which allows better control over which stack frames are printed in the logs. This is especially important for Java EE, where the code you wrote is at times a needle in the haystack of layers of proxy calls and reflection. It is possible to create such an API in Java 8, though its slickness would be limited by the fact that it would need to be applied on top of existing Throwable and Exception classes (you’d have to call it just before logging the exception, perhaps making a deep clone of the original exception with altered stack frame elements). It would be nicer to be able to add a filtering predicate or a more complete object we might call a StackTracePrinter to the Exception object before logging.

So I’m not sure in what cases I’d actually use StackWalker. It could be useful for one-off diagnostics that never get committed to production. Also, in the debugger you have the full stack trace in the UI, but maybe it could be faster to walk the stack in the debugger to find certain targeted elements. I’m stretching it a bit, because I don’t see a lot of use cases. Feel free to add a comment if you have a great use case for StackWalker which is not possible pre-Java 9.

Advertisements

A gentle introduction to Domain Driven Design

Domain Driven Design (DDD), first described in Eric Evans’s so-called “Blue Book” is a diverse set of practices. Because these practices are loosely associated and not tied to any specific tool, it can sometimes even be unclear whether you are in fact doing DDD or something else.

There are, however, some core practices in DDD which are worth understanding and doing. The goal of DDD is to put the focus of developers on the domain by removing so-called “accidental” complexity (a key goal for any software craftsperson). Some of the DDD ideas and practices can also be viewed as bringing some of the benefits of functional programming to the object-oriented world.

Key ideas:

Ubiquitous language

The idea of ubiquitous language is simple: The code is developed to solve a problem for some people, so developers and their code should use the same terms as the people for whom they are solving the problem.
This seems like common sense, except that it’s rare in practice for institutional/organizational reasons. Developers are off in their cloister building stuff using the language of developers and architects, and they end up having to translate between what they are building and what the domain experts and end users are expecting. This is accidental complexity which increases the likelihood of and the difficulty of correcting specification problems.

Object types:

DDD divides objects into several categories. Objects which fall clearly into one of these categories are easier to understand and work with in the context of a business domain…

  • Value objects

A value object represents a piece of information in the domain which does not change but which could be replaced. For example, a bank account could contain $55. The value object is a currency object with a numeric value of 55 and a currency type value of “USD”. It could be replaced by a value of $85 if a deposit of $30 is received.
A value object can normally be immutable (which makes it easier to do functional-style processing with no confusing side effects). If a value object is replaced by another value object of the same type with identical values, no one notices or cares.

  • Entities

An entity has identity and state. A customer is a good example of an entity. The customer might have a customer id. Unlike value objects, entities are not interchangeable. An entity will often have “invariants” which must be enforced (do not expose any public methods, including constructors, which could allow violation of an invariant, unless it’s absolutely required by a framework you need to use). An invariant is a logical condition which applies to an entity (for example, an order always has a product ID and a customer ID and an order date, or a Flight always has a departure airport code and an arrival airport code).

  • Aggregates

An aggregate is a cluster of associated objects with a “root”, which acts as a façade – objects outside the aggregate interact only with the root. The aggregate root is responsible for assuring in each of its methods that by the end of the method call all invariants are satisfied within the aggregate (within a single transaction, generally). An example of an aggregate root can be found in this article by Julie Lerman.

  • Domain Events

A domain event encapsulates an event which could potentially trigger actions in various components of your system.  The use of domain events is a kind of decoupled publish/subscribe design pattern (similar to Observer) which allows code to emit events which are handled without being responsible for how and by which components they are handled.

Modules

You already know what a module is in general. In DDD, according to Eric Evans, modules should “tell the story of the system and contain a cohesive set of concepts”, and their names should be part of the ubiquitous language and “reflect insight into the domain”.

Repositories

Repositories are objects which allow retrieval of entities, aggregate roots. In DDD a repository serves as an adapter between the domain object model and the persistence model.

Bounded contexts

A complex application may have to solve different problems for different people. Sometimes the problems are similar enough to each other that you’re tempted to solve both using the same objects and paradigms. Don’t. It’s been drilled into most of us that you need to reuse objects (and database tables) as much as possible to reduce the amount of code (DRY, which is a useful practice in general), but DDD optimizes above all to minimize confusion and cognitive load rather than minimizing lines of code. A bounded context means that you solve problem A with one set of classes using one set of terms and approaches and you solve problem B with a separate set of classes using distinct terms and approaches. Even if subsystems A and B have to communicate with each other, it’s generally easier in the long run to translate between A objects and B objects for communication than to mix A objects into the B implementation or vice-versa.

Shared Kernel

Speaking of our bounded contexts A and B, there can be a core domain which is used in both context A and context B. This is called a shared kernel.
In practice, a shared kernel is a set of interfaces and objects (typically in its own library, such as a jar file in Java) which allow code outside a bound context to interact with the context. This could be service interfaces, aggregate interfaces or DTO’s (data transfer objects – generally behavior-free objects used to pass entity or aggregate data to and from a service implementation). Since the shared kernel can be used in many different places by different teams and products, it should be as stable as possible.

Supple Design

Supple design is a set of patterns/practices, some borrowed from functional programming, which make it easier to reason about behavior of a part of the system without worrying about what the rest of the system is doing.

This includes:

  • Intention-revealing interfaces
    The idea is that an interface (via method names, signatures, etc.) reveals what it does, not how it’s done.
  • Side-effect-free functions
    If a method never changes the state of anything, other than perhaps creating one or more completely new objects, then it won’t cause deadlock or change the behavior of other methods being executed simultaneously. And if called again with the same inputs, you get the same result. It’s easy to verify with unit tests that there are no bugs.
  • Standalone classes
    Reduce coupling between different classes to reduce unnecessary complexity.
  • Assertions
    The idea is to enforce invariants and post-conditions of operations either in the implementation code or with unit tests (or both).

Associated design practices/architectures:

  • A Hexagonal Architecture makes DDD easier because it further separates the technical infrastructure from the domain.
  • CQRS with or without Event Sourcing is a complementary architecture/practice which simplifies the handling of data.  It’s a big topic beyond the scope of this article, but CQRS is a useful choice for reducing hidden complexity in the spirit of DDD by separating things that do not have to go together (queries and data modification).  Event sourcing is a useful technique for dealing with data that may change over time if you might ever need to know what the state was at a given moment in the past.
  • Behavior-driven development (BDD) is complementary to DDD because it encapsulates the domain in tests based on the ubiquitous language.
  • TDD is a good practice for pretty much any development project, and ensures that DDD assertions are validated early and often.
  • Functional programming (lambdas and streams in Java, for example) works well where supple design is applied.
  • Simple CRUD can coexist with DDD.  Some persisted objects are simple enough that there’s no need for assertions and other domain logic.

Getting started

Well, that’s a lot of interesting ideas, but how to get started using them in development projects? It’s not like a framework where there’s a tutorial that will give you a “Hello DDD” in half an hour.

Some early steps toward DDD are possible, though :

  • Start using value objects.  For example, you can replace numeric dollar amounts by immutable Money objects which have an amount and a currency code.
  • Start using a ubiquitous language.  Align the terminology used in the code and project structure with the domain language.
  • Identify and segregate modules and bounded contexts. You may already have some notion of modules, but maybe you can find that you’re sharing some code between them which shouldn’t be shared.
  • Get started using domain events.  There are some examples on the internet that help you get started, including this C#-oriented article by Udi Dahan.
  • Separate out shared kernels. The flip side of enforcing bounded contexts is sharing precisely what your module needs to share with other modules and the outside world.
  • Use repositories. You may already be doing this.  The key idea is to have domain code using domain storage logic (CRUD and search for entities and aggregates) rather than database logic.

 

Software Craftsmanship and Heroism

A certain restaurant in the Silicon Valley has a sign which says “This is a bad place for a diet. This is a good place for a diet.” There is no contradiction – just 2 different contexts.

So it is with Software Craftsmanship and heroism. The opposite of heroism in software development is a smooth operation high up the CMMI ladder, where everyone knows what to do and no one wastes time or stresses out because the process is moving forward. On the bottom of the CMMI ladder, on the other hand, it’s sometimes chaos, and a developer can be put under pressure to deliver in a hurry because of bad communication or poorly designed and/or poorly tested code encountering demanding clients.

Software craftsmanship is bad for heroism because it reduces the chaos that requires a heroic developer to step in and fix things in a hurry. There’s a software factory pipeline that doesn’t let logic failures reach the client.

Software craftsmanship is good for heroism because sometimes even with a solid agile process and a finely-tuned software factory pipeline and clean, unit-tested code, misconceptions can slip in and a client can do something that the unit tests did not foresee and (rightly) complain about it. And why is craftsmanship good for heroism in this case? Because the code with the error is clean, extensible and unit-testable. If you have an idea for a fix, you can try it out in minutes in a unit test instead of having to code blindly, build, run and test manually and then repeat the process until it’s debugged or you decide to take a different approach. Ultimately, the software craftsperson has to build and run and test manually, but only AFTER the key idea for a solution has been tested and debugged. So with craftsmanship, you may have some shining moments of glory, but the heroic struggle happens gradually, every day, maintaining the quality of the code and the software factory.

To mock or not to mock

That is the question… debated in a recent civilized exchange of comments on this blog.

Like Ringo Star, I’m not afraid to position myself as pro-mock.

Of course mocking is not the goal, just a means of quickly writing tests which are focused on a small piece of logic. The goal is maintainable, thoroughly-tested code which functions as expected.

I’m not religious about mocking. If the method you’re testing has a collaborating class which you can just instantiate without any hassles and it works perfectly in the test from the start, then go for it. No need to mock. But that’s not something I see very often, apart from data structures or purely mathematical functions with no side effects. Most collaborating classes will use the network or disk or exhibit some non-repeatable behavior. At a minimum, there may be behavior that’s dependent on its state and which could evolve as the product evolves. You will likely need an integration test to show that your classes work together, but when you’re unit testing, you should be focused on the method you’re testing and not its collaborators.

For me (and many others), an interface is like a contract. If an interface is injected into the class I’m testing, I assume the object that implements it will do what’s expected, and I don’t care how it’s done. I would rather mock (or stub) the interface than write a test which gets involved with the details of its implementation. The contract doesn’t say the implementation won’t change, it just promises a certain behavior (a result and/ or side effects). If requirements evolve and the contract changes (different behavior, different return type, and/or different arguments), there will be some pain whether or not you use mocks, but it’s often more painful with mocks because many tests have to change (though the changes tend to be simple to implement). On the flip side, if, with no contract change, a bug slips into the implementation of the collaborating class, none of the tests which mock it will fail. If the creator of the collaborating implemented good unit tests, the bug will be spotted instantly in that case. On the other hand, without mocks, all the tests using the real collaborating class could fail at once, in which case it can take a little digging to find the bug.

There’s also the perhaps more frequent case of implementation details changing without a change to the interface. In that case, if the tests use mocks, there’s no issue at all. If, on the other hand, the tests call a constructor of the collaborating class which changes to add a new dependency, or if suddenly the collaborating class requires database access, all the tests without mocks are broken, often with no easy fix other than mocking.

For me, mocks are part of a modular approach to development. You develop little components, and testing quickly and simply with mocks you are confident that each component works exactly as specified. A few integration tests prove that the components can work together. This is the same sort of philosophy used with physical products such as cars. If you change the tire of your car, you don’t lose sleep over the fact that the tire has never been tested on the wheel in question. You assume that the tire has been tested, and you know that the wheel worked with a different tire with the same standards, and that’s that.

See which lines of code are really tested with PIT

PIT (a.k.a. pitest) is a mutation testing tool which is fairly easy to set up (in maven or gradle) for most java applications.

The basic idea with mutation testing is that the tool will iterate over each line of code, changing that 1 line, and then run all the tests against the mutated code base. If the changed line provokes an error in at least one test, it’s considered covered. If no test fails, then the line is not covered.

It’s possible to configure PIT to ignore your slow integration tests or end-to-end tests, which is a good idea, because PIT will run the same tests over and over again. With isolated unit tests, it’s pretty fast and a great coverage tool.

I tried it out on my interface-it project. Here’s a link to my pom.xml file which uses the pitest maven plugin. I struggled a little with the configuration of target classes, and their getting started document was slightly off, but it did not take long to set up.

Here are a few screen captures of the resulting report, which includes drill-down to see each line of code and an explanation of the mutation for the line and the result of the mutation :

Main page

pitestmethod

pitestexplanation

 

When an upgrade is a downgrade

I recently blogged about software factory tools, but now I have to rant about one of those tools.

Sometimes, it seems, for non-technological reasons, a software editor will, perhaps intentionally, make the new version of its software worse than the previous one. Windows 8 comes to mind.

I did not expect a minor release of SonarQube server to fall into this category, but release 6.2 seems like a man-made disaster to me. The creators of SonarQube decided that it would be a good idea to suddenly and irrevocably remove several of its central features. They removed the distinction between unit and integration tests because it’s too hard for them to deal with. That could really mess up the data in your customized reports, but don’t worry – they also deleted your custom reports! All of them!

Apparently they removed dashboards and widgets because they don’t want SonarQube to be “seen as a multi-purpose aggregation and reporting tool”.  It’s true that as of 6.1 the reports they generated automatically were pretty good, but the reports you meticulously set up that your boss has been using in executive progress reports ? Gone forever. The only alternative is restoring the SonarQube database and re-installing version 6.1 without ever having the possibility to upgrade again. How’s that for progress?

Unit Test Culture

Nobody gets paid proportionally to the number of unit tests they produce. That’s a good thing.

But how, then, do we motivate the many developers who think that they should do unit tests and/or TDD but don’t? It’s not easy to do serious unit testing and TDD when there’s legacy code (i.e. code without unit tests), and it’s never the most important or urgent thing.

The short answer is “culture”. The whole team should understand that you write unit tests and do TDD because it’s what a professional developer does. A professional surgeon washes hands before surgery. A professional chef makes sure the kitchen is clean at closing time. You get the idea.

It also helps to have a software factory set up where the coverage rate is tracked. Requiring a specific coverage rate is a questionable practice because 100% is sometimes too much (for example, if you need to implement a 3rd-party interface with 50 methods just so you can use one of them, do you have to write bogus unit tests for the 49 empty method implementations which are required by the compiler?), and less than 100% can lead to developers avoiding testing the most difficult-to-test but most important code. However, it’s motivating to see the effect that one’s commit has on the coverage number, be it the overall coverage or the coverage for the specific classes or packages which were modified.

Key Software Factory Tools in the Java World

Here are some popular free and/or freemium tools for creating a software factory for a Java project :

I’ve written about these build tools before.

Here’s a chart showing the relative popularity of these 3 tools.

There are many more unit test facilitators available for free, and I’ve developed some lesser-known open-source unit test facilitators myself.

  • Integration test facilitators
    • Arquillian – allows you to run a test on a Java EE server (either a remote server or one configured and launched by your test code)
    • OpenEJB – a lightweight EJB framework implementation that can be launched in a test
    • DBUnit – set up and clean up after tests that use real database connections
    • Docker – actually, Docker is a facilitator for just about any sort of automation you want to do, but I put it in the integration test category because it allows you to spin up a database or other server from a test.

Note: do not mix the Arquillian and OpenEJB together in the same test suite – jar conflicts can ensue when tests are run manually in Eclipse

There are surely other tools I’ve left out.  Feel free to mention your favorites in the comments.

Software factories

In the real world, most of us are not quite there, but it’s now possible with only free tools to have a reliable product release process which consists of just developing and committing code. In the case of an editor releasing a versioned software product, or for a site where new features are released gradually or via AB testing, or in all cases where human testers and/or management approval are necessary, there would also have to be the click of a button to approve a release. I’m speaking in broad, hand-waving generality, but such a process, when achieved, can provide a competitive advantage to a product or site by reducing time-to-market for new features and reducing the risk of quality issues (translation: easier to develop faster with fewer bugs).

Of course setting up such a process requires a lot of up-front work, considerable expertise, and occasional maintenance down the road, and such a system will (by design, normally) come to a halt very quickly if it is fed anything other than clean, well-tested code.

This notion has been called a “software factory” or “continuous delivery” or “continuous deployment” system.

I’ll have more to say on this subject soon, but today I will just point you to a couple of interesting diagrams:

“Web Security Basics” on Martin Fowler’s Site

Security should be at the heart of our profession, but I’m not sure it ever has been. Job interview preparation for developer jobs is largely focused on algorithms, API’s and problem-solving. It’s probably happened somewhere, but I’ve never heard of an interviewer asking a candidate for a typical developer job about how to prevent SQL injection or XSS attacks. I’ve never seen a job posting for an ordinary developer job which mentioned OWASP in the list of desired buzzwords.

That may be changing slowly. The value created by software development has attracted the attention of a lot of thieves who would be happy to steal it. The software craftsperson has a duty to produce software which will discourage and repel those thieves, but good learning materials in the field of security have been scarce. The OWASP site has a great deal of info about security best practices, but it’s not organized in a way which makes it easy to learn.

Martin Fowler’s blog is now hosting an evolving document written by two software security experts which gives a lot of good advice which is also well-organized and well-explained. It’s a bit presumptious for them to call the document “Web Security Basics”, since most web sites do not yet follow all of their recommended practices (such as using only https for all connections), but the article may move a lot of companies to better protect their users’ data and information security. It should be required reading for software craftspeople.