Month: January 2017

See which lines of code are really tested with PIT

PIT (a.k.a. pitest) is a mutation testing tool which is fairly easy to set up (in maven or gradle) for most java applications.

The basic idea with mutation testing is that the tool will iterate over each line of code, changing that 1 line, and then run all the tests against the mutated code base. If the changed line provokes an error in at least one test, it’s considered covered. If no test fails, then the line is not covered.

It’s possible to configure PIT to ignore your slow integration tests or end-to-end tests, which is a good idea, because PIT will run the same tests over and over again. With isolated unit tests, it’s pretty fast and a great coverage tool.

I tried it out on my interface-it project. Here’s a link to my pom.xml file which uses the pitest maven plugin. I struggled a little with the configuration of target classes, and their getting started document was slightly off, but it did not take long to set up.

Here are a few screen captures of the resulting report, which includes drill-down to see each line of code and an explanation of the mutation for the line and the result of the mutation :

Main page

pitestmethod

pitestexplanation

 

Advertisements

When an upgrade is a downgrade

I recently blogged about software factory tools, but now I have to rant about one of those tools.

Sometimes, it seems, for non-technological reasons, a software editor will, perhaps intentionally, make the new version of its software worse than the previous one. Windows 8 comes to mind.

I did not expect a minor release of SonarQube server to fall into this category, but release 6.2 seems like a man-made disaster to me. The creators of SonarQube decided that it would be a good idea to suddenly and irrevocably remove several of its central features. They removed the distinction between unit and integration tests because it’s too hard for them to deal with. That could really mess up the data in your customized reports, but don’t worry – they also deleted your custom reports! All of them!

Apparently they removed dashboards and widgets because they don’t want SonarQube to be “seen as a multi-purpose aggregation and reporting tool”.  It’s true that as of 6.1 the reports they generated automatically were pretty good, but the reports you meticulously set up that your boss has been using in executive progress reports ? Gone forever. The only alternative is restoring the SonarQube database and re-installing version 6.1 without ever having the possibility to upgrade again. How’s that for progress?