Java Code Splitter

This week I managed to work a bit with our test suite and make it run faster. When you work on a project, which has a codebase that grows and grows, it is natural that more and more tests are getting added. After some months you will be sitting with a test suite that runs a minute or even longer. This is the execution time for all our unit tests in the different Maven modules.

	[INFO] ------------------------------------------------------------------------
	[INFO] Reactor Summary:
	[INFO] ------------------------------------------------------------------------
	[INFO] Parent Project .............................. SUCCESS [3.800s]
	[INFO] Unit Testing Utilities ...................... SUCCESS [4.781s]
	[INFO] XML Parsing ................................. SUCCESS [12.239s]
	[INFO] Protobuf Generator .......................... SUCCESS [6.199s]
	[INFO] Domain Specific Code ........................ SUCCESS [21.804s]
	[INFO] Web Application ............................. SUCCESS [1:05.495s]
	[INFO] Debug Tool .................................. SUCCESS [7.753s]
	[INFO] ------------------------------------------------------------------------
	[INFO] ------------------------------------------------------------------------
	[INFO] BUILD SUCCESSFUL
	[INFO] ------------------------------------------------------------------------

view raw testtimes-before.txt hosted with ❤ by GitHub

Last year I read an article in the German Java Magazin about a library called org.patterntesting. The library comes with TestRunner that can be used to run all test methods within a test class in parallel. Just change your test to look like this:

	@RunWith(ParallelRunner.class)
	public class Base64EncoderTest {

view raw Base64EncoderTest.java hosted with ❤ by GitHub

This will of course not work for all your tests immediately as not all tests can be run in parallel. Often this is due to bad test or software design. For instance tests requiring write access to the same physical File, tests altering shared fields within a test class, tests changing static field values - just to name a few. As you refactor your tests, so that they can run concurrently, you will automatically improve the design and testability of your application. We had a couple of these "smelling" unit tests that needed to be refactored. So this is what the execution time looked like after running the tests with patterntesting.

	[INFO] ------------------------------------------------------------------------
	[INFO] Reactor Summary:
	[INFO] ------------------------------------------------------------------------
	[INFO] Parent Project .............................. SUCCESS [2.275s]
	[INFO] Unit Testing Utilities ...................... SUCCESS [1.813s]
	[INFO] XML Parsing ................................. SUCCESS [7.313s]
	[INFO] Protobuf Generator .......................... SUCCESS [2.970s]
	[INFO] Domain Specific Code ........................ SUCCESS [16.092s]
	[INFO] Web Application ............................. SUCCESS [47.017s]
	[INFO] Debug Tool .................................. SUCCESS [2.705s]
	[INFO] ------------------------------------------------------------------------
	[INFO] ------------------------------------------------------------------------
	[INFO] BUILD SUCCESSFUL
	[INFO] ------------------------------------------------------------------------

view raw testtimes-after.txt hosted with ❤ by GitHub

Saving 40 seconds does not seem a lot. But 40 seconds times 15 builds per day times 3 developers times 21 working days in a month brings you to 10,5 hours. Unfortunately, it isn't always as easy. Sometimes your test is already using the TestRunner, so you cannot just switch and use ParallelRunner. This is the case for all our Spring tests which were using the SpringJUnit4ClassRunner from Spring. I contacted one of the authors of the patterntesting library and got some help. In the latest version, patterntesting 1.2, there is a new TestRunner class ParallelProxyRunner, which can be used in connection with the DelegateTo annotation, to delegate to the original TestRunner while running the test in parallel. This works for the SpringJUnit4ClassRunner, but you have to be aware that the SpringJUnit4ClassRunner is not thread-safe (a problem that will be fixed in Spring 3.2). Though as a user of the patterntesting library you will never be affected by this - the ParallelProxyRunner will hide this problem for you.

This isn't everything patterntesting has to offer. My favorite thing is the @Broken annotation which replaces the @Ignore annotation in Junit.

	@Broken(till = "01-03-2012", why = "Fix needed to go out in a hurry, will fix later", user = "reik")
	@Test
	public void testValidateCollectibleCanBeGivenBasedOnClicks() {

view raw Broken.java hosted with ❤ by GitHub

One big anti-pattern in test driven development is developers adding @Ignore annotations and then never look at the test case again. When introduced the patterntesting library to other EA developers, I got a lot of responses like: "why do you have tests flagged as ignore or broken in the first place?" - it's bad practice. Yes, you are all right. But often reality is different. Game producers can get very pushy. Developers are forced to commit hot-fixes which can potentially break existing tests. Then the developer might not be able to fix the test for various reasons.

He or she is new in the team and doesn't have the big picture.
He or she is junior and doesn't know how stuff works.
The test is overly complicated so that only the author understands it.
It takes too long to fix it and something else has higher priority.

Just to name a few. Patterntesting comes adds other useful stuff for the testing toolbox. Here are some examples:

	@RunTestOn(osName = "Windows")
	@Test
	public void testCryptBillsFile() throws IOException {
	File tempFile = new File("C:/Temp", "file.txt");
	checkCrypt(tempFile);
	}

view raw RunTestOn.java hosted with ❤ by GitHub

	// tests if both equals and hashcode have been overwritten
	ObjectTester.assertEquals(x, y);

	// tests if a Serializable is correctly implemented
	SerializableTester.assertSerialization(x);

	// compare content of two files
	FileTester.assertContentEquals(file1, file2)

	// same as above for InputStream and Reader
	IOTester.assertContentEquals(in1, in2)

	// assert enough memory
	RuntimeTester.assertFreeMemory(required)

view raw Other.java hosted with ❤ by GitHub

More examples can be found here.

Testing just got better