In Part 1 of this tutorial, we looked at performing parameterized tests in JUnit. In this portion, we will look at a more recent addition to JUnit – Rules.

If you’ve been using JUnit, you’re almost certainly familiar with its @Before and @After annotations. A method annotated with @Before is run just before each test method is called, and a method annotated with @After is run just afterwards.

One obvious use of these methods is pre-test setup and post-test cleanup. Indeed, in JUnit 3, the methods were named setUp and tearDown. One alternate use of the @After method, however, it to do common “check that things went right” testing. As one example, if you are using mock objects from packages such as EasyMock, you might take the approach of calling verify on each of your mocks from the @After method. This saves you from having to remember to call verify in each and every test method. (Some people believe that this is poor test design. I’m not going to argue the point one way or another – I’m just going to say that in my experience it’s not uncommon.)

This approach has a problem however. The @After method is called after the test method whether or not the method succeeded. If something in the body of a test fails, it doesn’t usually make sense to verify your mocks, since we already know that we have a failure. In addition, if the test failed, it is quite likely that mock is also going to fail to verify, throwing additional exceptions. “Piling on” those errors may obscure exactly what went wrong.

Let’s set up a deliberately-simplified example:


public class ProblemsWithBeforeAfter
{
	private String stringThatShouldBeEvenLength;

	public ProblemsWithBeforeAfter()
	{
	}

	@Before
	public void before()
	{
		System.out.println("before");
	}

	@After
	public void checkCommonResultsAfterEachTest()
	{
		System.out.println("after");

		assertTrue(stringThatShouldBeEvenLength.length() % 2 == 0);
	}

	@Test
	public void aTestThatWorks()
	{
		System.out.println("aTestThatWorks");
		stringThatShouldBeEvenLength = "1234";
	}

	@Test
	public void aTestThatFailsUnexpectedly()
	{
		System.out.println("aTestThatFailsUnexpectedly");
		fail("test fails unexpectedly and doesn't set 'stringThatShouldBeEvenLength'");
	}
}

Here, each of the test cases is nominally supposed to end up outputting a string that is supposed to be of even length, so the length verification has been put into the @After method. The second test case, however, simulates a test method that fails unexpectedly, and thus doesn’t output the string. If you execute this test class, you will see the following on the console:

before
aTestThatWorks
after
before
aTestThatFailsUnexpectedly
after

This is as expected. JUnit will correctly indicate that the test has failed:
ProblemsWithBeforeAfter
but you can see that in addition to the failure, we’ve generated a NullPointerException that is the result of the @After method being called when its expectations were not met. This is distracting noise in the tests. Ideally, we’d like to run the verification only if the test passed. Unfortunately, you can’t do this, because the @After method doesn’t have access to the success or failure of the test.

As soon as you start talking about “I’d like to do X just after each method” or similar things, somebody’s going to shout out “Aspect Oriented Programming” or “AOP.” This is exactly the kind of thing that AOP was designed for. The @Before and @After annotations were a valuable step towards this, but are not fully generalized. JUnit’s Rule support, introduced in JUnit 4.8, is a much more “pure” AOP approach.

First, some concepts. At least as it pertains to Rules, JUnit represents each individual test invocation as a Statement. A Statement represents the class instance and specific method associated with that test. Thus, running tests consists of creating and invoking Statements. Statement is actually an interface, and has only one method – evaluate. What a Rule does is to allow you to intercept each Statement and, if you wish, wrap it up in another Statement that does stuff before and/or after the nested Statement is evaluated. In AOP-ese, this is providing “around advice” (the extra “before or after stuff”) on a “join point” (the individual test execution). Because your code now surrounds the invocation of the test method, you have much more control on exactly what happens before and after.

Here’s an example of how we could fix the problem we mentioned earlier with a Rule:

public class RulesInsteadOfBeforeAfter
{
	@Rule public PostCheckRule rule = new PostCheckRule();

	private String stringThatShouldBeEvenLength;

	public RulesInsteadOfBeforeAfter()
	{
	}

	@Test
	public void aTestThatWorks()
	{
		System.out.println("aTestThatWorks");
		stringThatShouldBeEvenLength = "1234";
	}

	@Test
	public void aTestThatFailsUnexpectedly()
	{
		System.out.println("aTestThatFailsUnexpectedly");
		fail("test fails unexpectedly and doesn't set 'stringThatShouldBeEvenLength'");
	}

	private class PostCheckRule implements TestRule
	{
		public PostCheckRule()
		{
		}

		public Statement apply(final Statement base, Description description)
		{
			return new Statement()
			{
				@Override
				public void evaluate() throws Throwable
				{
					System.out.println("before");
					base.evaluate();
					System.out.println("after");
					assertTrue(stringThatShouldBeEvenLength.length() % 2 == 0);
				}
			};
		}
	}
}

As you can see, on line 3 we have a public member variable annotated as a @Rule. Our PostCheckRule implements the JUnit TestRule interface as required. (In older versions of JUnit this was called MethodRule. MethodRule is still supported, but is deprecated.) TestRule has a single method named apply which, as you might expect, applies the rule to the Statement that is about to be evaluated. The apply method is passed the original Statement, and is expected to return a Statement that will result in the rule being applied. In our case, what we do is create a new Statement that prints "before", evaluates the original Statement, and then prints "after" and runs our check.

If we run this test class, we will see the following printed to the console:

before
aTestThatWorks
after
before
aTestThatFailsUnexpectedly

Note the omission of the final after. JUnit tests “fail” by throwing an exception, so when aTestThatFailsUnexpectedly fails, the exception causes the remainder of our custom Statement to be skipped. Of course, if it had been important for us to catch the error we could have. Thus, instead of skipping code if a test failed, we could have inserted code that only ran if the test failed by wrapping the base.evaluate call in a try/catch block. One practical example I’ve seen involves Selenium Web Driver, in which a screen shot of the offending web page is captured and logged when, and only when, a test fails.

Although this was perhaps a somewhat weak example, there are other things you can do with a Rule. The Description parameter passed to apply provides convenient access to information about the test Statement. One piece of information that can be useful on occasion are the annotations on individual test methods.

Suppose that we had a test class that required some kind of initialization on a test-by-test basis. Clearly, of course, we can do this using @Before methods, but what if the specific details of the initialization needed to be somewhat different from test method to test method? In this case, we’d have to put the initialization code (or calls to helper methods) in each and every test method. What if we could instead just annotate each method with the configuration it needed, and then create reusable code that would do the setup? Here’s an example:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@interface Configuration
{
	String value();
}

public class RulesThatCheckAnnotations
{
	@Rule
	public ConfigurationRule configurationRule = new ConfigurationRule();

	private String config;

	public RulesThatCheckAnnotations()
	{
	}

	@Configuration("1")
	@Test
	public void firstConfiguration()
	{
		assertEquals("config1", config);
	}

	@Configuration("2")
	@Test
	public void secondConfiguration()
	{
		assertEquals("config2", config);
	}

	private class ConfigurationRule implements TestRule
	{
		public ConfigurationRule()
		{
		}

		public Statement apply(final Statement base, final Description description)
		{
			return new Statement()
			{
				@Override
				public void evaluate() throws Throwable
				{
					Configuration configuration = description.getAnnotation(Configuration.class);
					config = "config" + configuration.value();
					base.evaluate();
				}
			};
		}
	}
}

Lines 1-6 define the @Configuration annotation we will use. In production use, of course, this would probably be in its own file, but we’ve included it here for simplicity. The two test methods are then each annotated with different values simulating the different configuration they expect.

As before, our ConfigurationRule creates a new Statement that wraps the original one. Here, it uses the description parameter to locate the Configuration annotation on the test method, and then uses information from that in order to do the configuration. In this simplistic example, it just populates a variable in the original test class, but obviously we could do just about anything required in terms of configuration.

If having extra capabilities isn’t enough, another of the advantages of using TestRule classes instead of @Before/@After coding is reusability. Properly designed, a TestRule class can be made reusable across test classes or projects. JUnit ships with a number of pre-defined TestRule classes, including:

  • ErrorCollector: collects multiple errors in one test method
  • ExpectedException: makes flexible assertions about thrown exceptions
  • ExternalResource: start and stop a server, for example
  • TemporaryFolder: create fresh files, and delete after test
  • TestName: remembers the test name for use during the method
  • TestWatcher: adds logic at events during method execution
  • Timeout: causes test to fail after a set time
  • Verifier: fails test if object state ends up incorrect

In AOP programming, you are not limited to a single piece of advice on a join point. This is true with JUnit rules as well. Here, we have a test class that implements two different rules:


public class RulesMultiple
{
	@Rule public MyRule one = new MyRule("one");
	@Rule public MyRule two = new MyRule("two");

	public RulesMultiple()
	{
	}

	@Test
	public void theTest()
	{
		System.out.println("test");
	}

	private class MyRule implements TestRule
	{
		private String name;

		public MyRule(String name)
		{
			this.name = name;
		}

		public Statement apply(final Statement base, final Description description)
		{
			return new Statement()
			{
				@Override
				public void evaluate() throws Throwable
				{
					System.out.println(name + " before");
					base.evaluate();
					System.out.println(name + " after");
				}
			};
		}
	}
}

If a test class has more than one Rule, all the rules will be concatenated. Thus, when I execute this test class, I see the following on the console:

two before
one before
test
one after
two after

When used this way, the order in which the rules is applied is not defined, so you might just as easily see

one before
two before
test
two after
one after

For many situations, the order may not make any difference. If they do, however, JUnit provides a convenient RuleChain class that provides deterministic ordering. Thus, if the pair of rules in the previous example were rewritten as

	@Rule
    public TestRule chain= RuleChain
                           .outerRule(new MyRule("outer"))
                           .around(new MyRule("inner"));

we would always be guaranteed to receive

outer before
inner before
test
inner after
outer after

Multiple .around invocations are supported, allowing rules to be nested as deeply as desired.

Finally:

  1. The examples above have focused on the @Rule annotation, which gives us access to the Statement associated with an individual test method. There is also a @ClassRule annotation that defines a rule at the class level. This gives access to a Statement associated with an entire class, providing the same type of AOP support that the @BeforeClass and @AfterClass annotations do.
  2. I mentioned that the fields annotated as @Rule must be public. Although the folks at JUnit could have gone spelunking inside our test classes to search for private fields, apparently they decided that was either not worth the effort, or they knew of cases in which that wouldn’t work. So, for the moment, the fields must be public. (public static for @ClassRule-annotated fields.)
  3. Rule support and @Before/@After support can be combined. Code annotated as either @Before or @After is executed inside that for a TestRule. Thus, you can think of the Statement passed to the inner-most TestRule as encapsulating both the test method and any @Before or @After methods. Similarly, @BeforeClass and @AfterClass processing are doine within the Statement provided to a @ClassRule.

Note that I’m not advocating that we completely discard @Before and @After annotations. In many cases they are perfectly adequate and, being a little more familiar to programmers, perhaps more obvious. But it’s nice to know that, when necessary, you have a more powerful tool available.

In Part 3 of this tutorial, we will cover JUnit’s assume functionality.