In the beginning was the Test...

JUnit offers many features besides the standard assertTrue/assertEquals methods most programmers use. Let’s browse through the newer and more exotic features. They might come in handy at some time.

JUnit 4.9 Feature Roundup

Assuming you know something about unit testing and JUnit in particular, I won’t start at the very bottom, but talk a little about the features introduces during the last few versions:

  • Matchers
  • Assumptions
  • Categories
  • Theories
  • Rules

I hope there is something new in here for you. JUnits javadoc documentation is very good, but there is no single place describing these features. It’s not my goal to give a thorough treatment of them here, but it might be a good starting point.

Matchers

By including Hamcrest (core) into the default JUnit distribution, JUnit now allows the usage of assertThat leading to much easier to read tests and better error messages:

@Test
public void testUsingAssertThat() {
  assertThat(42, is(greaterThan(43))); // note, this will fail
}

JUnit includes only the Hamcrest core matchers, if you want/need more matchers, include hamcrest-all 1.1. Included matchers are documented here for Hamcrest and here for JUnit additions.

Output:

java.lang.AssertionError: 
Expected: is a value greater than <43>
     got: <42>

Assumptions

Assumptions allow tests to be ignored if the assumed condition isn’t met (instead of failling).

This test will be ignored if it is run on a Windows OS (for example):

@Test
public void testUsingAssumeThat() {
  assumeThat(File.separator, is("/"));
  ...
}

It is also possible to use assumptions in @Before or @BeforeClass methods.

Output (for example):

Test 'org.interlinked.junit.assumption.BasicTest.testUsingAssumeThat' ignored
org.junit.internal.AssumptionViolatedException: got: "\", expected: is "/"

Categories

Using categories it is possible to run only a subset of the tests. For example slow tests, integration tests etc.

Here, TestCategoryA and TestCategoryB are empty interfaces used to mark the tests:

@Test
@Category(TestCategoryA.class)
public void testCatA() {
  System.out.println("Category A test");
}

@Test
@Category(TestCategoryB.class)
public void testCatB() {
  System.out.println("Category B test");
}

@Test
@Category({ TestCategoryA.class, TestCategoryB.class })
public void testCatAB() {
  System.out.println("Category A and B test");
}

Using the Categories suite, we can now execute only those tests that are in “Category A”, but not in “Category B”:

@RunWith(Categories.class)
@Categories.IncludeCategory(TestCategoryA.class) // this would run tests CatA and CatAB
@Categories.ExcludeCategory(TestCategoryB.class) // now test CatAB is excluded too
@Suite.SuiteClasses(BasicTest.class)
public class CategoryASuite { }

Output:

Category A test

Theories

With theories we can write parameterized tests. We define a few theories and some datapoints. JUnit will match the types of the datapoints and the theories.

Again, we have to use a special suite class Theories:

@RunWith(Theories.class)
public class TheoryTest {
  @DataPoint public static final String POINT1 = "POINT1";
  @DataPoint public static final String POINT2 = "POINT2";

  // mind the plural!
  // uses only the items of the array, never the whole array!
  @DataPoints public static final String[] POINTS = new String[] {"abc", "cde", "efg", "ghi"};

  @DataPoint public static final String[] POINTS_ARRAY = POINTS;

  @Theory
  public void testTheory(String param) {
      System.out.println("Got: " + param);
  }

  @Theory
  public void testTheoryWithTwoParams(String param1, String param2) {
      System.out.println("Got " + param1 + " and " + param2);
  }

  @Theory
  public void testArray(String[] array) { // gets called with POINTS_ARRAY, nothing else
      System.out.println("Got called...");
      assertThat(array.length, is(equalTo(POINTS_ARRAY.length)));
  }
}

Output:

Got POINT1 and POINT1
Got POINT1 and POINT2
Got POINT1 and abc
Got POINT1 and cde
Got POINT1 and efg
Got POINT1 and ghi
Got POINT2 and POINT1
Got POINT2 and POINT2
Got POINT2 and abc
Got POINT2 and cde
Got POINT2 and efg
Got POINT2 and ghi
Got abc and POINT1
Got abc and POINT2
Got abc and abc
Got abc and cde
Got abc and efg
Got abc and ghi
Got cde and POINT1
Got cde and POINT2
Got cde and abc
Got cde and cde
Got cde and efg
Got cde and ghi
Got efg and POINT1
Got efg and POINT2
Got efg and abc
Got efg and cde
Got efg and efg
Got efg and ghi
Got ghi and POINT1
Got ghi and POINT2
Got ghi and abc
Got ghi and cde
Got ghi and efg
Got ghi and ghi
Got called...

Rules

Finally, rules allow us to add behaviour to tests. They can be thought of some kind of AOP for JUnit. Using rules, we can often omit class hierarchies and still reuse functionality using delegation.

JUnit includes some rules to start with, but it is very easy to write our own rules.

public class RuleTest {
  @Rule public TemporaryFolder temporaryFolder = new TemporaryFolder();
  @Rule public TestName testName = new TestName();
  private static boolean fileCreated = false;
  @Rule public LoggingRule loggingRuld = new LoggingRule();

  @Before
  public void printTestName() {
    System.out.println(testName.getMethodName());
  }

  @Test
  public void testCreatingAFile() throws IOException {
    File newFile = temporaryFolder.newFile("test1");
    assertThat(newFile.isFile(), is(true));
    fileCreated = true;
  }

  @Test
  public  void testCheckIfItExists() { // depends on testCreatingAFile...
    assumeTrue(fileCreated); // just to be sure ;)
    File file = new File(temporaryFolder.getRoot().getAbsolutePath() + "/test1");
    // the file should not exist (unless we use ClassRule for the TemporaryFolder, for example)
    assertThat(file.isFile(), is(false));
  }

  @Test(expected = NullPointerException.class)
  public void testThrowException() {
    throw new NullPointerException();
  }
}

Output:

Starting: testCreatingAFile
testCreatingAFile
Finished: testCreatingAFile
Starting: testCheckIfItExists
testCheckIfItExists
Finished: testCheckIfItExists
Starting: testThrowException
testThrowException
Finished: testThrowException

The TemporaryFolder and TestName rules are included in JUnit, the LoggingRule is a simple example:

public class LoggingRule extends TestWatcher {
  @Override
  protected void starting(Description description) {
    System.out.println("Starting: " + description.getMethodName());
  }

  @Override
  protected void finished(Description description) {
    System.out.println("Finished: " + description.getMethodName());
  }
}

Other rules included (see JUnit’s javadoc):

  • ErrorCollector: collect multiple errors in one test method
  • ExpectedException: make flexible assertions about thrown exceptions
  • ExternalResource: start and stop a server, for example
  • TemporaryFolder: create fresh files, and delete after test
  • TestName: remember the test name for use during the method
  • TestWatcher: add logic at events during method execution
  • Timeout: cause test to fail after a set time
  • Verifier: fail test if object state ends up incorrect

Unfortunately, rules seem to be local to the defining class, so you can’t put the into the suite class like @Before and @BeforeClass (which is really nice for opening external resources once for all tests).

Misc additions

Infinitest

For each change you make, Infinitest runs all the dependent tests. It’s continous testing for Eclipse and IDEA - free and open source (written by inproving works)!

ClasspathSuite

Most IDEs have their own ways for finding test classes to run, but usually I like to be IDE independent. Using the ClasspathSuite it is possible to have JUnit detect all test classes (or a subset of them) within the classpath (written by Johannes Link). There are efforts to include it into the standard distribution of JUnit.

Every programmer knows they should write tests for their code. Few do. The universal response to “Why not?” is “I’m in too much of a hurry.” This quickly becomes a vicious cycle- the more pressure you feel, the fewer tests you write. The fewer tests you write, the less productive you are and the less stable your code becomes. The less productive and accurate you are, the more pressure you feel.
— Kent Beck/Erich Gamma – JUnit Test Infected