I've been experimenting with JUnit4 extensions in an attempt to find a
more simple approach to our basic testing needs. The two major
requirements that I see in Harmony's test infrastructure (at least the
classlib stuff) are - exclusions and platform-specific tests. (I'm
ignoring the bootstrap-classpath stuff for now.) To handle these
today, we have exclusion files, which are loaded and used to exclude
certain files and then we branch the test folders by platform. Each
of these has issues - exclusions are skipping whole files, not just
specific failing tests; exclusions aren't well reported on; test files
in multiple branches can have duplicate code, etc.
One concept I've been working with is using annotations to describe
the tests for the purposes of exclusions and for platform definition.
The annotations can then be utilized in many ways via JUnit - method
rules, request processing filters and others. Here's an example of how
the tests might look.
class FileTest {
@Test
@Platform(os="windows")
public void testSomethingOnWindows { }
@Test
@Platform(os="linux", arch="x86_64")
public void testSomethingOnLinuxX86_64 {}
@Test
@Exclude(os="windows",vm="drlvm")
public void testSomethingExcludedOnWindowsDRLVM {}
}
Thoughts? Comments?
