We seem to have made great progess on the JUnit testing front, we have some base classes with utility methods, about 40 Junit tests and a wiki page describing Derby's JUnit use:
http://wiki.apache.org/db-derby/DerbyJUnitTesting I think now would be a good time to establish some of the policies and goals around JUnit in Derby. My current fear is that we are thinking too much about getting JUnit tests to run in the existing harness, rather than think of JUnit as the only harness moving forward. As an example, we have eight JUnit tests in jdbcapi package, seven of these are in the jdbcapi suite for the existing harness as *individual* tests. If one run these eight Junit tests as a single JUnit suite then the time taken is ~250 seconds, running them (all eight) as a derby harness test suite takes ~500 seconds. Imagine if that performance improvement was across all tests, derbyall would drop to two-three hours on my laptop and around an hour on my desktop. Great savings for everyone. So why not run these eight tests as a single JUnit suite from the derby test harness suite, why force them to be individual tests? Running the single JUnit suite in the derby harness took ~310 seconds. My goal would be every test in Derby is a Junit test and the Derby harness is removed. I think this will take some time and will be an incremental process, but that should be the underlying thought when working with tests. I've been thinking that this goal would factor down into some requirements/expectations/policies: - Any Derby Junit test will run sucessfully as a standalone test using JUnit test runners (i.e. any public non-abstract class in Derby that extends TestCase) - Every Derby package that contains JUnit tests has a class _Suite that runs all the Junit tests in that package (see below for an example). This can be directly for each JUnit test or through sub-suites. - Ideally this _Suite uses a single database for all the tests - Higher level suites would combine all the _Suites from the packages resulting in something like derbyall - Some policy & utilities around cleanup and database removal. - utility methods to automatically clean the database (remove tables etc.) based upon database meta data - utility method to remove the database and derby.log and anything else - assumption tests will leave a clean database and remove everything once complete, but optionally leave the database for the next test (set by the calling suite?) - some clear policy on the configuration stuff, I couldn't see from a quick look any guidance on how configurations are meant to work, e.g. how would I set up tests to run under embedded and client, would this be a high level suite that runs the jdbcapi._Suite test once in embedded and then once as client? There's probably a list of functions the current harness performs that needs to be implemented in JUnit (as test decorators?) but these could be tackled incrementally. My itch would be the security manager setup. Would be good to get a list of these. Thoughts, comments? Dan. ----- package org.apache.derbyTesting.functionTests.tests.jdbcapi; import org.apache.derbyTesting.functionTests.util.BaseTestCase; import junit.framework.Test; import junit.framework.TestSuite; public class _Suite extends BaseTestCase { public _Suite(String name) { super(name); } public static Test suite() { TestSuite suite= new TestSuite(); suite.addTest(ProcedureTest.suite()); suite.addTestSuite(ScrollResultSetTest.class); suite.addTest(ConcurrencyTest.suite()); suite.addTestSuite(HoldabilityTest.class); suite.addTest(SURQueryMixTest.suite()); suite.addTest(SURTest.suite()); suite.addTestSuite(UpdateXXXTest.class); suite.addTestSuite(URCoveringIndexTest.class); return suite; } }
