Date: 2005-01-28T15:44:39 Editor: MichelleCaisse Wiki: Apache JDO Wiki Page: TestRunner URL: http://wiki.apache.org/jdo/TestRunner
no comment Change Log: ------------------------------------------------------------------------------ @@ -1,26 +1,31 @@ -== About TestRunner == -This project is about extending the TCK extensions of the JUnit test runners so that the same TCK tests can be run in multiple configurations. For example, the same TCK test program needs to be run with and without security turned on, and with Application Identity and Datastore Identity. When we add different mappings for Chapter 18 (ORM) tests, the same test will need to be run with security, identity, and different mappings. += About TestRunner = +This project is about extending the TCK extensions of the JUnit test runners so that the same TCK tests can be run in multiple configurations. For example, the same TCK test program needs to be run with and without security turned on, and with Application Identity and Datastore Identity. When we add different mappings for Chapter 18 (ORM) tests, the same test will need to be run with security, identity, and different mappings. W envision the test runner reading parameters from a configuration file. -== Configuration Parameters == -This is the list of configuration parameters that the test runner should be able to handle, read from an xml file: - *Test description string - *Test class - *Security on/off - *Application identity / Datastore identity - *Mapping file (.orm) - *Test data - -== Open Questions == - *Should the test runner execute a single test (comprised of a single combination of description, class, security setting, identity setting, mapping, and data)? Should it be able to run all test classes, or a specific list of test classes, with both security settings? with both identity types? What is the mechanism for running a batch of tests? - -== A Stab at Some Requirements == - - *Each test execution must report the full set of parameters, as listed above. Perhaps this report is the test description and no additional description need be specified by the test writer. - *All TCK tests must be run both with security on and security off (true or false?) - *All TCK tests must be run both with application identity and datastore identity (true or false?) - *Only a small subset of tests, some of those for chapter 18, must be run with multiple mapping files and multiple test data sets (true?) += Requirements = + 1. The user must be allowed to set all of the following configuration parameters: -This may indicate that security and identity should be handled differently than mapping and data. + Test description string + + Test class + + Security on/off + + Application identity / Datastore identity + + Mapping file (.orm) + + Test data + + 1.#2 To configure a test run, users should be able to set all configuration parameters in a text file, probably an xml file. + 1. In the configuration file the user must be able to specify either a single test or a list of tests to be run with a given set of configuration parameters. + 1. The user must be able to specify multiple configuration settings within one configuration file, each to be run with one or more test cases. + 1. Each test execution must report the full set of parameters, as listed above. + += Implementation Issues = + +== Role of build.xml == + +Because classpath and system properties must be set before the test runner is invoked, build.xml must play a role in the implementation. Michael will investigate whether the build process can read from the configuration file so that all parameters can be set in one location, or whether the user will have to set some parameters in build.xml. This affects identity and security settings. == Test Runner and Identity Setting == @@ -28,6 +33,10 @@ This does not support easy switching the identity setting. The idea is to keep the enhanced class files for application and datastore identity in separate directories together with their metadata. During JDOTCK setup we would enhance for both identity types and keep the result in separate directories. Then configuring the identity setting for a test class (or a list of test classes) would be setting the correct classpath for the test run. -Open questions: - *How about nondurable identity? - *What happens if the JDO implementation under test does support allication or datastore identity? Would it help to have a build property that specifies with identity types are supported? +== Lack of support for application or datastore identity == + +The user will be required to configure the build to specify the implementation's support for identity types. This will prevent test failure due to lack of support. + += Open Questions = + *What about nondurable identity? Currently, nondurable identity is not supported by the RI, so we cannot verify that the TCK runs properly with it. This situation is not likely to change with JDO 2.0. + *Can the build process read parameters from the configuration file? See section "Role of build.xml" on this page.
