curcuru 00/12/06 12:52:04
Modified: test/java/xdocs/sources/tests getstarted.xml overview.xml
run.xml submit.xml
Log:
Editing pass courtesy of [dleslie]; plus minor updates for new tests
Revision Changes Path
1.2 +18 -18 xml-xalan/test/java/xdocs/sources/tests/getstarted.xml
Index: getstarted.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/getstarted.xml,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- getstarted.xml 2000/11/21 19:55:00 1.1
+++ getstarted.xml 2000/12/06 20:51:54 1.2
@@ -15,12 +15,12 @@
if you are testing a Xalan-C build.</note>
<p>To use the tests, you will need both a working build of Xalan-J
as well as the sources for the tests themselves.
- </p><p>To download Xalan builds, see either:
+ </p><p>To download Xalan builds, see the:
<jump href="http://xml.apache.org/xalan-j/dist/";>Xalan-J download
page</jump> or the
<jump href="http://xml.apache.org/xalan-c/dist/";>Xalan-C download
page</jump>
- </p><p>To get the test sources, you can either:
- <br/>Checkout the xml-xalan\test repository <jump
href="http://xml.apache.org/cvs.html";>directly from CVS</jump>
- (read-only access is available to all, not just committers), or:
+ </p><p>To get the test sources, do either of the following:
+ <br/>Check out the xml-xalan\test repository <jump
href="http://xml.apache.org/cvs.html";>directly from CVS</jump>
+ (<jump href="http://xml.apache.org/cvs.html";>read-only access</jump> is
available to all), or:
<br/><br/>Get the latest <jump
href="http://xml.apache.org/from-cvs/xml-xalan/";>dev-snapshot</jump>:
these are .tar.gz files of the entire xml-xalan repository, including
both the
development sources and the test sources (they are just source, however,
and are not prebuilt).
@@ -30,30 +30,30 @@
<anchor name="how-to-build"/>
<s2 title="Building the Tests">
- <p>(This builds both the test harness/framework/etc. and the specific
Java API test files)
- Works the same for DOS/Windows or UNIXes; in most cases *.sh files are
+ <p>This builds both the test harness/framework/etc. and the specific
Java API test files.
+ It works similarly for DOS/Windows or your flavor of UNIX; in most cases
*.sh files are
provided to match the *.bat files used here.</p>
- <p>1: Checkout the whole xml-xalan repository locally
+ <p>1: Use CVS to check out the whole xml-xalan repository locally.
<br/><code>e:\builds\xml-xalan</code></p>
<p>2: Set an environment variable JARDIR to point to directory where you
have all applicable jars.
- Be sure that you delete the testxsl.jar there, since you want to
rebuild a new one!
- <br/>Required: xalan.jar and bsf.jar from a Xalan-J build;
+ Delete testxsl.jar if present there.
+ <br/>Required: at least xalan.jar and bsf.jar from a Xalan-J build;
xerces.jar from the <ref>same</ref> Xalan-J build (or your parser's
JAXP-compatible jar instead of xerces.jar);
- and js.jar (see <jump
href="http://xml.apache.org/xalan-j/extensions.html#supported-lang";>Xalan-J doc
for information</jump>)
+ and js.jar (see <jump
href="http://xml.apache.org/xalan-j/extensions.html#supported-lang";>Xalan-J doc
for information on js.jar</jump>)
<br/><code>set JARDIR=e:\builds\myjars</code>
</p>
- <p>3: Copy all product jars to the aforementioned JARDIR, obviously</p>
- <p>4: cd to the test directory and be safe: cleanup the tests first</p>
+ <p>3: Copy all product jars to the JARDIR</p>
+ <p>4: cd to the test directory and cleanup the tests.</p>
<p><code>cd e:\builds\xml-xalan\test\java</code>
<br/><code>build.bat clean</code>
</p>
- <p>5: Build the tests appropriate to your product
+ <p>5: Build the tests appropriate to your product.
<br/>Xalan-J 1.x:
<br/><code>build.bat package.xalan</code>
<br/>Xalan-J 2.x:
- <br/><code>build.bat package.trax</code>
+ <br/><code>build.bat package.xalan2</code> (<ref>or package.trax</ref>)
</p>
- <p>This will create <code>build\testxsl.jar</code>, which you should
manually copy into your JARDIR set above before running any tests.</p>
+ <p>This will create <code>build\testxsl.jar</code>, which you should
manually copy into JARDIR before you run any tests.</p>
<p>Note that ProcessorWrapper subclasses for XT and SAXON are currently
checked in
both as .java source files and as precompiled .class files - the .class
files are
merely copied into the jar by default, so you don't need the other
processors
@@ -61,7 +61,7 @@
unless you're running tests with those flavors.</p>
<p>Building the <link idref="run"
anchor="how-to-run-c">CConformanceTest</link>
for testing Xalan-C currently is done with <code>build.bat
package.xsl</code>.
- Instructions for building/running other Xalan-C API tests is still to be
written.</p>
+ Instructions for building/running other Xalan-C API tests are still to
be written.</p>
<p>Building the Javadocs for the tests is done by <code>build.bat
javadocs</code>, and
is best done under JDK 1.2.2 or higher - they will build with JDK 1.1.8,
but not
all the links will work properly.</p>
@@ -69,7 +69,7 @@
be done with <code>build.bat docs</code> and must be done under JDK
1.2.2 or higher,
since the Xalan-related stylebook code that we use requires that. Note
also that
building the docs may require a Xalan-J 1.x build, since we use Xalan to
- process the source XML documents into HTML (and we have it setup for the
1.x build
- currently for convenience).</p>
+ process the source XML documents into HTML (and we currently have it
setup
+ for the 1.x build).</p>
</s2>
</s1>
1.3 +60 -39 xml-xalan/test/java/xdocs/sources/tests/overview.xml
Index: overview.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/overview.xml,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- overview.xml 2000/11/21 19:55:00 1.2
+++ overview.xml 2000/12/06 20:51:56 1.3
@@ -9,19 +9,16 @@
<li><link anchor="credits">Credits for the tests</link></li>
</ul>
-<note>The documentation for the tests is still in the process of being
-migrated into Apache, so expect changes and improvements over the next
couple of weeks.
-The doc will also be much better organized once
-I get Don to edit it!</note>
-
<anchor name="purpose"/>
<s2 title="Purpose of these tests">
<p>These tests are provided for Xalan contributors to evaluate the
impact of code changes.
- Run the tests on the unchanged code, make the change and rebuild, then
run the tests again
+ Run the tests on the unchanged code, make the change and rebuild. Then
run the tests again
and compare the results. Results should also be compared to the files in
the "-gold"
directory trees. Even though not all tests have "gold" files, it's still
valuable to run
- the tests before and after a code change. That way you can at least
ensure yourself that
- your changes didn't cause any regressions to the code before checking
in.</p>
+ the tests before and after a code change. That way you can at least
ensure that
+ your changes didn't cause any regressions to the code before you check
your
+ changes in. In the future, we hope to provide a tool to compare test
results from
+ one run to another to make this process even simpler.</p>
</s2>
<anchor name="dir-map"/>
@@ -30,14 +27,15 @@
<label>Brief overview of directory structure:</label>
<label><code>xml-xalan/test</code></label>
<item>Top level dir for all Xalan versions/products tests</item>
+ <label></label>
<label><code>xml-xalan/test/java</code></label>
<item>Top level for Java test automation</item>
<label><code>xml-xalan/test/java/bin</code></label>
<item>Java test automation dependencies (includes
<jump href="http://jakarta.apache.org/ant/index.html";>Ant
1.2</jump>)</item>
<label><code>xml-xalan/test/java/src</code></label>
- <item>Java test automation source tree - this includes both
- a generic testing framework as well as both specific API tests for
parts of Xalan
+ <item>Java test automation source tree - this includes
+ a generic testing framework as well as specific API tests for parts of
Xalan
and several test drivers for testing conformance / performance / etc.
over a large
number of xsl test stylesheets.
<br/>Primary packages are:<br/>
@@ -47,19 +45,23 @@
<jump
href="apidocs/org/apache/qetest/trax/dom/package-summary.html">org.apache.qetest.trax.dom</jump><br/>
<jump
href="apidocs/org/apache/qetest/trax/stream/package-summary.html">org.apache.qetest.trax.stream</jump><br/>
<jump
href="apidocs/org/apache/qetest/xalanj1/package-summary.html">org.apache.qetest.xalanj1</jump><br/>
- </item>
+ <br/></item>
<label><code>xml-xalan/test/tests</code></label><item>Top level for
XSLT stylesheet trees</item>
<label><code>xml-xalan/test/tests/contrib</code></label><item>Directory tree of
user-contributed stylesheets</item>
<label><code>xml-xalan/test/tests/contrib-gold</code></label><item>Directory
tree of user-contributed stylesheets gold
- output reference files (this tree should mirror the structure of
contrib)</item>
+ output reference files (this tree should mirror the structure of
contrib)<br/></item>
<label><code>xml-xalan/test/tests/api</code></label><item>Directory
tree for stylesheets used in Java API tests</item>
<label><code>xml-xalan/test/tests/api/trax</code></label><item>Stylesheets used
in Java API tests in
<jump
href="apidocs/org/apache/qetest/trax/package-summary.html">org.apache.qetest.trax</jump></item>
-
<label><code>xml-xalan/test/tests/api-gold</code></label><item>Matching
Directory tree of gold files for Java API tests</item>
+
<label><code>xml-xalan/test/tests/api/trax/dom</code></label><item>Stylesheets
used in Java API tests in
+ <jump
href="apidocs/org/apache/qetest/trax/dom/package-summary.html">org.apache.qetest.trax.dom</jump></item>
+ <label></label><item>etc. - often the directory tree in the stylesheet
area
+ will match the Java sources directory/package tree.</item>
+
<label><code>xml-xalan/test/tests/api-gold</code></label><item>Matching
Directory tree of gold files for Java API tests<br/></item>
<label><code>xml-xalan/test/tests/extension</code></label><item>Directory tree
for stylesheets used in Xalan-specific extension tests</item>
<label><code>xml-xalan/test/tests/extension/xalanj1</code></label><item>Xalan-J
1.x version-specific extension tests</item>
<label><code>xml-xalan/test/tests/extension/xalanj2</code></label><item>Xalan-J
2.x version-specific extension tests</item>
-
<label><code>xml-xalan/test/tests/extension-gold</code></label><item>Matching
Directory tree of gold files for Xalan-specific extension tests</item>
+
<label><code>xml-xalan/test/tests/extension-gold</code></label><item>Matching
Directory tree of gold files for Xalan-specific extension tests<br/></item>
<label><code>xml-xalan/test/tests/...</code></label><item>More
stylesheet trees of testfiles to be added!
(namely: tests/conf - Conformance tests to the XSLT spec)</item>
<label><code>xml-xalan/test/tests/...-gold</code></label><item></item>
@@ -68,21 +70,34 @@
<anchor name="test-map"/>
<s2 title="Listing of Java tests and drivers">
-<p>Java Test Drivers (primarily to execute directory trees full of or
-explicit fileLists of xsl/xml file pairs to produce output files, which
-are then verified against a corresponding '-gold' tree)</p>
+<p>Java Test Drivers</p>
+<p>A Java Test Driver executes a test for each xml/xsl file pair in
+the specified directory tree or each pair in the specified fileList.
+For each test, the driver generates an output file which may be
+compared to the corresponding file in the "gold" tree. The Test
+Drivers supply the code to determine which files to operate on
+and the order to do those operations (including logging performance
+and other data out).</p>
+<p>The Test Drivers rely on ProcessorWrapper
+subclasses to perform the actual test of processing or transformation
+of the xml.xsl file pair into the output file. We can then plug
+in different ProcessorWrapper "flavors" easily. Different
+ProcessorWrappers can process or transform in various ways, like
+using DOM trees, SAX events, or input/output streams.</p>
<gloss>
<label>org.apache.qetest.xsl.ConformanceTest</label>
<item>basic test driver, either takes an
inputDir to iterate over (using ConformanceDirRules/ConformanceFileRules),
-or an explicit -fileList. Processes all files using a specific
+or an explicit fileList. Processes all files using a specific
-flavor of a <jump
href="apidocs/org/apache/qetest/xslwrapper/ProcessorWrapper.html">ProcessorWrapper</jump>,
so identical test runs can be done
-using different processors (eg. -flavor xalan = XalanWrapper = Xalan-J 1.x;
+using different processors (e.g., -flavor xalan = XalanWrapper = Xalan-J
1.x;
-flavor trax = TraxWrapper = Trax interface using streams by default;
-flavor trax.d2d = TraxWrapper = Trax interface using DOMs)
<br/>Actually, 'ConformanceTest' is a bad name - this is a generic
stylesheet
-test driver that can be used to run any sort of stylesheet tests, not
-just conformance tests - this is a common bit of confusion. Alternate names
welcomed.
+test driver that can be used to run using any matching xml source
+and xsl stylesheet file names, as specified in the ConformanceFileRules,
+not just conformance tests.
+<br/>Suggestions for alternate names welcomed!
'StylesheetTestDriver' perhaps?</item>
<label>org.apache.qetest.xsl.PerformanceTest</label><item>essentially the
same as ConformanceTest,
but provides additional timing/memory output, as well as an -iterations
@@ -102,52 +117,56 @@
<item><ref>utility:</ref> wraps javax.xml.transform.ErrorListener, and logs
info</item>
<label>LoggingURIResolver.java</label>
<item><ref>utility:</ref> wraps javax.xml.transform.URIResolver, and logs
info</item>
-<label>TransformerAPITest.java (sc)</label>
+<label>ExamplesTest.java</label>
+<item>A testing version of samples/trax/Examples.java, a sample file
+provided in Xalan-J 2.x showing various uses of the TRAX or
+javax.xml.transform API to process stylesheets.</item>
+<label>TransformerAPITest.java</label>
<item>API coverage tests for javax.xml.transform.Transformer</item>
-<label>TransformerFactoryAPITest.java (sc)</label>
+<label>TransformerFactoryAPITest.java</label>
<item>API coverage tests for javax.xml.transform.TransformerFactory</item>
-<label>TemplatesAPITest.java (sc)</label>
+<label>TemplatesAPITest.java</label>
<item>API coverage tests for javax.xml.transform.Templates</item>
-<label>ResultAPITest.java (sc)</label>
+<label>ResultAPITest.java</label>
<item>API test for Result class - may be obsolete, should
have separate tests for SAXResult, DOMResult, StreamResult</item>
-<label>ProcessorAPITest.java (sc)</label>
+<label>ProcessorAPITest.java</label>
<item>API test: obsolete: from a previous version of TRAX</item>
-<label>TestThreads.java (sc)</label>
+<label>TestThreads.java</label>
<item>MANUALLY executed test for running multiple threads
and transforming multiple stylesheets simultaneously.</item>
</gloss>
<p>All in subpackages of: org.apache.qetest.trax</p>
<gloss>
-<label>stream.StreamSourceAPITest.java (sc)</label>
+<label>stream.StreamSourceAPITest.java</label>
<item>API coverage tests for javax.xml.transform.stream.StreamSource (mostly
done)</item>
-<label>stream.StreamResultAPITest.java (sc)</label>
+<label>stream.StreamResultAPITest.java</label>
<item>API coverage tests for javax.xml.transform.stream.StreamResult (mostly
done)</item>
-<label>dom.DOMSourceAPITest.java (sc)</label>
+<label>dom.DOMSourceAPITest.java</label>
<item>API coverage tests for javax.xml.transform.dom.DOMSource (mostly
done)</item>
-<label>dom.DOMResultAPITest.java (sc)</label>
+<label>dom.DOMResultAPITest.java</label>
<item>API coverage tests for javax.xml.transform.dom.DOMResult (mostly
done)</item>
-<label>dom.DOMLocatorAPITest.java (sc)</label>
+<label>dom.DOMLocatorAPITest.java</label>
<item>API coverage tests for javax.xml.transform.dom.DOMLocator
(@todo)</item>
-<label>sax.SAXSourceAPITest.java (km/rm)</label>
+<label>sax.SAXSourceAPITest.java (to be done)</label>
<item>API coverage tests for javax.xml.transform.sax.SAXSource (@todo)</item>
-<label>sax.SAXResultAPITest.java (km/rm)</label>
+<label>sax.SAXResultAPITest.java (to be done)</label>
<item>API coverage tests for javax.xml.transform.sax.SAXResult (@todo)</item>
-<label>sax.SAXTransformerFactoryAPITest.java (km/rm)</label>
+<label>sax.SAXTransformerFactoryAPITest.java (to be done)</label>
<item>API coverage tests for javax.xml.transform.sax.SAXTransformerFactory
(@todo)</item>
-<label>sax.TemplatesHandlerAPITest.java (km/rm)</label>
+<label>sax.TemplatesHandlerAPITest.java (to be done)</label>
<item>API coverage tests for javax.xml.transform.sax.TemplatesHandler
(@todo)</item>
-<label>sax.TransformerHandlerAPITest.java (km/rm)</label>
+<label>sax.TransformerHandlerAPITest.java (to be done)</label>
<item>API coverage tests for javax.xml.transform.sax.TransformerHandler
(@todo)</item>
</gloss>
<p>Java API tests for Xalan-J 1.x.</p>
<gloss>
<label>org.apache.qetest.xalanj1.ParamTest</label>
-<item>setting parameters and verifies
+<item>setting stylesheet parameters and verifies
they're used correctly in a transform/process</item>
</gloss>
@@ -157,7 +176,9 @@
yet had time to confirm the correct output. It's still useful to run these
tests (although the ConformanceTest driver will report an AMBG or
'Ambiguous'
result) because you can still see if the output looks basically correct, and
-compare the output to previous test runs before your code changes, etc.
+compare the output to previous test runs before you submit your code
changes, etc.</p>
+<p>The tests have several different types of results beyond just pass or
fail,
+which are documented in <jump
href="apidocs/org/apache/qetest/Logger.html#PASS_RESULT">org.apache.qetest.Logger</jump>.
At Lotus/IBM, we have some additional test cases that we can't give away, so
a
patch may pass all these tests and fail others, but those cases should be
rare.</p>
1.2 +15 -12 xml-xalan/test/java/xdocs/sources/tests/run.xml
Index: run.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/run.xml,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- run.xml 2000/11/21 19:55:00 1.1
+++ run.xml 2000/12/06 20:51:58 1.2
@@ -12,11 +12,12 @@
<anchor name="how-to-run"/>
<s2 title="How-to: Run tests">
- <p>1: <link idref="getstarted" anchor="how-to-build">Build a fresh copy
of testxsl.jar</link>,
- including setting the environment variable JARDIR and putting all the
needed .jar files
- there, including testxsl.jar<br/></p>
- <p>2: cd xml-xalan\test<br/></p>
- <p>3: (<ref>either</ref>) Run Any of the 'convenience' batch files
provided:<br/>
+ <p>1: <link idref="getstarted" anchor="how-to-build">Build a fresh copy
of testxsl.jar.</link>
+ <br/></p>
+ <p>2: Set the JARDIR environment variable, and put
<code>testxsl.jar</code> and the other required JAR files in the JARDIR
directory.<br/></p>
+ <p>3: cd xml-xalan\test<br/></p>
+ <p>4: Run any of the convenience batch files (see below) or run java.exe
with the desired test class.<br/></p>
+ <p>
<code>traxapitest.bat TRAXAPITestClassName [<link
anchor="test-options">options</link>]</code>
<br/>(runs TRAX interface tests with Xalan-J 2.x, equivalent to
<code>runtest trax.TRAXAPITestClassName -load APITest.properties
[<link anchor="test-options">options</link>]</code><br/><br/>
@@ -31,12 +32,12 @@
need to provide the last part of the end_pkg name as well as the
classname - as well
as providing any extra arguments for the test too.<br/>
</p>
- <p>3: (<ref>or</ref>) Run java.exe and the desired test class
yourself:<br/></p>
+ <p>Alternately: Run java.exe and the desired test class
yourself:<br/></p>
<p>Note: the above batch and shell files, and the JARDIR environment
variable,
are simply conveniences for running the tests. The Java API tests and
test drivers can
also be executed by hand, for those who wish to manage their own
classpaths and/or
simply pass all arguments to the tests on the command line, etc.</p>
- <p>Sorry! We don't have .sh equivalents for all the convenience .bat
files -
+ <p>Sorry! We don't have .sh equivalents for the convenience .bat files -
submissions of ports of these files are welcomed!</p>
<note>Running tests with alternate JAXP parsers: all
org.apache.qetest.trax.*
tests can be run with Xalan-J 2.x and any JAXP 1.1 compatible parser,
like
@@ -52,7 +53,7 @@
output results file is easier to deal with. The basic format is
fairly simple and you can certainly read it as-is.</p>
<p>To 'pretty-print' results or produce reports, please use the
- viewResults.xsl stylesheet and associated batch/shell files, like so:
+ viewResults.xsl stylesheet and associated batch/shell files, like
so:<br/>
<code>cd \xml-xalan\test</code><br/><br/>
<code>runTest.bat MyTest -options blah <link
anchor="test-options-logfile">-logFile
results\MyResults.xml</link></code><br/><br/>
<code>viewResults.bat results\MyResults.xml
results\MyPrettyResults.html [options]</code><br/><br/>
@@ -70,12 +71,16 @@
or simply from the command line (which overrides the properties file)
like:<br/>
<code> TestName -arg1 value1 -arg2 -arg3
value3</code><br/><br/></p>
<p>To see all options, call a test with an illegal argument to force
it
- to print out it's .usage().</p>
- <p>For another description of options, see
<code>xml-xalan\test\ContribTest.properties</code>,
+ to print out it's .usage(). You may mix setting options from a
properties
+ file and from the command line; command line options will take
precedence.</p>
+ <p>For another description of options, see
<br/><code>xml-xalan\test\ContribTest.properties</code>,<br/>
which describes most of them as used in the context of running the
ConformanceTest driver
over the xml-xalan\tests\contrib tree of stylesheets. Simply change
inputDir and goldDir
to run over a different set of files (like a conformance test suite,
which we hope to
add soon).</p>
+ <note>Path-like options set in a properties file must use the local
system's
+ File.separator character, and backslashes \ must be escaped \\. The
checked in
+ copy of ContribTest.properties is for a Windows platform.</note>
<p>Quick list of options</p>
<anchor name="test-options-logfile"/>
<gloss>
@@ -90,8 +95,6 @@
<label>-goldDir <ref>path/to/gold</ref></label>
<item>path to a directory tree of reference output - this tree
should be
a parallel structure to the inputDir tree</item>
- <label>(paths should be appropriate to the platform you're on,
- and must be \\escaped\\properly if in a .properties file)</label>
<label>-category <ref>dirName</ref></label>
<item>only run this single named subdir within inputDir</item>
<label>-excludes <ref>'test1.xsl;test2.xsl'</ref></label>
1.2 +1 -1 xml-xalan/test/java/xdocs/sources/tests/submit.xml
Index: submit.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/submit.xml,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- submit.xml 2000/11/21 19:55:00 1.1
+++ submit.xml 2000/12/06 20:51:58 1.2
@@ -46,7 +46,7 @@
information about this project, visit
<jump
href="http://www.oasis-open.org/committees/xslt/index.html";>http://www.oasis-open.org/committees/xslt/index.html</jump></p>
-<p>The OASIS project will combine test cases from different sources and
provide a way to
+<p>The OASIS project will combine test cases from different sources and will
provide a way to
customize the set of tests according to design decisions made for each
processor
under test. We expect that when the OASIS test system is delivered, it will
supplant
most or all of the "conf" group provided here. Conformance tests are
designed as
