curcuru 00/11/21 11:55:00
Modified: test/java/xdocs/sources xalantest.xml
test/java/xdocs/sources/tests overview.xml
Added: test/java/xdocs/sources/tests design.xml getstarted.xml
run.xml submit.xml
Log:
Implement Xalan-style docs for all the tests
Revision Changes Path
1.2 +14 -10 xml-xalan/test/java/xdocs/sources/xalantest.xml
Index: xalantest.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/xalantest.xml,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- xalantest.xml 2000/11/10 22:37:54 1.1
+++ xalantest.xml 2000/11/21 19:54:59 1.2
@@ -9,29 +9,33 @@
label="Overview"
source="tests/overview.xml"/>
- <!--document id="getstarted"
+ <document id="getstarted"
label="Getting Started"
- source="tests/getstarted.xml"/-->
+ source="tests/getstarted.xml"/>
<separator/>
- <external href="apidocs/index.html" label="Java API"/>
+ <external href="apidocs/index.html" label="Java API"/>
- <!--separator/>
-
+ <separator/>
+
<document id="run"
label="Running Tests"
source="tests/run.xml"/>
<document id="submit"
- label="Submitting New Tests"
+ label="Writing New Tests"
source="tests/submit.xml"/>
- <separator/>
-
<document id="design"
- label="Testing Design"
- source="tests/design.xml"/-->
+ label="Test Standards"
+ source="tests/design.xml"/>
+
+ <separator/>
+ <!-- How do you create a separator or item with text, but no link? -->
+ <external href="http://xml.apache.org/xalan-j"; label="Xalan-J 2.x"/>
+ <external href="http://xml.apache.org/xalan"; label="Xalan-J 1.x"/>
+ <external href="http://xml.apache.org/xalan-c"; label="Xalan-C 1.x"/>
</book>
1.2 +171 -2 xml-xalan/test/java/xdocs/sources/tests/overview.xml
Index: overview.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/overview.xml,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- overview.xml 2000/11/10 22:37:54 1.1
+++ overview.xml 2000/11/21 19:55:00 1.2
@@ -2,8 +2,177 @@
<!DOCTYPE s1 SYSTEM "sbk:/style/dtd/document.dtd">
<s1 title="Overview">
+<ul>
+<li><link anchor="purpose">Purpose of these tests</link></li>
+<li><link anchor="dir-map">Directory Structure</link></li>
+<li><link anchor="test-map">Listing of Java tests and drivers</link></li>
+<li><link anchor="credits">Credits for the tests</link></li>
+</ul>
+
+<note>The documentation for the tests is still in the process of being
+migrated into Apache, so expect changes and improvements over the next
couple of weeks.
+The doc will also be much better organized once
+I get Don to edit it!</note>
+
+ <anchor name="purpose"/>
+ <s2 title="Purpose of these tests">
+ <p>These tests are provided for Xalan contributors to evaluate the
impact of code changes.
+ Run the tests on the unchanged code, make the change and rebuild, then
run the tests again
+ and compare the results. Results should also be compared to the files in
the "-gold"
+ directory trees. Even though not all tests have "gold" files, it's still
valuable to run
+ the tests before and after a code change. That way you can at least
ensure yourself that
+ your changes didn't cause any regressions to the code before checking
in.</p>
+ </s2>
- <s2 title="Introduction to Xalan Testing">
+ <anchor name="dir-map"/>
+ <s2 title="Directory Structure">
+ <gloss>
+ <label>Brief overview of directory structure:</label>
+ <label><code>xml-xalan/test</code></label>
+ <item>Top level dir for all Xalan versions/products tests</item>
+ <label><code>xml-xalan/test/java</code></label>
+ <item>Top level for Java test automation</item>
+ <label><code>xml-xalan/test/java/bin</code></label>
+ <item>Java test automation dependencies (includes
+ <jump href="http://jakarta.apache.org/ant/index.html";>Ant
1.2</jump>)</item>
+ <label><code>xml-xalan/test/java/src</code></label>
+ <item>Java test automation source tree - this includes both
+ a generic testing framework as well as both specific API tests for
parts of Xalan
+ and several test drivers for testing conformance / performance / etc.
over a large
+ number of xsl test stylesheets.
+ <br/>Primary packages are:<br/>
+ <jump
href="apidocs/org/apache/qetest/package-summary.html">org.apache.qetest</jump><br/>
+ <jump
href="apidocs/org/apache/qetest/xsl/package-summary.html">org.apache.qetest.xsl</jump><br/>
+ <jump
href="apidocs/org/apache/qetest/trax/package-summary.html">org.apache.qetest.trax</jump><br/>
+ <jump
href="apidocs/org/apache/qetest/trax/dom/package-summary.html">org.apache.qetest.trax.dom</jump><br/>
+ <jump
href="apidocs/org/apache/qetest/trax/stream/package-summary.html">org.apache.qetest.trax.stream</jump><br/>
+ <jump
href="apidocs/org/apache/qetest/xalanj1/package-summary.html">org.apache.qetest.xalanj1</jump><br/>
+ </item>
+ <label><code>xml-xalan/test/tests</code></label><item>Top level for
XSLT stylesheet trees</item>
+
<label><code>xml-xalan/test/tests/contrib</code></label><item>Directory tree of
user-contributed stylesheets</item>
+
<label><code>xml-xalan/test/tests/contrib-gold</code></label><item>Directory
tree of user-contributed stylesheets gold
+ output reference files (this tree should mirror the structure of
contrib)</item>
+ <label><code>xml-xalan/test/tests/api</code></label><item>Directory
tree for stylesheets used in Java API tests</item>
+
<label><code>xml-xalan/test/tests/api/trax</code></label><item>Stylesheets used
in Java API tests in
+ <jump
href="apidocs/org/apache/qetest/trax/package-summary.html">org.apache.qetest.trax</jump></item>
+
<label><code>xml-xalan/test/tests/api-gold</code></label><item>Matching
Directory tree of gold files for Java API tests</item>
+
<label><code>xml-xalan/test/tests/extension</code></label><item>Directory tree
for stylesheets used in Xalan-specific extension tests</item>
+
<label><code>xml-xalan/test/tests/extension/xalanj1</code></label><item>Xalan-J
1.x version-specific extension tests</item>
+
<label><code>xml-xalan/test/tests/extension/xalanj2</code></label><item>Xalan-J
2.x version-specific extension tests</item>
+
<label><code>xml-xalan/test/tests/extension-gold</code></label><item>Matching
Directory tree of gold files for Xalan-specific extension tests</item>
+ <label><code>xml-xalan/test/tests/...</code></label><item>More
stylesheet trees of testfiles to be added!
+ (namely: tests/conf - Conformance tests to the XSLT spec)</item>
+ <label><code>xml-xalan/test/tests/...-gold</code></label><item></item>
+ </gloss>
+ </s2>
+
+ <anchor name="test-map"/>
+ <s2 title="Listing of Java tests and drivers">
+<p>Java Test Drivers (primarily to execute directory trees full of or
+explicit fileLists of xsl/xml file pairs to produce output files, which
+are then verified against a corresponding '-gold' tree)</p>
+<gloss>
+<label>org.apache.qetest.xsl.ConformanceTest</label>
+<item>basic test driver, either takes an
+inputDir to iterate over (using ConformanceDirRules/ConformanceFileRules),
+or an explicit -fileList. Processes all files using a specific
+-flavor of a <jump
href="apidocs/org/apache/qetest/xslwrapper/ProcessorWrapper.html">ProcessorWrapper</jump>,
so identical test runs can be done
+using different processors (eg. -flavor xalan = XalanWrapper = Xalan-J 1.x;
+-flavor trax = TraxWrapper = Trax interface using streams by default;
+-flavor trax.d2d = TraxWrapper = Trax interface using DOMs)
+<br/>Actually, 'ConformanceTest' is a bad name - this is a generic
stylesheet
+test driver that can be used to run any sort of stylesheet tests, not
+just conformance tests - this is a common bit of confusion. Alternate names
welcomed.
+'StylesheetTestDriver' perhaps?</item>
+<label>org.apache.qetest.xsl.PerformanceTest</label><item>essentially the
same as ConformanceTest,
+but provides additional timing/memory output, as well as an -iterations
+parameter to iterate over each file a bunch of times to get average timing
data
+</item>
+<label>org.apache.qetest.xsl.<link idref="run"
anchor="how-to-run-c">CConformanceTest</link></label>
+<item>essentially the same as ConformanceTest, but for Xalan-C.</item>
+</gloss>
+
+<p>Java API tests for the TRAX (or javax.xml.transform) interface, that
+Xalan-J 2.x implements.<br/>
+All in package: org.apache.qetest.trax</p>
+<gloss>
+<label>REPLACE_template_for_new_tests.java</label>
+<item>a template for creating new TRAX API tests, see <link idref="submit"
anchor="write-API-tests">Submitting New Tests</link></item>
+<label>LoggingErrorListener.java</label>
+<item><ref>utility:</ref> wraps javax.xml.transform.ErrorListener, and logs
info</item>
+<label>LoggingURIResolver.java</label>
+<item><ref>utility:</ref> wraps javax.xml.transform.URIResolver, and logs
info</item>
+<label>TransformerAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.Transformer</item>
+<label>TransformerFactoryAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.TransformerFactory</item>
+<label>TemplatesAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.Templates</item>
+<label>ResultAPITest.java (sc)</label>
+<item>API test for Result class - may be obsolete, should
+have separate tests for SAXResult, DOMResult, StreamResult</item>
+<label>ProcessorAPITest.java (sc)</label>
+<item>API test: obsolete: from a previous version of TRAX</item>
+<label>TestThreads.java (sc)</label>
+<item>MANUALLY executed test for running multiple threads
+and transforming multiple stylesheets simultaneously.</item>
+</gloss>
+
+<p>All in subpackages of: org.apache.qetest.trax</p>
+<gloss>
+<label>stream.StreamSourceAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.stream.StreamSource (mostly
done)</item>
+<label>stream.StreamResultAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.stream.StreamResult (mostly
done)</item>
+
+<label>dom.DOMSourceAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.dom.DOMSource (mostly
done)</item>
+<label>dom.DOMResultAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.dom.DOMResult (mostly
done)</item>
+<label>dom.DOMLocatorAPITest.java (sc)</label>
+<item>API coverage tests for javax.xml.transform.dom.DOMLocator
(@todo)</item>
+
+<label>sax.SAXSourceAPITest.java (km/rm)</label>
+<item>API coverage tests for javax.xml.transform.sax.SAXSource (@todo)</item>
+<label>sax.SAXResultAPITest.java (km/rm)</label>
+<item>API coverage tests for javax.xml.transform.sax.SAXResult (@todo)</item>
+<label>sax.SAXTransformerFactoryAPITest.java (km/rm)</label>
+<item>API coverage tests for javax.xml.transform.sax.SAXTransformerFactory
(@todo)</item>
+<label>sax.TemplatesHandlerAPITest.java (km/rm)</label>
+<item>API coverage tests for javax.xml.transform.sax.TemplatesHandler
(@todo)</item>
+<label>sax.TransformerHandlerAPITest.java (km/rm)</label>
+<item>API coverage tests for javax.xml.transform.sax.TransformerHandler
(@todo)</item>
+</gloss>
+
+<p>Java API tests for Xalan-J 1.x.</p>
+<gloss>
+<label>org.apache.qetest.xalanj1.ParamTest</label>
+<item>setting parameters and verifies
+they're used correctly in a transform/process</item>
+</gloss>
+
+<p>Some tests are ones that Xalan has not passed to date, but we know the
+correct ("gold") result by analysis or by trying the test on other
processors.
+A number of tests may also be missing matching "gold" files, if we haven't
+yet had time to confirm the correct output. It's still useful to run these
+tests (although the ConformanceTest driver will report an AMBG or
'Ambiguous'
+result) because you can still see if the output looks basically correct, and
+compare the output to previous test runs before your code changes, etc.
+At Lotus/IBM, we have some additional test cases that we can't give away, so
a
+patch may pass all these tests and fail others, but those cases should be
rare.</p>
+
+ </s2>
+
+ <anchor name="credits"/>
+ <s2 title="Credits for the tests">
+ <ul>
+ <li><jump href="mailto:[EMAIL PROTECTED]">Shane Curcuru</jump></li>
+ <li><jump href="mailto:[EMAIL PROTECTED]">Paul Dick</jump></li>
+ <li><jump href="mailto:[EMAIL PROTECTED]">David Marston</jump></li>
+ <li><jump href="mailto:[EMAIL PROTECTED]">Gary L Peskin</jump></li>
+ <li>Many other <jump
href="http://xml.apache.org/mail.html";>xalan-dev</jump> subscribers</li>
+ <li>Many other helpers who we still need to credit! Sorry!</li>
+ </ul>
+ </s2>
- </s2>
</s1>
1.1 xml-xalan/test/java/xdocs/sources/tests/design.xml
Index: design.xml
===================================================================
<?xml version="1.0" standalone="no"?>
<!DOCTYPE s1 SYSTEM "sbk:/style/dtd/document.dtd">
<s1 title="Testing Design/Standards">
<ul>
<li><link anchor="standards-api-tests">Standards for API Tests</link></li>
<li><link anchor="standards-xsl-tests">Standards for Stylesheet
Tests</link></li>
</ul>
<anchor name="standards-api-tests"/>
<s2 title="Standards for API Tests">
<p>In progress. Both the overall Java testing framework, the test
drivers,
and the specific API tests have a number of design decisions detailed
in the javadoc
<jump href="apidocs/org/apache/qetest/package-summary.html">here</jump>
and
<jump
href="apidocs/org/apache/qetest/xsl/package-summary.html">here</jump>.</p>
<p>Please: if you plan to submit Java API tests, use the existing
framework
as <link idref="submit" anchor="write-API-tests">described</link>.</p>
<p>Contact <jump href="[EMAIL PROTECTED]">[EMAIL PROTECTED]</jump> if
you'd
like to help in the Xalan-C API testing effort.</p>
</s2>
<anchor name="standards-xsl-tests"/>
<s2 title="Standards for Stylesheet Tests">
<p>In progress. See the <link idref="submit"
anchor="write-xsl-tests">discussion about OASIS</link> for an overview.</p>
</s2>
</s1>
1.1 xml-xalan/test/java/xdocs/sources/tests/getstarted.xml
Index: getstarted.xml
===================================================================
<?xml version="1.0" standalone="no"?>
<!DOCTYPE s1 SYSTEM "sbk:/style/dtd/document.dtd">
<s1 title="Getting Started">
<ul>
<li><link anchor="downloading">Downloading the code</link></li>
<li><link anchor="how-to-build">Building the Tests</link></li>
</ul>
<anchor name="downloading"/>
<s2 title="Downloading the code">
<note>Since these tests are primarily to help developers test their
changes to Xalan source code, we don't currently provide prebuilt
builds of the test code. Most tests also require Xalan-J, even
if you are testing a Xalan-C build.</note>
<p>To use the tests, you will need both a working build of Xalan-J
as well as the sources for the tests themselves.
</p><p>To download Xalan builds, see either:
<jump href="http://xml.apache.org/xalan-j/dist/";>Xalan-J download
page</jump> or the
<jump href="http://xml.apache.org/xalan-c/dist/";>Xalan-C download
page</jump>
</p><p>To get the test sources, you can either:
<br/>Checkout the xml-xalan\test repository <jump
href="http://xml.apache.org/cvs.html";>directly from CVS</jump>
(read-only access is available to all, not just committers), or:
<br/><br/>Get the latest <jump
href="http://xml.apache.org/from-cvs/xml-xalan/";>dev-snapshot</jump>:
these are .tar.gz files of the entire xml-xalan repository, including
both the
development sources and the test sources (they are just source, however,
and are not prebuilt).
</p>
</s2>
<anchor name="how-to-build"/>
<s2 title="Building the Tests">
<p>(This builds both the test harness/framework/etc. and the specific
Java API test files)
Works the same for DOS/Windows or UNIXes; in most cases *.sh files are
provided to match the *.bat files used here.</p>
<p>1: Checkout the whole xml-xalan repository locally
<br/><code>e:\builds\xml-xalan</code></p>
<p>2: Set an environment variable JARDIR to point to directory where you
have all applicable jars.
Be sure that you delete the testxsl.jar there, since you want to
rebuild a new one!
<br/>Required: xalan.jar and bsf.jar from a Xalan-J build;
xerces.jar from the <ref>same</ref> Xalan-J build (or your parser's
JAXP-compatible jar instead of xerces.jar);
and js.jar (see <jump
href="http://xml.apache.org/xalan-j/extensions.html#supported-lang";>Xalan-J doc
for information</jump>)
<br/><code>set JARDIR=e:\builds\myjars</code>
</p>
<p>3: Copy all product jars to the aforementioned JARDIR, obviously</p>
<p>4: cd to the test directory and be safe: cleanup the tests first</p>
<p><code>cd e:\builds\xml-xalan\test\java</code>
<br/><code>build.bat clean</code>
</p>
<p>5: Build the tests appropriate to your product
<br/>Xalan-J 1.x:
<br/><code>build.bat package.xalan</code>
<br/>Xalan-J 2.x:
<br/><code>build.bat package.trax</code>
</p>
<p>This will create <code>build\testxsl.jar</code>, which you should
manually copy into your JARDIR set above before running any tests.</p>
<p>Note that ProcessorWrapper subclasses for XT and SAXON are currently
checked in
both as .java source files and as precompiled .class files - the .class
files are
merely copied into the jar by default, so you don't need the other
processors
on the classpath when building the jar. These classes aren't strictly
necessary
unless you're running tests with those flavors.</p>
<p>Building the <link idref="run"
anchor="how-to-run-c">CConformanceTest</link>
for testing Xalan-C currently is done with <code>build.bat
package.xsl</code>.
Instructions for building/running other Xalan-C API tests is still to be
written.</p>
<p>Building the Javadocs for the tests is done by <code>build.bat
javadocs</code>, and
is best done under JDK 1.2.2 or higher - they will build with JDK 1.1.8,
but not
all the links will work properly.</p>
<p>Building these top-level documents in the xdocs directory can
be done with <code>build.bat docs</code> and must be done under JDK 1.2.2
or higher,
since the Xalan-related stylebook code that we use requires that. Note
also that
building the docs may require a Xalan-J 1.x build, since we use Xalan to
process the source XML documents into HTML (and we have it setup for the
1.x build
currently for convenience).</p>
</s2>
</s1>
1.1 xml-xalan/test/java/xdocs/sources/tests/run.xml
Index: run.xml
===================================================================
<?xml version="1.0" standalone="no"?>
<!DOCTYPE s1 SYSTEM "sbk:/style/dtd/document.dtd">
<s1 title="Running Tests">
<ul>
<li><link anchor="how-to-run">How-to: Run Xalan-J tests</link></li>
<li><link anchor="how-to-view-results">How-to: View Test Results</link></li>
<li><link anchor="test-options">Common Test Options</link></li>
<li><link anchor="how-to-run-c">How-to: Run Xalan-C tests</link></li>
</ul>
<anchor name="how-to-run"/>
<s2 title="How-to: Run tests">
<p>1: <link idref="getstarted" anchor="how-to-build">Build a fresh copy
of testxsl.jar</link>,
including setting the environment variable JARDIR and putting all the
needed .jar files
there, including testxsl.jar<br/></p>
<p>2: cd xml-xalan\test<br/></p>
<p>3: (<ref>either</ref>) Run Any of the 'convenience' batch files
provided:<br/>
<code>traxapitest.bat TRAXAPITestClassName [<link
anchor="test-options">options</link>]</code>
<br/>(runs TRAX interface tests with Xalan-J 2.x, equivalent to
<code>runtest trax.TRAXAPITestClassName -load APITest.properties
[<link anchor="test-options">options</link>]</code><br/><br/>
<code>xalanj1apitest.bat ParamTest [<link
anchor="test-options">options</link>]</code>
<br/>(runs Xalan-J 1.x API tests, equivalent to
<code>runtest xalanj1.ParamTest -load APITest.properties [<link
anchor="test-options">options</link>]</code><br/><br/>
<code>contribtest.bat [<link
anchor="test-options">options</link>]</code>
<br/>(runs ConformanceTest driver over contrib test tree)<br/><br/>
<code>runtest.bat end_pkg.AnyTestClassName [<link
anchor="test-options">options</link>]</code>
<br/>(see batch file for comments) This is a generic way to run any
tests -
it assumes org.apache.qetest as the start of the package; you may
need to provide the last part of the end_pkg name as well as the
classname - as well
as providing any extra arguments for the test too.<br/>
</p>
<p>3: (<ref>or</ref>) Run java.exe and the desired test class
yourself:<br/></p>
<p>Note: the above batch and shell files, and the JARDIR environment
variable,
are simply conveniences for running the tests. The Java API tests and
test drivers can
also be executed by hand, for those who wish to manage their own
classpaths and/or
simply pass all arguments to the tests on the command line, etc.</p>
<p>Sorry! We don't have .sh equivalents for all the convenience .bat
files -
submissions of ports of these files are welcomed!</p>
<note>Running tests with alternate JAXP parsers: all
org.apache.qetest.trax.*
tests can be run with Xalan-J 2.x and any JAXP 1.1 compatible parser,
like
crimson.jar. Be sure to manually set the appropriate system properties
to use
your parser instead of xerces.jar, which is the default for
Xalan-J.</note>
</s2>
<anchor name="how-to-view-results"/>
<s2 title="How-to: View Test Results">
<p>Most tests both send basic results to System.out, as well as
writing a full set of output results to their <code><link
anchor="test-options-logfile">logFile</link></code>, as
set from a .properties file or the command line. Generally the
output results file is easier to deal with. The basic format is
fairly simple and you can certainly read it as-is.</p>
<p>To 'pretty-print' results or produce reports, please use the
viewResults.xsl stylesheet and associated batch/shell files, like so:
<code>cd \xml-xalan\test</code><br/><br/>
<code>runTest.bat MyTest -options blah <link
anchor="test-options-logfile">-logFile
results\MyResults.xml</link></code><br/><br/>
<code>viewResults.bat results\MyResults.xml
results\MyPrettyResults.html [options]</code><br/><br/>
These options are passed to Xalan's command line to transform the
xml formatted output results into a nicer-looking html file. The most
common option would be <code>-param loggingLevel 99</code> to get
more output shown in the results (higher numbers up to 99 show more
details,
lower numbers, down to 0, show fewer details).
</p>
</s2>
<anchor name="test-options"/>
<s2 title="Common Test Options">
<p>Most tests can either accept options from a properties file,
via:<br/>
<code> TestName -load file.properties</code><br/><br/>
or simply from the command line (which overrides the properties file)
like:<br/>
<code> TestName -arg1 value1 -arg2 -arg3
value3</code><br/><br/></p>
<p>To see all options, call a test with an illegal argument to force it
to print out it's .usage().</p>
<p>For another description of options, see
<code>xml-xalan\test\ContribTest.properties</code>,
which describes most of them as used in the context of running the
ConformanceTest driver
over the xml-xalan\tests\contrib tree of stylesheets. Simply change
inputDir and goldDir
to run over a different set of files (like a conformance test suite,
which we hope to
add soon).</p>
<p>Quick list of options</p>
<anchor name="test-options-logfile"/>
<gloss>
<label>-logFile <ref>resultsFileName.xml</ref></label>
<item>sends test results to an XML-based results file</item>
<label>-load <ref>file.properties</ref></label>
<item>(read in a .properties file, that can set any/all of the
other opts)</item>
<label>-inputDir <ref>path/to/tests</ref></label>
<item>path to a directory tree of input *.xml/*.xsl files, using
your system's separator</item>
<label>-outputDir <ref>path/to/output/area</ref></label>
<item>where all output is sent</item>
<label>-goldDir <ref>path/to/gold</ref></label>
<item>path to a directory tree of reference output - this tree
should be
a parallel structure to the inputDir tree</item>
<label>(paths should be appropriate to the platform you're on,
and must be \\escaped\\properly if in a .properties file)</label>
<label>-category <ref>dirName</ref></label>
<item>only run this single named subdir within inputDir</item>
<label>-excludes <ref>'test1.xsl;test2.xsl'</ref></label>
<item>will skip running any specifically named tests; do not use
any path elements here</item>
<label>-flavor <ref>xalan|trax|trax.d2d</ref></label>
<item>which kind/flavor of Processor to test; see
<jump
href="apidocs/org/apache/qetest/xslwrapper/ProcessorWrapper.html">ProcessorWrapper.java</jump>
</item>
<label>-noReuse</label>
<item>will force the ProcessorWrapper recreate a new processor for
each file processed
(normally, it re-uses the same processor when possible by calling
reset() or some
like method)</item>
<label>-debug</label>
<item>prints extra test debugging info</item>
<label>-precompile</label>
<item>will use a precompiled stylesheet, if applicable to that
ProcessorWrapper</item>
<label>-noErrTest</label>
<item>will skip running '*err' subdirectory tests, if applicable
(<ref>subject to change</ref>)</item>
</gloss>
<p>Note that most options work equivalently with either Xalan-J or
Xalan-C tests.</p>
</s2>
<anchor name="how-to-run-c"/>
<s2 title="How-to: Run Xalan-C tests">
<p>In progress. A few C++ API tests are checked into the
<code>xml-xalan/c/Tests</code>
repository area already. To execute any set of 'conformance' tests
with the
Xalan-C processor, we currently use the
org.apache.qetest.xsl.<jump
href="apidocs/org/apache/qetest/xsl/CConformanceTest.html">CConformanceTest</jump>
driver. This is written in Java to take advantage of the framework and
results reporting, but basically constructs a command line for each
test
and then shells out to <code>TestXSLT.exe -in file.xsl...</code> to run
the test;
it then uses the same validation routines as the Java
ConformanceTest.</p>
<p>Contact <jump href="[EMAIL PROTECTED]">[EMAIL PROTECTED]</jump> if
you'd
like to help in the Xalan-C API testing effort.</p>
</s2>
</s1>
1.1 xml-xalan/test/java/xdocs/sources/tests/submit.xml
Index: submit.xml
===================================================================
<?xml version="1.0" standalone="no"?>
<!DOCTYPE s1 SYSTEM "sbk:/style/dtd/document.dtd">
<s1 title="Submitting New Tests">
<ul>
<li><link anchor="write-API-tests">How to Write API Tests</link></li>
<li><link anchor="write-xsl-tests">How to Write Stylesheet Tests</link></li>
</ul>
<anchor name="write-API-tests"/>
<s2 title="How to Write API Tests">
<p>Use the existing framework! It provides lots of useful functionality,
and will help us to maintain your tests.</p>
<p>For example: To write a new TRAX/javax.xml.transform API test:</p>
<p>- open org.apache.qetest.trax.REPLACE_template_for_new_tests.java</p>
<p>- follow directions to rename the file (and put in the correct
dom/sax/stream
subdir, if needed) and search-and-replace all REPLACE_* tokens</p>
<p>- write one-time-only setup code in testFileInit()</p>
<p>- write a number of testCase<ref>n</ref> methods. Each one should
be independent from the other test cases. Try to test between one
and ten or so individual test points (or calls to reporter.check(...)
for each testCase method.</p>
<p>- Never use System.out/.err - always use reporter.log*Msg() (to report
general messages), or reporter.check() (to validate a specific test point)</p>
<note>This is an important point. Bottlenecking all output from the tests
through a <jump
href="apidocs/org/apache/qetest/Reporter.html">Reporter</jump>
allows us to manage and analyze the results much more easily. Reporters also
put all output to both the console and to your <link idref="run"
anchor="test-options-logfile">logFile</link>.</note>
<p>- Build the tests, including your new one, <link idref="getstarted"
anchor="how-to-build">as described</link></p>
<p>- Put your test's supporting xml/xsl files in
xml-xalan/test/tests/api/trax or
subdirectories</p>
<p>- Use xml-xalan\test\traxapitest.bat (and APITest.properties) <link
idref="run" anchor="how-to-run">to run your test</link>!
Results will be placed by default into
xml-xalan\test\results-api\APITest.xml</p>
<p>The same basic template can be used for other kinds of API tests, with
appropriate
changes to the package name, etc.</p>
<p>You can pretty-print the results by using the <link idref="run"
anchor="how-to-view-results">viewResults.xsl stylesheet</link> to turn
the XML into an HTML format.</p>
<p>The same file can be used with minor modifications for either Xalan-J 1.x
or Xalan-J 2.x API tests, with proper package and import modifications.</p>
</s2>
<anchor name="write-xsl-tests"/>
<s2 title="How to Write Stylesheet Tests">
<p>Test cases in the upcoming "conf" group will ultimately be submitted
to OASIS as part of
their vendor-independent project on XSLT/XPath Conformance Testing. For more
information about this project, visit
<jump
href="http://www.oasis-open.org/committees/xslt/index.html";>http://www.oasis-open.org/committees/xslt/index.html</jump></p>
<p>The OASIS project will combine test cases from different sources and
provide a way to
customize the set of tests according to design decisions made for each
processor
under test. We expect that when the OASIS test system is delivered, it will
supplant
most or all of the "conf" group provided here. Conformance tests are designed
as
streamlined "atomic" (or at least "molecular") tests, so there are several
hundred of them.
Currently, Lotus/IBM is working on submitting a significant body of
Conformance tests
to OASIS. We also would like to temporarily provide most of the tests here on
Apache for Xalan developer's use, while OASIS finalizes it's test suite.</p>
<p>You are invited to submit additional test cases by checking them into the
"contrib"
area or mailing them to <jump href="mailto:[EMAIL PROTECTED]">[EMAIL
PROTECTED]</jump>.
If you want to help with testing Xalan and wish to be
assigned to write some cases, send email to David Marston stating your
interest.
Contributed tests may be sent along to the OASIS conformance project if they
test
conformance. Guidelines for comments in tests are still evolving as part of
the
OASIS project, so it may be necessary to add more comments or other
annotation
after the test is submitted. We hope to have a template for contributing
stylesheet tests available soon that will closely match the eventual OASIS
format.</p>
</s2>
</s1>
