curcuru 01/09/05 11:55:06
Modified: test/java/xdocs/sources/tests getstarted.xml overview.xml
run.xml submit.xml
Log:
Documentation updates; mostly caught up to new methodologies with better
descriptions
Revision Changes Path
1.5 +14 -13 xml-xalan/test/java/xdocs/sources/tests/getstarted.xml
Index: getstarted.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/getstarted.xml,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- getstarted.xml 2001/08/21 19:58:31 1.4
+++ getstarted.xml 2001/09/05 18:55:06 1.5
@@ -57,7 +57,8 @@
the normal conf tests, but only on the axes subdir, and using the
TraxSaxWrapper class.
<br/><code>build api -DtestClass=TransformStateTest
-Dapi.loggingLevel=30 </code> This runs
the org.apache.qetest.xalanj2.TransformStateTest with a lower
loggingLevel (so less is output).
- Note that testClass is one of the few properties that is not prefixed.
+ Note that testClass is one of the few properties that is not prefixed,
since it is
+ not passed on to the test itself, but is only used by the Ant script to
load the test.
</p>
</s2>
@@ -110,26 +111,26 @@
for a simple alternate way to set your classpath using JARDIR. This
allows QE/QA/test
people to run the same set of tests quickly against different versions
of the product.</p>
<note>(Aug-01 Section needs updating: JARDIR docs -sc)</note>
- <p>ALTERNATE: Set an environment variable JARDIR to point to directory
where you have all applicable jars.
- Delete testxsl.jar if present there.
+ <p>ALTERNATE way using JARDIR:</p>
+ <p>1: Set an environment variable JARDIR to point to directory where you
have all applicable jars.
+ <br/>Delete testxsl.jar if already present there (if you want to
rebuild it).
<br/>Required: at least xalan.jar and bsf.jar from a Xalan-J build;
xerces.jar from the <ref>same</ref> Xalan-J build (or your parser's
JAXP-compatible jar instead of xerces.jar);
- and js.jar (see <jump
href="http://xml.apache.org/xalan-j/extensions.html#supported-lang";>Xalan-J doc
for information on js.jar</jump>)
+ and js.jar (see <jump
href="http://xml.apache.org/xalan-j/extensions.html#supported-lang";>Xalan-J doc
for information on js.jar, only needed for extensions tests</jump>)
<br/><code>set JARDIR=e:\builds\myjars </code>
</p>
- <p>3: Copy all product jars to the JARDIR</p>
- <p>4: cd to the test directory and cleanup the tests.</p>
+ <p>2: Copy needed product jars to the JARDIR</p>
+ <p>3: cd to the test directory and cleanup the tests</p>
<p><code>cd e:\builds\xml-xalan\test </code>
- <br/><code>build.bat clean </code>
+ <br/><code>build clean </code>
</p>
- <p>5: Build the tests appropriate to your product.
- <br/>Xalan-J 1.x:
- <br/><code>build.bat package.xalan </code>
- <br/>Xalan-J 2.x:
- <br/><code>build.bat package.xalan2 </code> (<ref>or
package.trax</ref>)
+ <p>4: Build the tests
+ <br/><code>build jar </code>
</p>
<p>This will create <code>build\testxsl.jar </code>, which you should
manually copy
- into JARDIR before you <link idref="run" anchor="how-to-run">run any
tests</link>.</p>
+ into JARDIR before you <link idref="run" anchor="how-to-run">run any
tests</link>.
+ Note that you might instead want to <code>build all</code> if you want
to also
+ run any of the bugzilla or extensions tests as well.</p>
<note>The use of JARDIR is not required; you're free to manage the
classpath
yourself, or the test build file will assume it should use the Xalan-J
2.0
directories by default. I've found that simply putting all the needed
jars in
1.6 +31 -7 xml-xalan/test/java/xdocs/sources/tests/overview.xml
Index: overview.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/overview.xml,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- overview.xml 2001/08/21 19:58:31 1.5
+++ overview.xml 2001/09/05 18:55:06 1.6
@@ -4,6 +4,7 @@
<s1 title="Overview">
<ul>
<li><link anchor="purpose">Purpose of these tests</link></li>
+<li><link anchor="brief-overview">Brief overview of testing
methods</link></li>
<li><link anchor="dir-map">Directory Structure</link></li>
<li><link anchor="test-map">Listing of Java tests and drivers</link></li>
<li><link anchor="credits">Credits for the tests</link></li>
@@ -18,9 +19,23 @@
the tests before and after a code change. That way you can at least
ensure that
your changes didn't cause any regressions to the code before you check
your
changes in. In the future, we hope to provide a tool to compare test
results from
- one run to another to make this process even simpler.</p>
+ one run to another (without necessarily having to re-run the test) to
make this process even simpler.</p>
</s2>
+ <anchor name="brief-overview"/>
+ <s2 title="Brief overview of testing methods">
+ <p>The Xalan tests include a richly featured and fully automated testing
framework
+ that the great majority of the tests use. The org.apache.qetest package
provides
+ an independent testing and automation harness for Xalan, including
logging and
+ reporting mechanisims.</p>
+ <p>Nearly all tests are automated, run without any user interaction and
with a minimum amount of setup, and produce
+ a rolled-up report of their pass/fail/other status. Our existing
testing library includes
+ a wide array of tests, from XSLT conformance to detailed API tests, and
welcomes
+ user-submitted tests as well. The 'smoketest' target (a subset of the
most important
+ tests) are also integrated into the <jump
href="http://jakarta.apache.org/builds/gump";>GUMP nightly build system</jump>,
and thus ensure a
+ minimum baseline of functionality on a daily basis.</p>
+ </s2>
+
<anchor name="dir-map"/>
<s2 title="Directory Structure">
<gloss>
@@ -30,7 +45,8 @@
<label></label>
<label><code>xml-xalan/test/bin</code></label>
<item>Java test automation dependencies (includes
- <jump href="http://jakarta.apache.org/ant/index.html";>Ant
1.2</jump>)</item>
+ <jump href="http://jakarta.apache.org/ant/index.html";>Ant 1.2</jump>
+ so that you don't have to install it yourself)</item>
<label><code>xml-xalan/test/java/src</code></label>
<item>Java test automation source tree - this includes
a generic testing framework as well as specific API tests for parts of
Xalan
@@ -69,13 +85,15 @@
<anchor name="test-map"/>
<s2 title="Listing of Java tests and drivers">
+ <note>This section is a sort of catalog of existing tests. Beginning
users
+ will probably want to see <link idref="run" anchor="how-to-run">how to
run tests</link> as well.</note>
<p>Java Test Drivers (data driven testing</p>
<p>A Java Test Driver executes a test for each xml/xsl file pair in
the specified directory tree or each pair in the specified fileList.
For each test, the driver iterates over the tree or list of files
and asks a Testlet to execute a test on each one. This is also similar to
data driven testing, where a common algorithim is defined for a test case,
and
-then a large number of data points are run through the test case in order.
+then a large number of data points (in this case, the xml/xsl file pairs)
are run through the test case in order.
The best example is <jump
href="apidocs/org/apache/qetest/xsl/StylesheetTestletDriver.html">StylesheetTestletDriver</jump></p>
<p>The Test Drivers rely on various Testlet implementations
to define the actual testing algorithim to apply to each xml.xsl
@@ -92,11 +110,15 @@
using DOM trees, SAX events, or input/output streams.</p>
<p>The three levels of iteration, test algorithim, and
processor flavor are all independently changeable, so we can
-easily try out different kinds of tests.</p>
+easily try out different kinds of tests. This technique is used to
+run the full sets of 'conf' (conformance), 'perf' (performance),
+extensions, and 'contrib' (user contributed stylesheets) tests.</p>
<gloss>
<label>org.apache.qetest.xsl.<link idref="run"
anchor="how-to-run-c">CConformanceTest</link></label>
-<item>essentially the same as ConformanceTest, but for Xalan-C. I plan to
-make this use the Testlet model soon to simplify this model, and
+<item>This is similar to the StylesheetTestlet, but for Xalan-C.
+It has a different method of discovering test files, and simply shells
+out to a command prompt to run each stylesheet through the TestXSLT.exe
program.
+I plan to make this use the Testlet model soon to simplify this model, and
to provide better support for differing command lines.</item>
</gloss>
@@ -142,7 +164,9 @@
<label>TestThreads.java</label>
<item>MANUALLY executed test for running multiple threads
-and transforming multiple stylesheets simultaneously.</item>
+and transforming multiple stylesheets simultaneously. An updated and
automated
+test is now available, org.apache.qetest.xsl.ThreadedTestletDriver, which
+should be used instead. </item>
</gloss>
<p>All in subpackages of: org.apache.qetest.trax</p>
1.6 +22 -25 xml-xalan/test/java/xdocs/sources/tests/run.xml
Index: run.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/run.xml,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- run.xml 2001/08/21 19:58:31 1.5
+++ run.xml 2001/09/05 18:55:06 1.6
@@ -13,8 +13,8 @@
<s2 title="How-to: Run tests">
<p>Nearly all tests for Xalan are independent Java classes built
into testxsl.jar that
- can be run either standalone on the command line, programmatically
- from your application, or from
+ can be run either individually on the command line, programmatically
+ from your application or our handy Ant build.xml file, or in batches
from
<jump
href="apidocs/org/apache/qetest/xsl/XSLTestHarness.html">XSLTestHarness</jump>.
There really isn't any magic to them: you can just set your classpath
and
execute java.exe to run them; some Tests and Testlets currently provide
defaults
@@ -35,21 +35,15 @@
<br/>(runs TRAX interface tests with Xalan-J 2.x<br/>
</p>
<p>Alternately: some convenience batch files are provided for the most
common
- tests - these simply turn around and call build.bat/.sh for you.<br/></p>
+ tests - these simply turn around and call build.bat/.sh for you.<br/>
+ (Namely conf.bat, perf.bat, and the like)</p>
<p>Alternately: Run java.exe and the desired test class
yourself:<br/></p>
- <p>Note: the above batch and shell files, and the JARDIR environment
variable,
- are simply conveniences for running the tests. The Java API tests and
test drivers can
- also be executed by hand, for those who wish to manage their own
classpaths and/or
- simply pass all arguments to the tests on the command line, etc.</p>
- <p>Sorry! We don't have .sh equivalents for the convenience .bat files -
- submissions of ports of these files are welcomed! We do plan to add an
Ant
- task that can run Xalan Tests and Testlets, so in the future both
building
- and running tests will mostly be done from an Ant buildfile, which is
- cross-platform by default.</p>
<note>Running tests with alternate JAXP parsers: all
org.apache.qetest.trax.*
tests can be run with Xalan-J 2.x and any JAXP 1.1 compatible parser,
like
crimson.jar. Be sure to manually set the appropriate system properties
to use
- your parser instead of xerces.jar, which is the default for
Xalan-J.</note>
+ your parser instead of xerces.jar, which is the default for Xalan-J.
Tests will
+ also run on any JAXP 1.1 compatible xslt processor, namely either our
default
+ xalan one or our new xsltc one.</note>
</s2>
<anchor name="how-to-view-results"/>
@@ -58,12 +52,13 @@
writing a full set of output results to their <code><link
anchor="test-options-logfile">logFile</link></code>, as
set from a .properties file or the command line. Generally the
output results file is easier to deal with. The basic format is
- fairly simple and you can certainly read it as-is.</p>
+ fairly simple and you can certainly read it as-is. Also, many tests
+ send only summary results to the console, but full output to the
results file.</p>
<p>To 'pretty-print' results or produce reports, please use the
viewResults.xsl stylesheet and associated batch/shell files, like
so:<br/>
<code>cd \xml-xalan\test</code><br/><br/>
<code>build conf -DoptionName=optionValue <link
anchor="test-options-logfile">-Dqetest.logFile
results/MyResults.xml</link></code><br/><br/>
- <code>viewResults.bat results/MyResults.xml
results\MyPrettyResults.html [options]</code><br/><br/>
+ <code>viewResults.bat results/MyResults.xml
results/MyPrettyResults.html [options]</code><br/><br/>
These options are passed to Xalan's command line to transform the
xml formatted output results into a nicer-looking html file. The
most
common option would be <code>-param loggingLevel 99</code> to get
@@ -84,14 +79,14 @@
<p>To see all options, call a test with an illegal argument to force
it
to print out it's .usage(). You may mix setting options from a
properties
file and from the command line; command line options will take
precedence.</p>
- <p>For another description of options, see
<br/><code>xml-xalan\test\conf.properties</code>,<br/>
- which describes most of them as used in the context of running the
StylesheetTestletDriver
- over the xml-xalan\tests\conf tree of stylesheets. Simply change
inputDir and goldDir
- to run over a different set of files (like a performance test suite,
which we hope to
- add soon).</p>
- <note>Path-like options set in a properties file must use the local
system's
- File.separator character, and backslashes \ must be escaped \\. The
checked in
- copy of conf.properties is for a Windows platform.</note>
+ <p>For another description of options, see
<br/><code>xml-xalan\test\test.properties</code>,<br/>
+ which describes most of them with comments. Remember that the
prefixes
+ 'qetest.', 'conf.' etc. are used by the Ant build.xml file to manage
which
+ properties are used for different kinds of tests, and are ripped off
before
+ being passed to the Java test script itself. Thus
qetest.loggingLevel=99 in
+ the test.properties file becomes just loggingLevel of 99 when passed
to the test.</p>
+ <note>Path-like options set in a properties file generally should use
+ forward slashes (legal in URL's), even on Windows platforms.</note>
<p>Quick list of options</p>
<anchor name="test-options-logfile"/>
<gloss>
@@ -103,8 +98,6 @@
<item>determines how much information is sent just to the default
ConsoleLogger:
since often you won't be watching the console as the test is
running, you can set this
lower than your loggingLevel to speed the tests up a little</item>
- <label>-load <ref>file.properties</ref></label>
- <item>(read in a .properties file, that can set any/all of the
other opts)</item>
<label>-inputDir <ref>path/to/tests</ref></label>
<item>path to a directory tree of input *.xml/*.xsl files, using
your system's separator</item>
<label>-outputDir <ref>path/to/output/area</ref></label>
@@ -121,6 +114,10 @@
<jump
href="apidocs/org/apache/qetest/xslwrapper/ProcessorWrapper.html">ProcessorWrapper.java</jump>
</item>
<label>-testlet <ref>TestletClassname</ref></label>
<item>For StylesheetTestletDriver, use a different class for the
testing algorithim</item>
+ <label>-load <ref>file.properties</ref></label>
+ <item>(read in a .properties file, that can set any/all of the
other opts)
+ This option is automatically used by the Ant build.xml file, so
you normally
+ would not specify it yourself.</item>
<label>-debug</label>
<item>prints extra test debugging info</item>
</gloss>
1.3 +5 -3 xml-xalan/test/java/xdocs/sources/tests/submit.xml
Index: submit.xml
===================================================================
RCS file: /home/cvs/xml-xalan/test/java/xdocs/sources/tests/submit.xml,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- submit.xml 2000/12/06 20:51:58 1.2
+++ submit.xml 2001/09/05 18:55:06 1.3
@@ -35,13 +35,11 @@
changes to the package name, etc.</p>
<p>You can pretty-print the results by using the <link idref="run"
anchor="how-to-view-results">viewResults.xsl stylesheet</link> to turn
the XML into an HTML format.</p>
-<p>The same file can be used with minor modifications for either Xalan-J 1.x
-or Xalan-J 2.x API tests, with proper package and import modifications.</p>
</s2>
<anchor name="write-xsl-tests"/>
<s2 title="How to Write Stylesheet Tests">
- <p>Test cases in the upcoming "conf" group will ultimately be submitted
to OASIS as part of
+ <p>Test cases in the "conf" group will ultimately be submitted to OASIS
as part of
their vendor-independent project on XSLT/XPath Conformance Testing. For more
information about this project, visit
<jump
href="http://www.oasis-open.org/committees/xslt/index.html";>http://www.oasis-open.org/committees/xslt/index.html</jump></p>
@@ -65,5 +63,9 @@
after the test is submitted. We hope to have a template for contributing
stylesheet tests available soon that will closely match the eventual OASIS
format.</p>
+<p>The Xalan team will continue to provide test automation, like the
+StylesheetTestletDriver and various StylesheetTestlet classes, that
+enables a user to easily run large suites of tests with fully rolled-up
+reporting and a rich set of options.</p>
</s2>
</s1>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
