Henry Zongaro writes:
>If we go with separate tags for the accept-gold tree, we'll end up with
>multi-level tagging - at least conceptually. When we put out a Xalan-J
>2.7.9 release, say, we'll have a xalanj279 tag for the source and some
>of the tests, and tags like xalanj279-accept-gold-xalan-interpretive
>xalanj279-accept-gold-xsltc for the different accept silver trees.
That seems wrong from a Quality Engineering perspective. Taking it from
general principles down to specifics:
The "conf" (conformance) cluster should be a set of tests for which all
XSLT processors produce the same results, after accounting for the
discretionary items. The "accept" (acceptance) cluster is a set of tests
which Xalan must pass in order to be deemed good enough to ship. In both
areas, the tests themselves represent what Xalan should have done ever
since Day One and should continue to do forever, thus there is no need
to apply a version number to these tests. As you've noticed, I've added
new tests in the past couple weeks, some of which Xalan has been passing
for a long time and some of which reveal bugs that have existed for a
long time. Still, the tests themselves are timeless.
I propose to accommodate the XSLT/XPath 2.0 standards in parallel
"conf2" and "accept2" clusters. A very few behaviors will change in a
non-compatible way, but the XSL Working Group of the W3C seems to be
heading toward a "1.0 compatibility module" under which processors
will react to version="1.0" in the stylesheet by behaving exactly as
an XSLT 1.0 processor. Since Xalan would implement this module, that
means the XSLT 2.0 version will need to pass all of conf2, accept2,
conf, and accept, the latter two via the compatibility module.
We account for known bugs through the test.properties file, and that
probably needs to be versioned for the various processors and
whatever else is spawning a separate accept-silver tree. But the tests
themselves can be applied to all flavors, even if the silver files
have to be different. (Note that this statement applies to accept but
not the extension tests.) A person developing a patch needs to check
out conf and accept tests, start with code that passes all such tests
as filtered by test.properties, make their patch, and still pass all
the (filtered) conf and accept tests just as cleanly after the patch.
If the patch is a fix for a known bug, then presumably the patched
version passes at least one more conf or accept test, which can then
be de-listed from the exclude list in test.properties. Subsequent
patches will need to pass a slightly larger test battery. (I know
that most readers of this list already know this policy, but I can't
recall when it was last written out like this.)
.................David Marston
