Regression testing.

[Finn Bock]

Hi,

I have a couple of small patches to the layout that I would like to 
submit but first I wanted to test them using your test suites. But I'm 
running into some problems that seems to indicate that I'm looking at 
the wrong test suite (or something).

First I have created a directory test/reference and copied an unmodifed
version of fop.jar into it. Then I try to run 'ant test' but that fails
because the RunTest ant task look for the 'org.apache.fop.apps.Options'
class in the reference jar. But there are no longer any such class in
fop.jar. So I edit RunTest and remove the Class.forName for this class.
Then the test fails because the build.xml expect the version to be 'FOP
1.0dev' but the actual version in the reference is '1.0dev'. So I edit
the assignment of property 'ref-version' in build.xml to be '1.0dev'.
Then the test fails because the avalon AbstractLogEnabled class is 
missing. So I add a hack to RunTest so that an url to the 
avalon-framework jar is added to the list of urls for the URLClassLoader.

Then the test fails because of a NPE in TestConverter.runTests because 
logging has not been enabled. So I add a call to enableLogging(..) 
similar to the call in the main() method.

Then the test fails because of a NPE deep inside xalan:

java.lang.RuntimeException: java.lang.NullPointerException
    at org.apache.xalan.transformer.TransformerImpl.run(3170)
    at org.apache.xalan.transformer.TransformerHandlerImpl.endDocument
    at org.apache.xerces.parsers.AbstractSAXParser.endDocument
    at org.apache.xerces.impl.XMLDocumentScannerImpl.endEntity
    at org.apache.xerces.impl.XMLEntityManager.endEntity
    at org.apache.xerces.impl.XMLEntityManager$EntityScanner.load
    at org.apache.xerces.impl.XMLEntityManager$EntityScanner.skipSpaces
    at org.apache.xerces.impl.XMLDocumentScannerImpl$TrailingMiscDispatc
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocume
    at org.apache.xerces.parsers.DTDConfiguration.parse
    at org.apache.xerces.parsers.DTDConfiguration.parse
    at org.apache.xerces.parsers.XMLParser.parse
    at org.apache.xerces.parsers.AbstractSAXParser.parse
    at org.apache.xalan.transformer.TrAXFilter.parse(134)
    at org.apache.fop.apps.Driver.render(Driver.java:622)
    at org.apache.fop.apps.Driver.render(Driver.java:558)
    at org.apache.fop.tools.TestConverter.runTest(314)
At this point I'm beginning to suspect that these are not the tests I'm 
looking for <wink>.

So how do you folks verify that the changes you make does not cause 
regressions?

regards,
finn