On Tuesday 28 December 2004 17:54, Jeroen Frijters wrote: > > And besides, if a broken test is added, this way there will be > > motivation to resolve the discrepancy. With a whitelist, a broken > > test can get added but no one will notice and then it just sits > > there getting stale. > > The alternative is worse, it's the situation we're in now. There are > lots of broken tests and we have no idea if the failures are due to > Classpath, VM or test bugs.
I may be mistaken, and all this is way more complicated then I think. But please bear with me; Just about all Mauve tests run correctly on Suns JVM, with a few exceptions since that one has bugs to. Agree? What about making that the 'baseline' test, with a file noting the specific exceptions. Something like; gnu.testlet.java.awt.RenderingHints.containsKey:4 = 1.0 - 1.4.2_05 meaning that test is expected to fail on the baseline JVM (Suns) from version 1.0 up-to-1.4.2_05 With that in place; all failing tests are bugs in the JVMs and/or Classpath. Next; lets write a test-runner that automatically creates one jar file (I just finished doing so, its 1Mb) with all tests from java.* and javax.* and then places that on a server, with the test-run-results from a couple of Sun JVMs. I could probably put that into a runnable jar where you can pass arguments like version and other options taken from the source files. Other JVMs either download that jar, or do a nightly/weekly compile on the same server and also publish their results in a similar file. Next; write an application that shows which tests fail on which JVM using all those test-results and draw conclusions. If a test fails on _all_ JVMs, except Suns, its probably a bug in Classpath. Did I misunderstand the conversation here? Or is this a much easier way to go? I'm willing to put up a jar somewhere (can I scp it to a red-hat server?) since I seem to be one of the few who has Suns JVM(s) and ant installed :) jar: http://members.home.nl/zander/alltests.jar (1mb) -- Thomas Zander
pgp9SKhqRelhO.pgp
Description: PGP signature
_______________________________________________ Classpath mailing list [email protected] http://lists.gnu.org/mailman/listinfo/classpath
