----- Original Message ----- From: "Tom Jordahl" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, May 30, 2002 1:46 PM Subject: RE: AXIS Test Restructuring
> > Me, I think that as well as cleaning up the build process, we need to work > > on the test suite so it runs more like a big fat junit test; you run all the > > tests and get back an XML file of what worked, what didnt...to date the way > > we go right now the first failure seems to stops the run. > > I know you guys are further along on this now, but I just wanted to note that I believe we need to go in the other direction. Right now we pretty much have to run the whole suite in a lump and what would be really useful is to be able to run individual tests, or a selection of tests. but of course. That is what patternsets are for. If you look at ant's own build file the main test run only runs if you havent told it to run a single test. The main test runs all tests that are valid with the current set of optional packages (i.e. do no SwA without activation.jar), the online stuff only if it thinks we are online, and omits the tests-that-fail set of tests, because we know they fail. What we need to do is move from a <junit> invocation to something what runs buld files directly instead.; the fail-on-first error versus keep-going option should be a controllable aspect of the build. > I also like the 'first failure stops the run' mode, and would not like that to change as the default. It is perfectly reasonable for automation to report all the tests that failed in a run however. well, right now we have 'first fail that isnt a network URL stops the run', which assumes that the failure to handle any network related URL is probably net related. Really we should probe each endpoint for being there and only continue with that test if it is. ...there is some discourse starting on ant-dev with involvement from maven developers on a task to run one or more build files as tests; I've cited Axis as a use case. -steve
