On Mon, 2007-03-05 at 10:21 +0000, Rupert Smith wrote: > As for gathering reports, I was thinking that each test (sender part) > would report pass/fail (message body containing reason for failure in > failure cases) to the coordinator, of which there is only one, and it > will write out the xml reports. Reason for that being that the code to > write the reports only needs to be written/maintained in one place. Well everybody needs to write their own XML fragments, the coordinator is just gluing them together into a report. Not hard though JUnit & CppUnit both have XML outputters already. I do see the advantages of having a central point collate all the results. Ideally the tests should be plain JUnit/CppUnit and all the extra wiring to send results should be in special test runners.
> Was also thinking of adding the requirement that each test client talk > to the coordinator over a seperate AMQ connection to that which is > sends its test messages over. The idea being, that if a test failure > causes closure of the connection, it should still manage to send its > report. A more serious melt-down that causes the test client to > completely fail, should still result in the test report being written > out as a fail, because the coordinator knows which tests/clients it > started -> which ones did not produce a report -> which ones to write > out a failure for. So no covered tracks. In this case, look in the log > for the dead test client to figure out what happened. Does this sound > ok? Yup, give it a go. If necessary we can have tests also log to a local file or console as a back-up in the event of a crash preventing the result message from being sent. > Producing updated working interop spec, with a view to putting it on > the wiki later today. Looking forward to it!
