Sorry to parasite this mailing-list, but I'm trying to design the testing aspect of XCVB right now. The constraint is that we want to be able to use make as a backend, and that test results should thus be reified as files if we want to avoid re-running already-computed tests.
My dim ideas about it include the following: * test reports are targets depending on properly compiled fasls or images, produce test reports. * a test report is a file the first form of which is a simple SEXP to be read, following some standardized structure to specify overall success, status of individual tests (including error message, maybe information as to last success, etc.). * from the test reports, a success witness (empty file) may be created on success. * the success witness fails to be created when the test wasn't a success, triggering an error for make to catch. * preparing the makefile erases the success witnesses. If ASDF's test-op is to be standardized, may I suggest that we agree on the format for a SEXP to be returned, that will report the test results? [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] A child of five would understand this. Send someone to fetch a child of five. -- Groucho Marx 2009/8/5 Robert Goldman <rpgold...@sift.info>: > Nikodemus Siivola wrote: >> 2009/8/5 Robert Goldman <rpgold...@sift.info>: >> >>> Part of the problem with test-op is that the desired behavior has not >>> been specified by the ASDF community. Because of the nature of ASDF, it >>> is impossible for >>> >>> (asdf:test-system <system>) >>> >>> to return a value indicating whether or not <system> has passed its tests. >> >> I would propose that as a matter of good style implementations of >> TEST-OP should signal an error is tests do not pass. It is the >> simplest way to make sure the result is not misinterpreted... >> >> Cheers, >> >> -- Nikodemus > > A second point is that test libraries are often very chatty, and it's > unpleasant to have their output mixed in the same stream with the output > of the compiler and whatever all else ASDF might happen to trigger > splatting onto the output stream. So I really think that some stream > argument is essential, even if we were to accept your proposal. > > best, > r _______________________________________________ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel