I have pushed the system `fiveam-asdf`, which supports integration
between ASDF and the FIVEAM test library, to the contribs directory in
the ASDF repo. Please have a look.
Be warned! It is old, and not being broke, hasn't been fixed. It
inappropriately is housed in the `ASDF` package and inappropriately
exports extensions from that package.
But I believe it's still useful as an example of how to raise conditions
when the test operation goes wrong. Catching those exceptions can be
used to cause a build to fail in a CI system, typically by running lisp
in batch mode and having it exit with a nonzero error code if the test
operation fails.
Best,
r
On 5 Jun 2018, at 4:09, Mark Evenson wrote:
We use ASDF to encapsulate the building and testing many systems under
an
automated test runner.
For a given system, ASDF:TEST-SYSTEM always returns boolean truth as
long as
the invocation of the underlying test suite succeeds.
This means there is no programatic way to provide a boolean as to
whether all
the tests passed or not on the invocation of a test suite to the
invoker of
ASDF:TEST-SYSTEM.
It seems that this issue has been raised before, as the ASDF manual
documents
TEST-OP
<https://www.common-lisp.net/project/asdf/asdf/Predefined-operations-of-ASDF.html#Predefined-operations-of-ASDF>
with:
The results of this operation are not defined by ASDF. It has proven
difficult
to define how the test operation should signal its results to the
user in a way
that is compatible with all of the various test libraries and test
techniques
in use in the community, and given the fact that ASDF operations do
not return
a value indicating success or failure. For those willing to go to the
effort,
we suggest defining conditions to signal when a test-op fails, and
storing in
those conditions information that describes which tests fail.
Is this still the best current practice to introspect the situation of
failing
tests? Can someone point me to an example implementation of this
technique?
--
"A screaming comes across the sky. It has happened before but there
is nothing
to compare to it now."
