On 2016/4/29 04:31, Ian Tegebo wrote: > In reading through the history and issues of ASDF [1], I came across > complaints that various LISP implementations were broken in ways that > complicated its development. Thinking that there must be a conformance > test suite out there, I found ansi-test. […] > Do you think it makes sense to include ansi-test into cl-test-grid?
As an ABCL "implementor"--more a maintainer of the work of others at this point--I have always found ANSI-TEST to be helpful and would love to see its output included in CL-TEST-GRID. But I get the sense that my enthusiasm for ANSI-TEST is not necessarily shared by other contributors to implementations. When the current ABCL maintainers "took over" in 2006, we committed a fair amount of energy to understanding and fixing why we were failing various parts of ANSI-TEST. For me personally, who was more-or-less learning Common Lisp at the same time, studying the ANSI-TEST code was an invaluable aid to understanding the finer points of the specification. But as I expanded my knowledge, I wondered why other open source CL implementations didn't seem to devote equal energy to fixing problems flagged by ANSI-TEST. I remember asking an SBCL contributor--whose name I have since forgotten--at a "Lisp Gathering" organized by Andreas Fuchs in Vienna in 2008 (?) about why the SBCL community, normally a hyper-competitive bunch, didn't seem to care that ABCL failed fewer parts of ANSI-TEST than SBCL. The essential gist of his reply was along the lines that after a certain point of conformance, chasing down all the individual "little" points wasn't as effective as working on core parts of the implementation. My reply was along the lines that, sure, an individual test doesn't tell you much about the quality of an implementation, but as one adds more and more "dumb" tests, at some point the test suite crosses a threshold where it appears quite "smart". At the time, ANSI-TEST was a pretty static entity, seeing the occasional bugfix. While it is not at all clear that my little anecdote accurately describes a widely held contemporary opinion of the ANSI-TEST suite relevance, I think that the nature of software development has changed enough over the past decade to the point that ANSI-TEST could become quite useful both for implementors and ASDF system creators. In contrast with a decade ago the effort needed to add a new test or to refer to a specific result of a test via tools like CL-TEST-GRID has been dramatically reduced, increasing the chances of collaboration. ANSI-TEST could become a communication mechanism for ASDF package developers to address the implementation community: when an actual conformance issue arises a specific facet of ANSI-TEST could be referenced (or first contributed if it didn't exist) in the process of working with implementations towards resolution. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
