Ryan, thank you very much for refocusing the discussion about testing on the essence and away from the superficial syntactic issues. Syntax, as we all know, is just a macro [which some people may get backwards and conclude that testing is just a macro. Nothing could be further from the truth, however].
Find someone to implement this idea. -- Matthias On Feb 17, 2011, at 8:21 PM, Ryan Culpepper wrote: > On 02/15/2011 07:28 AM, Eli Barzilay wrote: >> And finaly, there's the litmus test for existing code: >> >> * Ryan: is something like this enough to implement the GUI layer? > > Not well, I think. The Test-Result type in Noel's racktest code is too > simple and inflexible. It represents the minimal essence of testing, > but it would be awkward to extend to richer testing sytems. Here's my > counter-proposal for representing the results of tests: > > > TestResult = (test-result header execution status) > where header is a TestHeader, execution is a TestExecution, > and status is a TestStatus > > A TestResult combines a description of the test, a record of its > behavior during execution, and its final status. > > (This fixes the awkward distribution of information in racktest's > Test-Result, with every status variant including a name field---except > for exn:fail, which is anonymous or perhaps stashes it in the > continuation marks.) > > > TestHeader = (test-header name suite info) > where name is (U string #f), suite is (Listof string) > and info is a dictionary. > > A TestHeader identifies the test that has been run via its name and > its enclosing test suite. (Suites can be nested.) It also accommodates > additional information that may be provided by some frameworks but not > others. Examples include the location of the test expression and the > mode of the test (expected to fail, test not yet implemented, etc). > > > TestExecution = dictionary > > TestExecution contains information about the execution of the test > that is unrelated to its success or failure. For example, the rackunit > gui reports the execution time and any custodian-managed resources that > were left around when the test finished. If output is captured during > execution, it would also belong here. > > > TestStatus is one of > - (test-success) > - (test-failure info) > where info is a dictionary > - (test-error raised-value) > > TestStatus represents the status of the test. For the error case, note > that any value can be raised, not just exn structures. > > Common info-dict keys include 'actual and 'expected, and perhaps > 'comparison, but those keys don't apply to all failures. If the > testing framework views failure as something that occurs during > execution of code (as in rackunit, as opposed to check-expect), then > the info-dict would also contain the continuation marks. > > Perhaps there should be another variant for skipped/pending tests, or > perhaps that should be represented by a successful TestStatus and an > annotation on the TestHeader. > > ---- > > And that's not quite the end of it. The rackunit gui creates an entry for a > test case as soon as it starts running, so the user can see what test case is > hanging and interrupt it if they choose. That requires additional > communication between test execution and test display. > > Ryan > _________________________________________________ > For list-related administrative tasks: > http://lists.racket-lang.org/listinfo/dev _________________________________________________ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
