On Wed, Nov 7, 2018 at 3:45 PM Neil Van Dyke <n...@neilvandyke.org> wrote:
> Greg Hendershott wrote on 11/6/18 9:56 AM: > > I'd suggest writing to stderr the same location format used by rackunit. > > OK, thank you (and David Storr). I looked at the current RackUnit. How > about the following proposal? > > Proposal: A test failure message is a one-line message that: > * in DrRacket, includes the clickable error icons; and > * on the command line (or in Emacs, Vim, etc.), is to stderr, and > includes the filename+line+column in a conventional non-Racket-specific > format. > > Details... > > The reason I don't want to use the RackUnit message format exactly is > that it has some excess stuff in it that won't work great with reporting > many tests, like I tend do them in Overeasy. Consider, for example, a > dozen tests failing, and you can eyeball the dozen one-line messages at > once to see whether they appear all related. > > To implement the DrRacket clickable icons, it looks from > "racket-7.1/share/racket/pkgs/rackunit-lib/rackunit/private/format.rkt" > like I can do something like this simplified demo: > > #lang racket/base > (with-handlers ((exn:fail? (lambda (exn) > ((error-display-handler) (exn-message > exn) exn)))) > (error "TEST-FAILURE: [foomodule barproc empty-string] Expected > 42, got 0.")) > > In DrRacket, that demo displays a concise single-line message, with the > clickable icons: > > <ICONS> TEST-FAILURE: [foomodule barproc empty-string] Expected 42, > got 0. > > From the command line, that demo displays multi-line, which I don't > want (mainly for the dozen-failures-eyeballing reason): > > TEST-FAILURE: [foomodule barproc empty-string] Expected 42, got 0. > context...: > "/home/user/paddle-test-play.rkt": [running body] > temp37_0 > for-loop > run-module-instance!125 > perform-require!78 > > So, how about the command line format omits the calling stack context, > and instead uses a simple conventional one-line format that works out of > the box with most tools, like one of: > > PROGRAM:FILE:LINE:COL: MESSAGE > PROGRAM:FILE:LINE: MESSAGE > FILE:LINE:COL: MESSAGE > FILE:LINE: MESSAGE > > where "PROGRAM" (if present) *might* be something like "TEST" or > "TEST-FAILURE", instead of "my-racket-test-engine" or "raco test" or > whatever Racket program invoked the tests. > > BTW, that "[foomodule barproc empty-string]" part of the failure message > is Overeasy's `test-section` symbolic way of identifying tests, which is > complementary to source location, and meaningful to programmers/testers: > https://www.neilvandyke.org/racket/overeasy/#%28part._.Test_.Sections%29 > > BTW, the "Expected ___, got ___." message would also have other forms, > for different test pass criteria or ways that tests could fail. That > simple form is just for the most common case, of two sets of normal > expression result values failing a equivalence check by `equal?`. > > Will test successes show output as well? For me that's the hard stop on using RackUnit -- the fact that it shows nothing on success means that there's no way to tell the difference between "everything ran and passed", "some things ran and passed but some things that should have run did not", and "nothing ran". -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.