Re: [racket-users] rackunit and logging
For those reading this later: there's a bunch of useful information in Alex Harsanyi's blog post and corresponding code: https://alex-hhh.github.io/2019/11/custom-rackunit-test-runner.html https://github.com/alex-hhh/ActivityLog2/blob/master/test/custom-test-runner.rkt Shriram -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CAJUf2yTpipwqQD51datA2OO7M%3D-ABzuswHPrNoNF8zkDipSjtg%40mail.gmail.com.
Re: [racket-users] rackunit and logging
> On May 23, 2020, at 08:53, Shriram Krishnamurthi wrote: > > Alex, thanks for that information. I'm going to go investigate that next. Related to that, I just remembered the existence of rackunit/text-ui and rackunit/gui, which implement two different reporters for RackUnit test cases/suites. Looking at their source might be informative, since neither is particularly complicated. The text reporter uses fold-test-results, and the GUI reporter uses foldts-test-suite. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/3E7DA1AD-851F-4F95-A65B-565D9953A7D5%40gmail.com.
Re: [racket-users] rackunit and logging
> On May 22, 2020, at 18:47, Shriram Krishnamurthi wrote: > > As an aside, I'm not entirely sure what `test-log!` is there for. Presumably > it's to record in the log "tests" run by operations that are not part of > rackunit? I'm curious how people have used it. Other people have answered other parts of your question, but I don’t think anyone has answered this point. The answer is that test-log! isn’t really part of RackUnit at all—it comes from the module rackunit/log, but note that it comes from the package testing-util-lib, not rackunit-lib. (I can’t say for certain, but my guess is that the module is named rackunit/log for historical reasons more than anything else.) test-log! is really mostly an API for cooperating with `raco test`. When you run `raco test`, you’ll find that it reports how many tests were run and how many of them passed, even if you test multiple modules at once. Somehow, test frameworks need to communicate this information to `raco test`, and test-log! is the mechanism through which they do that. Aside from that, `raco test` mostly just instantiates modules in the usual way. All of the other success or failure reporting is just ordinary side-effectful printing performed by test frameworks themselves. All of RackUnit’s “check info stack” machinery is specific to RackUnit; `raco test` doesn’t know or care about that at all. RackUnit calls test-log! internally during execution of each check, so if you use define-check, you don’t have to call test-log! yourself (and indeed, you shouldn’t, or else your tests will be counted twice). > TL;DR: If I want to record what happened on all the checks for post-execution > processing, do I need to (a) create my own log and, to do so, (b) rewrite all > the checking predicates to provide the information that the detailed log > needs? As the above explanation implies, if you don’t want to use RackUnit, you can do whatever you want. You can print your own (arbitrary) messages upon failure, and if you want to integrate with `raco test`, you should just make sure to call test-log! once for each executed test case. If you do want to use RackUnit, then yes, you have to cooperate with all of RackUnit’s machinery for implementing checks. Personally, I find RackUnit’s design frustrating here. As you have discovered, checks aren’t very compositional, so it’s not easy to implement a new check in terms of an existing one (while still providing good error messages on failure). If what you’re doing is difficult to express in RackUnit, consider just ditching it and doing your own thing. Test suites in Racket are just simple, side-effectful programs, and RackUnit doesn’t do that much all things considered. Alexis -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/6744FD8B-9427-44D0-A603-C040FB1CC07B%40gmail.com.
Re: [racket-users] rackunit and logging
Hi Shriram, I have a module, handy/test-more (https://pkgs.racket-lang.org/package/handy), that I think does everything you want; the downside is that the documentation is thorough but it's in the form of essay-style comment sections instead of Scribble. Breaking that out into actual Scribble is on my todo list but hasn't happened yet. You can read it on GitHub here https://github.com/dstorrs/racket-dstorrs-libs/blob/master/test-more.rkt If you think you would find this useful, I'll make a point to do the Scribble this weekend. Here's the synopsis statement: ;;== ;;The racket testing module has a few things I wish it did differently: ;; ;; 1) The test function names are verbose and redundant. check-this, check-that, etc ;; ;; 2) The test functions display nothing on success. There's no way ;; to tell the difference between "no tests ran" and "all tests ;; succeeded" ;; ;; 3) The tests return nothing. You can't do conditional tests like: ;;(unless (is os 'windows) (ok test-that-won't-pass-on-windows)) Some quick examples; there's quite a lot more that can be done with it if this looks useful. The output is shown after the code. #lang racket (require handy/test-more) (expect-n-tests 4) (test-suite "examples" (struct person (name age) #:transparent) (define people (hash 'alice (person "alice" 19) 'bob (person "bob" 17))) ; basic equality testing (is (* 1 2) 2 "(* 1 2) = 2") (is-false (hash-has-key? people 'charlie) "as expected, don't know charlie") (isnt (hash-has-key? people 'charlie) "same as above") ; Tests return values so you can chain them or nest them (lives (thunk (and (ok (hash-has-key? people 'alice) "we have an entry for alice") (let ([alice (hash-ref people 'alice)]) (is (person-age alice) 19 "alice is 19") (like (person-name alice) #px"^alice$" "alice's name is string, lowercase, trimmed" "alice-related tests did not raise exception") ; exceptions (define thnk (thunk (+ 1 "foo"))) (dies thnk "it died, that's all I care about") (throws thnk exn:fail:contract? "match against an arbitrary predicate") (throws thnk #px"expected: number.+\"foo\"" "match against a regex") (is (throws thnk (lambda (e) 'ok) "throws can pass exn to arbitrary one-arg proc for testing") 'ok #:op (lambda (got expected) (if (exn? got) 'ok 'nope)) "tested 'throws' with hand-rolled predicate, then chained into `is` using hand-rolled equality testing (the equality test is illustrative but not sensible)") ) When you run the above code, the following will be sent to STDOUT. Note that it tracks the number of tests it runs, numbers them, and warns you that you didn't run the expected number. (START test-suite: examples) ok 1 - (* 1 2) = 2 ok 2 - as expected, don't know charlie ok 3 ok 4 - we have an entry for alice ok 5 - alice is 19 ok 6 - alice's name is string, lowercase, trimmed ok 7 - alice-related tests did not raise exception ok 8 - it died, that's all I care about ok 9 - match against an arbitrary predicate ok 10 - match against a regex ok 11 - throws can pass exn to arbitrary one-arg proc for testing ok 12 - tested 'throws' with hand-rolled predicate, then chained into `is` using hand-rolled e$ ok 13 - test-suite completed without throwing uncaught exception Total tests passed so far: 13 Total tests failed so far: 0 (END test-suite: examples) !!ERROR!!: Expected 4 tests, actually saw 13 On Fri, May 22, 2020 at 7:47 PM Shriram Krishnamurthi wrote: > I'm trying to understand the design of the logging portion of rackunit: > > > https://docs.racket-lang.org/rackunit/Testing_Utilities.html#%28part._.Logging_.Test_.Results%29 > >1. The "log" seems to only be a *count*, not the actual rackunit >output (as the term "log" would suggest). Since I want to show a summary of >tests — including output — on a different medium, after the test suite has >run, it looks like I need to essentially create my own logging support? >(Perhaps the "check-info stack" is useful here, but I don't think so.) >2. Why do the check-… procedures not return any value? It would seem >natural for them to return, say, false in case of passing and a failure >information structure in case of failing (or a structure in both cases). >But they seem to only return void, so I'm not entirely sure how else to >extract the information for the log without rewriting the check's. >3. As an aside, I'm not entirely sure what `test-log!` is there for. >Presumably it's to record in the log "tests" run by operations that are not >part of rackunit? I'm curious how people have used it. > > TL;DR: If I want to record what happened on all the checks for > post-execution processing, do I need to (a) create my own log and, to do > so, (b) rewrite all the checking predicates to provide the
[racket-users] rackunit and logging
I'm trying to understand the design of the logging portion of rackunit: https://docs.racket-lang.org/rackunit/Testing_Utilities.html#%28part._.Logging_.Test_.Results%29 1. The "log" seems to only be a *count*, not the actual rackunit output (as the term "log" would suggest). Since I want to show a summary of tests — including output — on a different medium, after the test suite has run, it looks like I need to essentially create my own logging support? (Perhaps the "check-info stack" is useful here, but I don't think so.) 2. Why do the check-… procedures not return any value? It would seem natural for them to return, say, false in case of passing and a failure information structure in case of failing (or a structure in both cases). But they seem to only return void, so I'm not entirely sure how else to extract the information for the log without rewriting the check's. 3. As an aside, I'm not entirely sure what `test-log!` is there for. Presumably it's to record in the log "tests" run by operations that are not part of rackunit? I'm curious how people have used it. TL;DR: If I want to record what happened on all the checks for post-execution processing, do I need to (a) create my own log and, to do so, (b) rewrite all the checking predicates to provide the information that the detailed log needs? Thanks, Shriram -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/e7cafb9d-793d-4f3b-9eb4-308e43df73f2%40googlegroups.com.
