The `with-limit' test is not completely deterministic. It's looking for an out-of-memory exception, and you got an out-of-time exception, instead. That sometimes happens if the machine is loaded or otherwise distracted.
The `find-seconds' test may be wrong or it may be a bug. I see it for 64-bit builds, but I haven't had time to investigate, yet. The number of tests run can vary for many reasons. Sometimes the platform makes a difference, and probably 32-bit versus 64-bit makes a difference. Maybe there are some random numbers that determine how many times some tests are run. I don't think you should worry about it. One of the reasons the tests don't print more nicely is that they're really old. To a lesser extent, they're using minimal infrastructure to exercise the low-level functionality on which better test tools are implemented. At Wed, 28 Sep 2011 21:02:14 +0100, Paulo J. Matos wrote: > ...if you're not used to them, that is. > > Hi, > > I am running the racket tests to know if something went wrong with my > fork of racket. The fork was done at tag 2ef9f5ae. > > I run the tests with from the install directory with: > ,---- > | racket$ bin/racket -r ./collects/tests/racket/quiet.rktl > `---- > > Attached, you can find the output I obtain for both my repository and > the main repository. The diff looks like: > Section(basic) > Section(unicode) > Section(rx) > Section(reading) > Section(readtable) > Section(printing) > Section(macro) > Section(syntax) > Section(procs) > Section(stx) > Section(module) > Section(numbers) > Section(unsafe) > Section(object) > Section(struct) > Section(unit) > Section(unit/sig) > Section(threads) > Section(logger) > Section(synchronization) > Section(deep) > Section(continuation-marks) > Section(prompt) > Section(wills) > Section(namespaces) > Section(modprot) > Section(chaperones) > Section(parameters) > Section(port) > Section(file) > Section(udp) > Section(file-after-udp) > Section(path) > Section(optimization) > Section(names) > Section(setup) > Section(for) > Section(list) > Section(math) > Section(vector) > Section(function) > Section(dict) > Section(contract) > Section(fixnum) > Section(flonum) > Section(mpair) > Section(etc) > Section(structlib) > Section(async-channel) > Section(restart) > Section(mzlib-string) > Section(path) > Section(filelib) > Section(portlib) > Section(threadlib) > Section(sets) > Section(date) > > Performed 35 expression tests (31 value expressions, 4 exn expressions) > and 6 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > (Other messages report successful tests of error-handling behavior.) > Section(compat) > Section(command-line) > Section(stream) > Section(sequence) > Section(generator) > Section(pconvert) > Section(pretty) > Section(control) > Section(serialization) > Section(packages) > Section(mzlib/contract) > Section(sandbox) > > Performed 281 expression tests (281 value expressions, 0 exn expressions) > and 0 exception field tests. > > Errors were: > (Section (got expected (call))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > (Other messages report successful tests of error-handling behavior.) > Section(shared) > Section(kw) > Section(macrolib) > Section(resource) > Section(syntax/....) > > Performed 152701 expression tests (139300 value expressions, 13401 exn > expressions) > and 26923 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > (Other messages report successful tests of error-handling behavior.) > Section(moddep) > Section(boundmap) > Section(id-table) > Section(cm) > Section(module-reader) > > Performed 153275 expression tests (139865 value expressions, 13410 exn > expressions) > and 26941 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > (Other messages report successful tests of error-handling behavior.) > Section(version) > Section(foreign) > Section(uni-norm) > > Performed 505944 expression tests (492534 value expressions, 13410 exn > expressions) > and 26941 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > Section(basic) > Section(unicode) > Section(rx) > Section(reading) > Section(readtable) > Section(printing) > Section(macro) > Section(syntax) > Section(procs) > Section(stx) > Section(module) > Section(numbers) > Section(unsafe) > Section(object) > Section(struct) > Section(unit) > Section(unit/sig) > Section(threads) > Section(logger) > Section(synchronization) > Section(deep) > Section(continuation-marks) > Section(prompt) > Section(wills) > Section(namespaces) > Section(modprot) > Section(chaperones) > Section(parameters) > Section(port) > Section(file) > Section(udp) > Section(file-after-udp) > Section(path) > Section(optimization) > Section(names) > Section(setup) > Section(for) > Section(list) > Section(math) > Section(vector) > Section(function) > Section(dict) > Section(contract) > Section(fixnum) > Section(flonum) > Section(mpair) > Section(etc) > Section(structlib) > Section(async-channel) > Section(restart) > Section(mzlib-string) > Section(path) > Section(filelib) > Section(portlib) > Section(threadlib) > Section(sets) > Section(date) > > Performed 35 expression tests (31 value expressions, 4 exn expressions) > and 6 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > (Other messages report successful tests of error-handling behavior.) > Section(compat) > Section(command-line) > Section(stream) > Section(sequence) > Section(generator) > Section(pconvert) > Section(pretty) > Section(control) > Section(serialization) > Section(packages) > Section(mzlib/contract) > Section(sandbox) > > Performed 281 expression tests (281 value expressions, 0 exn expressions) > and 0 exception field tests. > > Errors were: > (Section (got expected (call))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > (Other messages report successful tests of error-handling behavior.) > Section(shared) > Section(kw) > Section(macrolib) > Section(resource) > Section(syntax/....) > > Performed 152694 expression tests (139295 value expressions, 13399 exn > expressions) > and 26919 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > (Other messages report successful tests of error-handling behavior.) > Section(moddep) > Section(boundmap) > Section(id-table) > Section(cm) > Section(module-reader) > > Performed 153268 expression tests (139860 value expressions, 13408 exn > expressions) > and 26937 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > (Other messages report successful tests of error-handling behavior.) > Section(version) > Section(foreign) > Section(uni-norm) > > Performed 505937 expression tests (492529 value expressions, 13408 exn > expressions) > and 26937 exception field tests. > > Errors were: > (Section (got expected (call))) > ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) > ((sandbox) ((bad-exception-message: "with-limit: out of time") #t > (#<procedure:e-match?> "out of memor(?:y)" #<procedure:run> > #<procedure:...ket/sandbox.rktl:71:55>))) > ((sandbox) (#f #t (#<procedure:run*> #<procedure:...ket/sandbox.rktl:67:55>))) > ,---- > | --- /home/pmatos/repositories/racket/test_results.txt 2011-09-28 > 20:44:41.000000000 +0100 > | +++ test_results.txt 2011-09-28 20:51:12.000000000 +0100 > | @@ -90,8 +90,8 @@ > | Section(resource) > | Section(syntax/....) > | > | -Performed 152701 expression tests (139300 value expressions, 13401 exn > expressions) > | -and 26923 exception field tests. > | +Performed 152694 expression tests (139295 value expressions, 13399 exn > expressions) > | +and 26919 exception field tests. > | > | Errors were: > | (Section (got expected (call))) > | @@ -105,8 +105,8 @@ > | Section(cm) > | Section(module-reader) > | > | -Performed 153275 expression tests (139865 value expressions, 13410 exn > expressions) > | -and 26941 exception field tests. > | +Performed 153268 expression tests (139860 value expressions, 13408 exn > expressions) > | +and 26937 exception field tests. > | > | Errors were: > | (Section (got expected (call))) > | @@ -118,8 +118,8 @@ > | Section(foreign) > | Section(uni-norm) > | > | -Performed 505944 expression tests (492534 value expressions, 13410 exn > expressions) > | -and 26941 exception field tests. > | +Performed 505937 expression tests (492529 value expressions, 13408 exn > expressions) > | +and 26937 exception field tests. > | > | Errors were: > | (Section (got expected (call))) > `---- > > Some of the interesting things are: > - My fork seems to, after `Section(syntax/....)', perform less > expression tests and less exception field tests. Why is this? > - Same applies for all the other performed tests that were reported. > > Now, what I don't understand is that I was expecting something like: > ,---- > | 10 tests PASS, 5 tests FAIL, 2 tests CHECK, ... > | <list of failed tests> > `---- > > How do I go about checking what failed or not and where is the code that > failed? > > Cheers, > > -- > PMatos > _________________________________________________ > For list-related administrative tasks: > http://lists.racket-lang.org/listinfo/dev _________________________________________________ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
