[racket-dev] Test results are slightly incomprehensible...
...if you're not used to them, that is. Hi, I am running the racket tests to know if something went wrong with my fork of racket. The fork was done at tag 2ef9f5ae. I run the tests with from the install directory with: , | racket$ bin/racket -r ./collects/tests/racket/quiet.rktl ` Attached, you can find the output I obtain for both my repository and the main repository. The diff looks like: Section(basic) Section(unicode) Section(rx) Section(reading) Section(readtable) Section(printing) Section(macro) Section(syntax) Section(procs) Section(stx) Section(module) Section(numbers) Section(unsafe) Section(object) Section(struct) Section(unit) Section(unit/sig) Section(threads) Section(logger) Section(synchronization) Section(deep) Section(continuation-marks) Section(prompt) Section(wills) Section(namespaces) Section(modprot) Section(chaperones) Section(parameters) Section(port) Section(file) Section(udp) Section(file-after-udp) Section(path) Section(optimization) Section(names) Section(setup) Section(for) Section(list) Section(math) Section(vector) Section(function) Section(dict) Section(contract) Section(fixnum) Section(flonum) Section(mpair) Section(etc) Section(structlib) Section(async-channel) Section(restart) Section(mzlib-string) Section(path) Section(filelib) Section(portlib) Section(threadlib) Section(sets) Section(date) Performed 35 expression tests (31 value expressions, 4 exn expressions) and 6 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) (Other messages report successful tests of error-handling behavior.) Section(compat) Section(command-line) Section(stream) Section(sequence) Section(generator) Section(pconvert) Section(pretty) Section(control) Section(serialization) Section(packages) Section(mzlib/contract) Section(sandbox) Performed 281 expression tests (281 value expressions, 0 exn expressions) and 0 exception field tests. Errors were: (Section (got expected (call))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) (Other messages report successful tests of error-handling behavior.) Section(shared) Section(kw) Section(macrolib) Section(resource) Section(syntax/) Performed 152701 expression tests (139300 value expressions, 13401 exn expressions) and 26923 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) (Other messages report successful tests of error-handling behavior.) Section(moddep) Section(boundmap) Section(id-table) Section(cm) Section(module-reader) Performed 153275 expression tests (139865 value expressions, 13410 exn expressions) and 26941 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) (Other messages report successful tests of error-handling behavior.) Section(version) Section(foreign) Section(uni-norm) Performed 505944 expression tests (492534 value expressions, 13410 exn expressions) and 26941 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) Section(basic) Section(unicode) Section(rx) Section(reading) Section(readtable) Section(printing) Section(macro) Section(syntax) Section(procs) Section(stx) Section(module) Section(numbers) Section(unsafe) Section(object) Section(struct) Section(unit) Section(unit/sig) Section(threads) Section(logger) Section(synchronization) Section(deep) Section(continuation-marks) Section(prompt) Section(wills) Section(namespaces) Section(modprot) Section(chaperones) Section(parameters) Section(port) Section(file) Section(udp) Section(file-after-udp) Section(path) Section(optimization) Section(names) Section(setup) Section(for) Section(list) Section(math) Section(vector) Section(function) Section(dict) Section(contract) Section(fixnum) Section(flonum) Section(mpair) Section(etc) Section(structlib) Section(async-channel) Section(restart) Section(mzlib-string) Section(path) Section(filelib) Section(portlib) Section(threadlib) Section(sets)
Re: [racket-dev] Test results are slightly incomprehensible...
The `with-limit' test is not completely deterministic. It's looking for an out-of-memory exception, and you got an out-of-time exception, instead. That sometimes happens if the machine is loaded or otherwise distracted. The `find-seconds' test may be wrong or it may be a bug. I see it for 64-bit builds, but I haven't had time to investigate, yet. The number of tests run can vary for many reasons. Sometimes the platform makes a difference, and probably 32-bit versus 64-bit makes a difference. Maybe there are some random numbers that determine how many times some tests are run. I don't think you should worry about it. One of the reasons the tests don't print more nicely is that they're really old. To a lesser extent, they're using minimal infrastructure to exercise the low-level functionality on which better test tools are implemented. At Wed, 28 Sep 2011 21:02:14 +0100, Paulo J. Matos wrote: ...if you're not used to them, that is. Hi, I am running the racket tests to know if something went wrong with my fork of racket. The fork was done at tag 2ef9f5ae. I run the tests with from the install directory with: , | racket$ bin/racket -r ./collects/tests/racket/quiet.rktl ` Attached, you can find the output I obtain for both my repository and the main repository. The diff looks like: Section(basic) Section(unicode) Section(rx) Section(reading) Section(readtable) Section(printing) Section(macro) Section(syntax) Section(procs) Section(stx) Section(module) Section(numbers) Section(unsafe) Section(object) Section(struct) Section(unit) Section(unit/sig) Section(threads) Section(logger) Section(synchronization) Section(deep) Section(continuation-marks) Section(prompt) Section(wills) Section(namespaces) Section(modprot) Section(chaperones) Section(parameters) Section(port) Section(file) Section(udp) Section(file-after-udp) Section(path) Section(optimization) Section(names) Section(setup) Section(for) Section(list) Section(math) Section(vector) Section(function) Section(dict) Section(contract) Section(fixnum) Section(flonum) Section(mpair) Section(etc) Section(structlib) Section(async-channel) Section(restart) Section(mzlib-string) Section(path) Section(filelib) Section(portlib) Section(threadlib) Section(sets) Section(date) Performed 35 expression tests (31 value expressions, 4 exn expressions) and 6 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) (Other messages report successful tests of error-handling behavior.) Section(compat) Section(command-line) Section(stream) Section(sequence) Section(generator) Section(pconvert) Section(pretty) Section(control) Section(serialization) Section(packages) Section(mzlib/contract) Section(sandbox) Performed 281 expression tests (281 value expressions, 0 exn expressions) and 0 exception field tests. Errors were: (Section (got expected (call))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) (Other messages report successful tests of error-handling behavior.) Section(shared) Section(kw) Section(macrolib) Section(resource) Section(syntax/) Performed 152701 expression tests (139300 value expressions, 13401 exn expressions) and 26923 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) (Other messages report successful tests of error-handling behavior.) Section(moddep) Section(boundmap) Section(id-table) Section(cm) Section(module-reader) Performed 153275 expression tests (139865 value expressions, 13410 exn expressions) and 26941 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run #procedure:...ket/sandbox.rktl:71:55))) ((sandbox) (#f #t (#procedure:run* #procedure:...ket/sandbox.rktl:67:55))) (Other messages report successful tests of error-handling behavior.) Section(version) Section(foreign) Section(uni-norm) Performed 505944 expression tests (492534 value expressions, 13410 exn expressions) and 26941 exception field tests. Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) ((sandbox) ((bad-exception-message: with-limit: out of time) #t (#procedure:e-match? out of memor(?:y) #procedure:run
Re: [racket-dev] Test results are slightly incomprehensible...
At Wed, 28 Sep 2011 21:02:14 +0100, Paulo J. Matos wrote: Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) After looking into this, I conclude that the test is faulty, but that it also highlights a bug/limitation on Windows: * The test assumes that the current timezone had daylight saving time starting at what would have been 2:00am on April 4, 1990. The test will fail on your machine if your timezone is set to a place with no or different DST rules (i.e., different than the standard US timezones). * Windows gets it wrong. It thinks that US Mountain Time started daylight saving sometime in March which is the right idea for the current DST calendar, but wrong for 1990. My guess is that timezone configuration in Windows cannot support different DST start-date rules for different years. I'll adjust the test, though I don't yet have a better idea than just removing it. _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Test results are slightly incomprehensible...
On Wed, Sep 28, 2011 at 6:04 PM, Matthew Flatt mfl...@cs.utah.edu wrote: At Wed, 28 Sep 2011 21:02:14 +0100, Paulo J. Matos wrote: Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) After looking into this, I conclude that the test is faulty, but that it also highlights a bug/limitation on Windows: * The test assumes that the current timezone had daylight saving time starting at what would have been 2:00am on April 4, 1990. The test will fail on your machine if your timezone is set to a place with no or different DST rules (i.e., different than the standard US timezones). Is there a way to check and see if the timezone is one of the US ones? * Windows gets it wrong. It thinks that US Mountain Time started daylight saving sometime in March which is the right idea for the current DST calendar, but wrong for 1990. My guess is that timezone configuration in Windows cannot support different DST start-date rules for different years. ... and not under windows, I guess? :) Robby I'll adjust the test, though I don't yet have a better idea than just removing it. _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Test results are slightly incomprehensible...
At Wed, 28 Sep 2011 18:12:34 -0500, Robby Findler wrote: On Wed, Sep 28, 2011 at 6:04 PM, Matthew Flatt mfl...@cs.utah.edu wrote: At Wed, 28 Sep 2011 21:02:14 +0100, Paulo J. Matos wrote: Errors were: (Section (got expected (call))) ((date) ((638931660) Error (find-seconds 0 1 2 1 4 1990))) After looking into this, I conclude that the test is faulty, but that it also highlights a bug/limitation on Windows: * The test assumes that the current timezone had daylight saving time starting at what would have been 2:00am on April 4, 1990. The test will fail on your machine if your timezone is set to a place with no or different DST rules (i.e., different than the standard US timezones). Is there a way to check and see if the timezone is one of the US ones? Not currently, but I guess I can add a way to get the current timezone name. _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
