Re: [PATCH] test: aggregate-results updates
Tomi Ollila writes: > On Mon, Aug 02 2021, David Bremner wrote: > >> Tomi Ollila writes: >> >>> - mkdir -p "$test_results_dir" >>> + test -d "$test_results_dir" || mkdir "$test_results_dir" >> >> Lately I've notice some complaints during parallel test running >> >> mkdir: cannot create directory >> '/home/bremner/software/upstream/notmuch/test/test-results': File exists >> >> It seems like this change might be the culprit. Can you explain why it >> was needed? > > ah, parallel test non-atomicity is the problem; fix would be to restore -p to > mkdir > (so it does not fail when another program executed mkdir between that test > check and mkdir execution) > Switched back to mkdir -p here, the occasional test failure is worse than the potential slowdown. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: aggregate-results updates
Tomi Ollila writes: > - mkdir -p "$test_results_dir" > + test -d "$test_results_dir" || mkdir "$test_results_dir" Lately I've notice some complaints during parallel test running mkdir: cannot create directory '/home/bremner/software/upstream/notmuch/test/test-results': File exists It seems like this change might be the culprit. Can you explain why it was needed? ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: aggregate-results updates
Tomi Ollila writes: > > Should such an extremely rare cases happen it should be enough just to > report it so it get noticed (and not add any extra code to handle such > cases any further). One checking the results would probably guess that > if output is as nonsensical as it is something went badly wrong. > > Tomi > Patch applied to master. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: aggregate-results updates
On Sat, Jun 05 2021, David Bremner wrote: > Tomi Ollila writes: >> >> testing all changed branches is somewhat hard; I did by adding >> sleep 20 and file what effective removed created results files; >> another change was to change 'total' to 'toxtal' in test_done, >> then I run (at least) the following command lines: > > changing test_done results in pretty nonsensical output. I guess that's > OK? Yes. It is very unlikely that if something is written to file not all data were written (as the amount of data to be written is low). A bit less unlikely is the case that output file is created but no data written. Should such an extremely rare cases happen it should be enough just to report it so it get noticed (and not add any extra code to handle such cases any further). One checking the results would probably guess that if output is as nonsensical as it is something went badly wrong. Tomi > > $ NOTMUCH_TESTS=T310-emacs.sh ./notmuch-test > > ... > > '/home/bremner/software/upstream/notmuch/test/test-results/T310-emacs' lacks > 'total ...'; results may be inconsistent. > Notmuch test suite complete. > 72/0 tests passed. > 1 test failed. > -73 tests skipped. > ___ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: aggregate-results updates
Tomi Ollila writes: > > testing all changed branches is somewhat hard; I did by adding > sleep 20 and file what effective removed created results files; > another change was to change 'total' to 'toxtal' in test_done, > then I run (at least) the following command lines: changing test_done results in pretty nonsensical output. I guess that's OK? $ NOTMUCH_TESTS=T310-emacs.sh ./notmuch-test ... '/home/bremner/software/upstream/notmuch/test/test-results/T310-emacs' lacks 'total ...'; results may be inconsistent. Notmuch test suite complete. 72/0 tests passed. 1 test failed. -73 tests skipped. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
