Tomi Ollila venit, vidit, dixit 2022-02-14 23:53:54: > On Mon, Feb 14 2022, David Bremner wrote: > > > Tomi Ollila <tomi.oll...@iki.fi> writes: > > > >> > >> Looked notmuch-new.c -- time_t (seconds since epoch) is used as timestamp > >> comparisons (which would indicate the subsecond resolution most fs' provide > >> is not used)... > >> > >> ... and if so, I wonder why some of our tests are not failing all the time > >> for everyone...? > > > > Not claiming everything is fine, but there is code there targetted at > > the failure mode you mentioned: > > > > /* If the directory's mtime is the same as the wall-clock time > > * when we stat'ed the directory, we skip updating the mtime in > > * the database because a message could be delivered later in this > > * same second. This may lead to unnecessary re-scans, but it > > * avoids overlooking messages. */ > > if (fs_mtime != stat_time) > > _filename_list_add (state->directory_mtimes, path)->mtime = fs_mtime; > > This sure had to be tested... :D > > so I outcommented the line as // if (fs_mtime != stat_time) > > and then build and run test -- a lot of failures... > > also > > ./test/T750-gzip.sh 2>&1 | grep -e PASS -e FAIL > > PASS Single new gzipped message > PASS Single new gzipped message (full-scan) > FAIL Multiple new messages, one gzipped > FAIL Multiple new messages, one gzipped (full-scan) > FAIL Renamed (gzipped) message > PASS notmuch search with partially gzipped mail store > FAIL notmuch search --output=files with partially gzipped mail store > PASS show un-gzipped message > PASS show un-gzipped message (format mbox) > PASS show un-gzipped message (format raw) > FAIL show gzipped message > FAIL show gzipped message (mbox) > FAIL show gzipped message (raw) > PASS new doesn't run out of file descriptors with many gzipped files > > (above was "lucky" run, usually that 6th test, ...partially gzipped... > test also FAILed (I'd guess second happened to change there)). > > then restored the fs_mtime != stat_time line -- then all of 750 passed. > > (finally, run that 750-gzip in a loop (dropped last, slow test), hundreds > of times already -- no FAILures... (ecryptfs on ext4)) > > Tomi > > > BTW, I have so far run the test suite 68 times in a row without failures > > on a Debian s390x host. The file system is ext4, mounted relatime. It > > would be interesting to know what file system is yielding the failures > > Michael is seeing.
Thanks for the detailed analysis. It convinces me that on notmuch's side everything is OK. This very much boils down to fs and stat issues. If I remember correctly, I've seen this isse once on a different release frpm epel-8 but only on epel-8 otherwise. This is on a build infrastructure where you may end up getting different hosts on each run, primed from the same "chroot". I'll try to find out more. Until now, we have been building notmuch release packages on Fedora without running the tests during the release build, because there used to be issues with the test suite (and before that that the separate test corpus download). With everything looking robust for a while, I'm switching tests on for Fedora release builds now, and at the same start packaging those extra packages for RedHat's enterprise platform. I want to avoid spurious release build failures, of course - otoh having those tests run is a good thing. So I'll try out a bit more, and if everything else fails, then running the test suite with --full-scan is still better than not running it at all. (I would have suggested a make variable, but it's really a corner case. given the stat/mtime check). Michael _______________________________________________ notmuch mailing list -- email@example.com To unsubscribe send an email to notmuch-le...@notmuchmail.org