Re: Test suite timing issues?
> > > BTW, I have so far run the test suite 68 times in a row without failures > > > on a Debian s390x host. The file system is ext4, mounted relatime. It > > > would be interesting to know what file system is yielding the failures > > > Michael is seeing. > ... > So I'll try out a bit more, and if everything else fails, then running > the test suite with --full-scan is still better than not running it at > all. (I would have suggested a make variable, but it's really a corner > case. given the stat/mtime check). Last update for now: It's still happening sporadically only, and the archs where it does happen use xfs (relatime,seclabel,attr2) resp. tmpfs (nosuid,nodev,seclabel). It has to be something weird on our infra or specifically el8. So, either I implement `make test FULLSCAN=1` or patch in `--full-scan` or disable the tests on epel. Thanks for your patience :) Cheers, Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Tomi Ollila venit, vidit, dixit 2022-02-14 23:53:54: > On Mon, Feb 14 2022, David Bremner wrote: > > > Tomi Ollila writes: > > > >> > >> Looked notmuch-new.c -- time_t (seconds since epoch) is used as timestamp > >> comparisons (which would indicate the subsecond resolution most fs' provide > >> is not used)... > >> > >> ... and if so, I wonder why some of our tests are not failing all the time > >> for everyone...? > > > > Not claiming everything is fine, but there is code there targetted at > > the failure mode you mentioned: > > > > /* If the directory's mtime is the same as the wall-clock time > > * when we stat'ed the directory, we skip updating the mtime in > > * the database because a message could be delivered later in this > > * same second. This may lead to unnecessary re-scans, but it > > * avoids overlooking messages. */ > > if (fs_mtime != stat_time) > > _filename_list_add (state->directory_mtimes, path)->mtime = fs_mtime; > > This sure had to be tested... :D > > so I outcommented the line as // if (fs_mtime != stat_time) > > and then build and run test -- a lot of failures... > > also > > ./test/T750-gzip.sh 2>&1 | grep -e PASS -e FAIL > > PASS Single new gzipped message > PASS Single new gzipped message (full-scan) > FAIL Multiple new messages, one gzipped > FAIL Multiple new messages, one gzipped (full-scan) > FAIL Renamed (gzipped) message > PASS notmuch search with partially gzipped mail store > FAIL notmuch search --output=files with partially gzipped mail store > PASS show un-gzipped message > PASS show un-gzipped message (format mbox) > PASS show un-gzipped message (format raw) > FAIL show gzipped message > FAIL show gzipped message (mbox) > FAIL show gzipped message (raw) > PASS new doesn't run out of file descriptors with many gzipped files > > (above was "lucky" run, usually that 6th test, ...partially gzipped... > test also FAILed (I'd guess second happened to change there)). > > then restored the fs_mtime != stat_time line -- then all of 750 passed. > > (finally, run that 750-gzip in a loop (dropped last, slow test), hundreds > of times already -- no FAILures... (ecryptfs on ext4)) > > Tomi > > > BTW, I have so far run the test suite 68 times in a row without failures > > on a Debian s390x host. The file system is ext4, mounted relatime. It > > would be interesting to know what file system is yielding the failures > > Michael is seeing. Thanks for the detailed analysis. It convinces me that on notmuch's side everything is OK. This very much boils down to fs and stat issues. If I remember correctly, I've seen this isse once on a different release frpm epel-8 but only on epel-8 otherwise. This is on a build infrastructure where you may end up getting different hosts on each run, primed from the same "chroot". I'll try to find out more. Until now, we have been building notmuch release packages on Fedora without running the tests during the release build, because there used to be issues with the test suite (and before that that the separate test corpus download). With everything looking robust for a while, I'm switching tests on for Fedora release builds now, and at the same start packaging those extra packages for RedHat's enterprise platform. I want to avoid spurious release build failures, of course - otoh having those tests run is a good thing. So I'll try out a bit more, and if everything else fails, then running the test suite with --full-scan is still better than not running it at all. (I would have suggested a make variable, but it's really a corner case. given the stat/mtime check). Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
On Mon, Feb 14 2022, David Bremner wrote: > Tomi Ollila writes: > >> >> Looked notmuch-new.c -- time_t (seconds since epoch) is used as timestamp >> comparisons (which would indicate the subsecond resolution most fs' provide >> is not used)... >> >> ... and if so, I wonder why some of our tests are not failing all the time >> for everyone...? > > Not claiming everything is fine, but there is code there targetted at > the failure mode you mentioned: > > /* If the directory's mtime is the same as the wall-clock time > * when we stat'ed the directory, we skip updating the mtime in > * the database because a message could be delivered later in this > * same second. This may lead to unnecessary re-scans, but it > * avoids overlooking messages. */ > if (fs_mtime != stat_time) > _filename_list_add (state->directory_mtimes, path)->mtime = fs_mtime; This sure had to be tested... :D so I outcommented the line as // if (fs_mtime != stat_time) and then build and run test -- a lot of failures... also ./test/T750-gzip.sh 2>&1 | grep -e PASS -e FAIL PASS Single new gzipped message PASS Single new gzipped message (full-scan) FAIL Multiple new messages, one gzipped FAIL Multiple new messages, one gzipped (full-scan) FAIL Renamed (gzipped) message PASS notmuch search with partially gzipped mail store FAIL notmuch search --output=files with partially gzipped mail store PASS show un-gzipped message PASS show un-gzipped message (format mbox) PASS show un-gzipped message (format raw) FAIL show gzipped message FAIL show gzipped message (mbox) FAIL show gzipped message (raw) PASS new doesn't run out of file descriptors with many gzipped files (above was "lucky" run, usually that 6th test, ...partially gzipped... test also FAILed (I'd guess second happened to change there)). then restored the fs_mtime != stat_time line -- then all of 750 passed. (finally, run that 750-gzip in a loop (dropped last, slow test), hundreds of times already -- no FAILures... (ecryptfs on ext4)) Tomi > BTW, I have so far run the test suite 68 times in a row without failures > on a Debian s390x host. The file system is ext4, mounted relatime. It > would be interesting to know what file system is yielding the failures > Michael is seeing. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
On Mon, Feb 14 2022, David Bremner wrote: > Tomi Ollila writes: > >> >> Looked notmuch-new.c -- time_t (seconds since epoch) is used as timestamp >> comparisons (which would indicate the subsecond resolution most fs' provide >> is not used)... >> >> ... and if so, I wonder why some of our tests are not failing all the time >> for everyone...? > > Not claiming everything is fine, but there is code there targetted at > the failure mode you mentioned: > > /* If the directory's mtime is the same as the wall-clock time > * when we stat'ed the directory, we skip updating the mtime in > * the database because a message could be delivered later in this > * same second. This may lead to unnecessary re-scans, but it > * avoids overlooking messages. */ > if (fs_mtime != stat_time) > _filename_list_add (state->directory_mtimes, path)->mtime = > fs_mtime; Right. Very much explains why we don't see test failures... > BTW, I have so far run the test suite 68 times in a row without failures > on a Debian s390x host. The file system is ext4, mounted relatime. It > would be interesting to know what file system is yielding the failures > Michael is seeing. indeed. Tomi ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Tomi Ollila writes: > > Looked notmuch-new.c -- time_t (seconds since epoch) is used as timestamp > comparisons (which would indicate the subsecond resolution most fs' provide > is not used)... > > ... and if so, I wonder why some of our tests are not failing all the time > for everyone...? Not claiming everything is fine, but there is code there targetted at the failure mode you mentioned: /* If the directory's mtime is the same as the wall-clock time * when we stat'ed the directory, we skip updating the mtime in * the database because a message could be delivered later in this * same second. This may lead to unnecessary re-scans, but it * avoids overlooking messages. */ if (fs_mtime != stat_time) _filename_list_add (state->directory_mtimes, path)->mtime = fs_mtime; BTW, I have so far run the test suite 68 times in a row without failures on a Debian s390x host. The file system is ext4, mounted relatime. It would be interesting to know what file system is yielding the failures Michael is seeing. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
On Sun, Feb 13 2022, Tomi Ollila wrote: > On Sat, Feb 12 2022, David Bremner wrote: > >> Tomi Ollila writes: >> >>> >>> Does such a change hide "buggy" functionality ? >> >> We mostly don't use add_message, call notmuch new via NOTMUCH_NEW in >> T050-new.sh. So I think it would mostly not hide bugs in notmuch >> new. OTOH, I'm surprised the same issues with timestamps don't show up >> there, if that is really the problem. >> >>> Or do we consider notmuch new buggy if it does not notice all new messages >>> arrived every time ? >> >> That's a harder question. Maybe? But I don't know how serious a bug it >> is for actual users (who often get mail delivered in various concurrent >> ways). > > The bug could be that the filesystem mtime resolution is 1 sec (hmm what > is the resolution in database?), and 2 emails are delivered to one > directory during one second -- and at the same time notmuch new is scanning > emails and found only one of those. then no more emails are delivered to > that one directory for a while -- so notmuch new does not find those. > > (now that I think of it that could happen to me :O -- just very improbable > (or not, `stat .` outputs Modify: 2022-02-12 22:47:19.550838056 +0200) -- > have to check that timestamp resolution in database). Looked notmuch-new.c -- time_t (seconds since epoch) is used as timestamp comparisons (which would indicate the subsecond resolution most fs' provide is not used)... ... and if so, I wonder why some of our tests are not failing all the time for everyone...? Tomi > > Tomi ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
On Sat, Feb 12 2022, David Bremner wrote: > Tomi Ollila writes: > >> >> Does such a change hide "buggy" functionality ? > > We mostly don't use add_message, call notmuch new via NOTMUCH_NEW in > T050-new.sh. So I think it would mostly not hide bugs in notmuch > new. OTOH, I'm surprised the same issues with timestamps don't show up > there, if that is really the problem. > >> Or do we consider notmuch new buggy if it does not notice all new messages >> arrived every time ? > > That's a harder question. Maybe? But I don't know how serious a bug it > is for actual users (who often get mail delivered in various concurrent > ways). The bug could be that the filesystem mtime resolution is 1 sec (hmm what is the resolution in database?), and 2 emails are delivered to one directory during one second -- and at the same time notmuch new is scanning emails and found only one of those. then no more emails are delivered to that one directory for a while -- so notmuch new does not find those. (now that I think of it that could happen to me :O -- just very improbable (or not, `stat .` outputs Modify: 2022-02-12 22:47:19.550838056 +0200) -- have to check that timestamp resolution in database). Tomi ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Am Sa., 12. Feb. 2022 um 22:10 Uhr schrieb David Bremner : > Tomi Ollila writes: > > > > > Does such a change hide "buggy" functionality ? > > We mostly don't use add_message, call notmuch new via NOTMUCH_NEW in > T050-new.sh. So I think it would mostly not hide bugs in notmuch > new. OTOH, I'm surprised the same issues with timestamps don't show up > there, if that is really the problem. > > And sure enough, I got a FAIL from NOTMUCH_NEW in T750-gzip just a few minutes ago. Trying with --full-scan there, too now. Alternatively, I'll just turn off tests again for package builds (like we had before)... ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Tomi Ollila writes: > > Does such a change hide "buggy" functionality ? We mostly don't use add_message, call notmuch new via NOTMUCH_NEW in T050-new.sh. So I think it would mostly not hide bugs in notmuch new. OTOH, I'm surprised the same issues with timestamps don't show up there, if that is really the problem. > Or do we consider notmuch new buggy if it does not notice all new messages > arrived every time ? That's a harder question. Maybe? But I don't know how serious a bug it is for actual users (who often get mail delivered in various concurrent ways). ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Am Sa., 12. Feb. 2022 um 21:45 Uhr schrieb Tomi Ollila : > On Sat, Feb 12 2022, David Bremner wrote: > > > Tomi Ollila writes: > > > >> On Sat, Feb 12 2022, Michael J. Gruber wrote: > >> > >> Only thing that came into mind are directory timestamps... if directory > >> (m)time is same as before notmuch will not scan it for files... > >> > >> ... following that if the granularity of directory timestamp were 1 > second, > >> then it could easily happen than first one new message is not seen, and > >> next time there is one extra message to be see... > > > > What do you think about adding --full-scan to the notmuch-new invocation > > in add_message? It doesn't make any tests fail and is about the same > > speed. I need to do a few more trials, but first time through it was > > actually faster (!), maybe because the cache is hot > > Does such a change hide "buggy" functionality ? > > Or do we consider notmuch new buggy if it does not notice all new messages > arrived every time ? > > The timestamping sounds like a perfect explanation of what I've been seeing. Unfortunately, I can't reproduce the issue "reliably" (with a certain probability), and so if everything succeeds with --full-scan 10 times it still does not mean much. As I understand, notmuch new without --full-sync may have issues when the time resolution is too low (or operations too fast) and will pick a message on the next run, so it's not really buggy - it uses a shortcut that may be too quick but does not loose messages in the long run. Michael > > ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
On Sat, Feb 12 2022, David Bremner wrote: > Tomi Ollila writes: > >> On Sat, Feb 12 2022, Michael J. Gruber wrote: >> >> Only thing that came into mind are directory timestamps... if directory >> (m)time is same as before notmuch will not scan it for files... >> >> ... following that if the granularity of directory timestamp were 1 second, >> then it could easily happen than first one new message is not seen, and >> next time there is one extra message to be see... > > What do you think about adding --full-scan to the notmuch-new invocation > in add_message? It doesn't make any tests fail and is about the same > speed. I need to do a few more trials, but first time through it was > actually faster (!), maybe because the cache is hot Does such a change hide "buggy" functionality ? Or do we consider notmuch new buggy if it does not notice all new messages arrived every time ? Tomi ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
David Bremner writes: > Tomi Ollila writes: > >> On Sat, Feb 12 2022, Michael J. Gruber wrote: >> >> Only thing that came into mind are directory timestamps... if directory >> (m)time is same as before notmuch will not scan it for files... >> >> ... following that if the granularity of directory timestamp were 1 second, >> then it could easily happen than first one new message is not seen, and >> next time there is one extra message to be see... > > What do you think about adding --full-scan to the notmuch-new invocation > in add_message? It doesn't make any tests fail and is about the same > speed. I need to do a few more trials, but first time through it was > actually faster (!), maybe because the cache is hot OK, I did 10 repetitions, and for the sequential version of the test suit it is about 2% slower in elapsed time. I would expect it to have more impact on slower disks (this SSD), but still not too bad of a slowdown. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Tomi Ollila writes: > On Sat, Feb 12 2022, Michael J. Gruber wrote: > > Only thing that came into mind are directory timestamps... if directory > (m)time is same as before notmuch will not scan it for files... > > ... following that if the granularity of directory timestamp were 1 second, > then it could easily happen than first one new message is not seen, and > next time there is one extra message to be see... What do you think about adding --full-scan to the notmuch-new invocation in add_message? It doesn't make any tests fail and is about the same speed. I need to do a few more trials, but first time through it was actually faster (!), maybe because the cache is hot ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
On Sat, Feb 12 2022, Michael J. Gruber wrote: > David Bremner venit, vidit, dixit 2022-02-12 01:03:00: >> Michael J Gruber writes: >> >> > Hi there, >> > >> > I'm trying to package notmuch for Redhat's enterprise linux and clones >> > (EPEL, extra packages for enterprise linux). >> > >> > This looks mostly fine, including the tests, except for intermittent >> > failures on epel-8-s390x. They look like the below, or in tests >> > following those, and apparantly all have to do with "db not synced yet" >> > or such, so that a message one subtest creates only shows up in the db >> > for the next subtest. >> > >> > I've also had completely fine test runs on epel-8-s390x, but I'm >> > starting to wonder whether I've just been lucky so far on other >> > platforms ... Could it be possible that generate_message(), even though >> > adding the message, still returns false and therefore add_message() does >> > not call "notmuch new"? One might want to drop the "&&" there in >> > test-lib-common, I dunno. This shouldn't be "intermittent". >> >> It's hard to see what can go wrong (maybe perl doesn't work?), but >> failing to generate a message should be a fatal error. Maybe try >> something like >> > > Well, as you can see in both reports, the pattern is as follows: > > The first subttest sees 1 less message than expected. > The second sees 1 more message than expected. Only thing that came into mind are directory timestamps... if directory (m)time is same as before notmuch will not scan it for files... ... following that if the granularity of directory timestamp were 1 second, then it could easily happen than first one new message is not seen, and next time there is one extra message to be see... Just wild guesses... Tomi > > So in summary, they are generated but "notmuch new" is not run or does > not pick them up. Though it's a Heisenbug on a specific arch, I'll try > your suggestion and hammer our infra with it ... ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Michael J Gruber writes: >> >> It's hard to see what can go wrong (maybe perl doesn't work?), but >> failing to generate a message should be a fatal error. Maybe try >> something like I was unclear. I meant "should" as in "in a perfect world", not as a prediction about the current code. So we should probably try to add some error checking either where the patch put it, or in generate_message itself. >> > > Well, as you can see in both reports, the pattern is as follows: > > The first subttest sees 1 less message than expected. > The second sees 1 more message than expected. > > So in summary, they are generated but "notmuch new" is not run or does > not pick them up. Though it's a Heisenbug on a specific arch, I'll try > your suggestion and hammer our infra with it ... > Yeah, debugging by batch submission is no fun, no matter what our grandparents tell us. I do have interactive access to s390x systems running Debian and Ubuntu, but the problems you report do not occur in Debian (probably not in Ubuntu either, or I would have heard about it, but I didn't check if e.g. Ubuntu disables the tests). There is not (supposed to be) anything asynchronous about the process of generating a message and indexing it. You could also try adding --full-scan to the relevant "notmuch new" invocation. This will disable the heuristics based on directory mtimes that notmuch uses to speed up indexing. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
David Bremner venit, vidit, dixit 2022-02-12 01:03:00:
> Michael J Gruber writes:
>
> > Hi there,
> >
> > I'm trying to package notmuch for Redhat's enterprise linux and clones
> > (EPEL, extra packages for enterprise linux).
> >
> > This looks mostly fine, including the tests, except for intermittent
> > failures on epel-8-s390x. They look like the below, or in tests
> > following those, and apparantly all have to do with "db not synced yet"
> > or such, so that a message one subtest creates only shows up in the db
> > for the next subtest.
> >
> > I've also had completely fine test runs on epel-8-s390x, but I'm
> > starting to wonder whether I've just been lucky so far on other
> > platforms ... Could it be possible that generate_message(), even though
> > adding the message, still returns false and therefore add_message() does
> > not call "notmuch new"? One might want to drop the "&&" there in
> > test-lib-common, I dunno. This shouldn't be "intermittent".
>
> It's hard to see what can go wrong (maybe perl doesn't work?), but
> failing to generate a message should be a fatal error. Maybe try
> something like
>
Well, as you can see in both reports, the pattern is as follows:
The first subttest sees 1 less message than expected.
The second sees 1 more message than expected.
So in summary, they are generated but "notmuch new" is not run or does
not pick them up. Though it's a Heisenbug on a specific arch, I'll try
your suggestion and hammer our infra with it ...
> diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
> index ebbf4cdf..076777ca 100644
> --- a/test/test-lib-common.sh
> +++ b/test/test-lib-common.sh
> @@ -225,7 +225,7 @@ EOF
> # All of the arguments and return values supported by generate_message
> # are also supported here, so see that function for details.
> add_message () {
> -generate_message "$@" &&
> +generate_message "$@" || error "failed to generate message"
> notmuch new > /dev/null
> }
>
>
> to see if you get more information
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Michael J Gruber writes:
> Hi there,
>
> I'm trying to package notmuch for Redhat's enterprise linux and clones
> (EPEL, extra packages for enterprise linux).
>
> This looks mostly fine, including the tests, except for intermittent
> failures on epel-8-s390x. They look like the below, or in tests
> following those, and apparantly all have to do with "db not synced yet"
> or such, so that a message one subtest creates only shows up in the db
> for the next subtest.
>
> I've also had completely fine test runs on epel-8-s390x, but I'm
> starting to wonder whether I've just been lucky so far on other
> platforms ... Could it be possible that generate_message(), even though
> adding the message, still returns false and therefore add_message() does
> not call "notmuch new"? One might want to drop the "&&" there in
> test-lib-common, I dunno. This shouldn't be "intermittent".
It's hard to see what can go wrong (maybe perl doesn't work?), but
failing to generate a message should be a fatal error. Maybe try
something like
diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index ebbf4cdf..076777ca 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -225,7 +225,7 @@ EOF
# All of the arguments and return values supported by generate_message
# are also supported here, so see that function for details.
add_message () {
-generate_message "$@" &&
+generate_message "$@" || error "failed to generate message"
notmuch new > /dev/null
}
to see if you get more information
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Test suite timing issues?
Michael J Gruber venit, vidit, dixit 2022-02-11 14:55:32: > Hi there, > > I'm trying to package notmuch for Redhat's enterprise linux and clones > (EPEL, extra packages for enterprise linux). > > This looks mostly fine, including the tests, except for intermittent > failures on epel-8-s390x. They look like the below, or in tests > following those, and apparantly all have to do with "db not synced yet" > or such, so that a message one subtest creates only shows up in the db > for the next subtest. > > I've also had completely fine test runs on epel-8-s390x, but I'm > starting to wonder whether I've just been lucky so far on other > platforms ... Could it be possible that generate_message(), even though > adding the message, still returns false and therefore add_message() does > not call "notmuch new"? One might want to drop the "&&" there in > test-lib-common, I dunno. This shouldn't be "intermittent". > > Michael > > T210-raw: Testing notmuch show --format=raw > FAIL Attempt to show multiple raw messages > --- T210-raw.1.expected 2022-02-11 13:22:28.011556841 + > +++ T210-raw.1.output 2022-02-11 13:22:28.011556841 + > @@ -1 +1,7 @@ > -Error: search term did not match precisely one message (matched 2 > messages). > +From: Notmuch Test Suite > +To: Notmuch Test Suite > +Message-Id: > +Subject: Test message #1 > +Date: Fri, 05 Jan 2001 15:43:56 + > + > +This is just a test message (#1) > PASS Show a raw message > FAIL Show another raw message > --- T210-raw.3.expected 2022-02-11 13:22:28.031556841 + > +++ T210-raw.3.output 2022-02-11 13:22:28.031556841 + > @@ -1,7 +1 @@ > -From: Notmuch Test Suite > -To: Notmuch Test Suite > -Message-Id: > -Subject: Test message #2 > -Date: GENERATED_DATE > > -This is just a test message (#2) > Error: search term did not match precisely one message (matched 0 messages). Similar apparent timing issue on epel-8-aarch64: FAIL Adding message with long ID --- T290-long-id.2.expected 2022-02-11 13:55:17.028415326 + +++ T290-long-id.2.output 2022-02-11 13:55:17.028415326 + @@ -1 +1 @@ -Added 1 new message to the database. +No new mail. FAIL Referencing long ID after adding --- T290-long-id.3.expected 2022-02-11 13:55:17.078415753 + +++ T290-long-id.3.output 2022-02-11 13:55:17.078415753 + @@ -1 +1 @@ -Added 1 new message to the database. +Added 2 new messages to the database. Note how that message from the 1st subtest shows up only in the second. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Test suite timing issues?
Hi there, I'm trying to package notmuch for Redhat's enterprise linux and clones (EPEL, extra packages for enterprise linux). This looks mostly fine, including the tests, except for intermittent failures on epel-8-s390x. They look like the below, or in tests following those, and apparantly all have to do with "db not synced yet" or such, so that a message one subtest creates only shows up in the db for the next subtest. I've also had completely fine test runs on epel-8-s390x, but I'm starting to wonder whether I've just been lucky so far on other platforms ... Could it be possible that generate_message(), even though adding the message, still returns false and therefore add_message() does not call "notmuch new"? One might want to drop the "&&" there in test-lib-common, I dunno. This shouldn't be "intermittent". Michael T210-raw: Testing notmuch show --format=raw FAIL Attempt to show multiple raw messages --- T210-raw.1.expected 2022-02-11 13:22:28.011556841 + +++ T210-raw.1.output 2022-02-11 13:22:28.011556841 + @@ -1 +1,7 @@ -Error: search term did not match precisely one message (matched 2 messages). +From: Notmuch Test Suite +To: Notmuch Test Suite +Message-Id: +Subject: Test message #1 +Date: Fri, 05 Jan 2001 15:43:56 + + +This is just a test message (#1) PASS Show a raw message FAIL Show another raw message --- T210-raw.3.expected 2022-02-11 13:22:28.031556841 + +++ T210-raw.3.output 2022-02-11 13:22:28.031556841 + @@ -1,7 +1 @@ -From: Notmuch Test Suite -To: Notmuch Test Suite -Message-Id: -Subject: Test message #2 -Date: GENERATED_DATE -This is just a test message (#2) Error: search term did not match precisely one message (matched 0 messages). ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
