Re: [PATCH] test: allow to use --full-sync
Michael J Gruber writes: > I don't mind carrying this locally and retrying without for the next > round of notmuch updates. Having the patch here on-list may help > someone else in the future in any case. OK, I suggest you do that for now. I'm not categorically opposed to (some version of) the change, but like Tomi I would like to understand what problem you are solving a bit better. I did try running the test suite on xfs (without bigtime set). This was amd64 so I couldn't exactly duplicate the conditions where you see failure. How much work is it to set up an environment that duplicates the failures? d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: allow to use --full-sync
On Wed, Feb 16 2022, Tomi Ollila wrote: > > I'd not merge this NOTMUCH_NEW_OPTIONS=--full-scan ;D too eagerly > ... I'd like we really knew the reason (which is then written in > commit message) I seem I did not get my message above (or why the ;D is there) expressed clear enough -- the idea there was that also NOTMUCH_NEW_OPTIONS=--full-scan make test would have worked (or, NOTMUCH_NEW_OPTIONS=--verbose, to get noisy output -- and lot of failed tests ;D) (to disable such a leakage of environment variable into test run NOTMUCH_NEW_OPTIONS= should have been there before checking that variable -- OTOH just adding NOTMUCH_NEW_OPTIONS=... as way to provide any options to notmuch new (for users to experiment) could be an option (or something...)) Anyway, I am not proposing adding such...now... but better concentrate figuring out what is happening there and why... > Tomi ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: allow to use --full-sync
On Wed, Feb 16 2022, Michael J. Gruber wrote: > David Bremner venit, vidit, dixit 2022-02-16 14:04:17: >> Michael J Gruber writes: >> >> > Some build infrastructure appears to habe problems with mtime/stat, >> >> have > > Huh, this looks as if my mother tongue slipped in, while it's just key > proximity. > >> > leading to spurious failures (noticed on s390x and aarch64 with epel8). >> > Allow the test suite to be run with --full-sync so that release builds >> > can use the test suite while avoiding spurious failures. >> > >> > Signed-off-by: Michael J Gruber >> > --- >> > test/README | 8 >> > test/test-lib-common.sh | 7 ++- >> > test/test-lib.sh| 2 +- >> > 3 files changed, 15 insertions(+), 2 deletions(-) >> > >> > diff --git a/test/README b/test/README >> > index 10f127cb..1de79b78 100644 >> > --- a/test/README >> > +++ b/test/README >> > @@ -110,6 +110,14 @@ printed on screen. This printing can be disabled by >> > setting the >> > NOTMUCH_TEST_QUIET variable to a non-null value. Message on test >> > failures and skips are still printed. >> >> typo in variable name. > > Not here, but in fact in the actual patch ;) > > >> How confident are you that this actually fixes your intermittent >> failures? There are quite a few literal invocations of "notmuch new" in >> the test suite. Would those need to be changed as well? > > About 80% confident ... The patch covers all areas in which I've ever > witnessed the spurious FAILs. I have not had a single failure with the > patch. Many of the other invocations are not immediately after message > creation. > > Some of them are immediately after, and - given that notmuch new catches > the case of same time - I do not understand why this happens at all. > That accounts for the other 20%. > > I don't mind carrying this locally and retrying without for the next > round of notmuch updates. Having the patch here on-list may help > someone else in the future in any case. I'd not merge this NOTMUCH_NEW_OPTIONS=--full-scan ;D too eagerly ... I'd like we really knew the reason (which is then written in commit message) > > Michael Tomi ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: allow to use --full-sync
David Bremner venit, vidit, dixit 2022-02-16 14:04:17: > Michael J Gruber writes: > > > Some build infrastructure appears to habe problems with mtime/stat, > > have Huh, this looks as if my mother tongue slipped in, while it's just key proximity. > > leading to spurious failures (noticed on s390x and aarch64 with epel8). > > Allow the test suite to be run with --full-sync so that release builds > > can use the test suite while avoiding spurious failures. > > > > Signed-off-by: Michael J Gruber > > --- > > test/README | 8 > > test/test-lib-common.sh | 7 ++- > > test/test-lib.sh| 2 +- > > 3 files changed, 15 insertions(+), 2 deletions(-) > > > > diff --git a/test/README b/test/README > > index 10f127cb..1de79b78 100644 > > --- a/test/README > > +++ b/test/README > > @@ -110,6 +110,14 @@ printed on screen. This printing can be disabled by > > setting the > > NOTMUCH_TEST_QUIET variable to a non-null value. Message on test > > failures and skips are still printed. > > typo in variable name. Not here, but in fact in the actual patch ;) > How confident are you that this actually fixes your intermittent > failures? There are quite a few literal invocations of "notmuch new" in > the test suite. Would those need to be changed as well? About 80% confident ... The patch covers all areas in which I've ever witnessed the spurious FAILs. I have not had a single failure with the patch. Many of the other invocations are not immediately after message creation. Some of them are immediately after, and - given that notmuch new catches the case of same time - I do not understand why this happens at all. That accounts for the other 20%. I don't mind carrying this locally and retrying without for the next round of notmuch updates. Having the patch here on-list may help someone else in the future in any case. Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] test: allow to use --full-sync
Michael J Gruber writes: > Some build infrastructure appears to habe problems with mtime/stat, have > leading to spurious failures (noticed on s390x and aarch64 with epel8). > Allow the test suite to be run with --full-sync so that release builds > can use the test suite while avoiding spurious failures. > > Signed-off-by: Michael J Gruber > --- > test/README | 8 > test/test-lib-common.sh | 7 ++- > test/test-lib.sh| 2 +- > 3 files changed, 15 insertions(+), 2 deletions(-) > > diff --git a/test/README b/test/README > index 10f127cb..1de79b78 100644 > --- a/test/README > +++ b/test/README > @@ -110,6 +110,14 @@ printed on screen. This printing can be disabled by > setting the > NOTMUCH_TEST_QUIET variable to a non-null value. Message on test > failures and skips are still printed. typo in variable name. How confident are you that this actually fixes your intermittent failures? There are quite a few literal invocations of "notmuch new" in the test suite. Would those need to be changed as well? d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] test: allow to use --full-sync
Some build infrastructure appears to habe problems with mtime/stat,
leading to spurious failures (noticed on s390x and aarch64 with epel8).
Allow the test suite to be run with --full-sync so that release builds
can use the test suite while avoiding spurious failures.
Signed-off-by: Michael J Gruber
---
test/README | 8
test/test-lib-common.sh | 7 ++-
test/test-lib.sh| 2 +-
3 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/test/README b/test/README
index 10f127cb..1de79b78 100644
--- a/test/README
+++ b/test/README
@@ -110,6 +110,14 @@ printed on screen. This printing can be disabled by
setting the
NOTMUCH_TEST_QUIET variable to a non-null value. Message on test
failures and skips are still printed.
+File system/mtime issues
+
+Normally, `notmuch new` uses directory modification times, and the test
+suite checks that this works. If you encounter spurious test failures
+where messages added by one subtest appear to show up in the next subtest
+only (indicating mtime/stat problems) you can set NOTMUCH_TEST_FULLSYNC
+so that the test suite uses the --full-sync option of notmuch new.
+
Skipping Tests
--
If, for any reason, you need to skip one or more tests, you can do so
diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index ebbf4cdf..91039478 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -29,6 +29,11 @@ if [[ -z "$NOTMUCH_SRCDIR" ]] || [[ -z "$NOTMUCH_BUILDDIR"
]]; then
exit 1
fi
+if test -n "$NOTMUCH_TEST_FULLSCAN"
+then
+ NOTMUCH_NEW_OPTIONS="--full-scan"
+fi
+
backup_database () {
test_name=$(basename $0 .sh)
rm -rf $TMP_DIRECTORY/notmuch-dir-backup."$test_name"
@@ -226,7 +231,7 @@ EOF
# are also supported here, so see that function for details.
add_message () {
generate_message "$@" &&
-notmuch new > /dev/null
+notmuch new $NOTMUCH_NEW_OPTIONS > /dev/null
}
if test -n "$valgrind"
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 833bf5fe..0167b95f 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -521,7 +521,7 @@ test_json_nodes () {
}
NOTMUCH_NEW () {
-notmuch new "${@}" | grep -v -E -e '^Processed [0-9]*( total)? file|Found
[0-9]* total file'
+notmuch new $NOTMUCH_NEW_OPTIONS "${@}" | grep -v -E -e '^Processed
[0-9]*( total)? file|Found [0-9]* total file'
}
NOTMUCH_DUMP_TAGS () {
--
2.35.1.476.gacce7e407c
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
