Re: [PATCH] config: Inform user if config file is broken
On Wed, Sep 06, 2023 at 12:48:15PM -0300, David Bremner wrote: > Eric Blake writes: > > > > > I'm not sure if this is the best approach (as this is my first ever > > patch to notmuch), but it's better than nothing. > > Unfortunately we can't just print from there because it is in a shared > library (whose clients might not appreciate output). Something _almost_ > equivalent can be done with _notmuch_database_log, but that still > requires the caller to read those logs with > notmuch_database_status_string. I'm out of time to spend further on this bug today; if you would like to take the ideas in my patch and rework it into something usable, be my guest. Otherwise, I might be able to return to this bug later in the week to see if I can figure out how to grab the database_log at the right point when status is NOTMUCH_STATUS_FILE_ERROR is returned (open.cc:notmuch_database_load_config DONE label looks like it should be able to grab from the database log if status_string is present). -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] config: Inform user if config file is broken
Eric Blake writes: > > I'm not sure if this is the best approach (as this is my first ever > patch to notmuch), but it's better than nothing. Unfortunately we can't just print from there because it is in a shared library (whose clients might not appreciate output). Something _almost_ equivalent can be done with _notmuch_database_log, but that still requires the caller to read those logs with notmuch_database_status_string. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: notmuch breaks on \. in config file with upgrade from glib2 2.76.1 to 2.76.5
On Wed, Sep 06, 2023 at 04:37:57PM +0200, Michael J Gruber wrote: > Hi there > > [snip] > > Last night, I filed > > https://bugzilla.redhat.com/show_bug.cgi?id=2237562. Later, I found > > this about glib 2.77 introducing regressions: > > https://bugzilla.redhat.com/show_bug.cgi?id=2225257; looks like Fedora > > backported enough of that into 2.76.5 to cause similar issues in > > relation to 2.76.1, even though a patchlevel release of glib shouldn't > > be changing behaviors. > > Fedora has no related patches in 2.76.5-1 at all (only hmac/eperm). > So, if that's the same regression as in 2.77 it was introduced > earlier, and purely in upstream. In the meantime, I've pinpointed where the problem was introduced in glib (71b7efd08a on mainline was backported as 3fd1b63453 on glib-2-76 just before 2.76.5 was cut); and they have a patch pending to fix it which I have now tested: https://gitlab.gnome.org/GNOME/glib/-/merge_requests/3565 > > > I presume that 'notmuch config set' should be the preferred way to > > modify the config file - but since it IS a human-readable file, > > notmuch should do a much better job of reporting errors whenever > > glib's gkeyfile API cannot parse the file (even if that failure to > > parse is caused by an unintended regression in glib behavior for > > rejecting something it used to accept). > > Yes. This round of glib2 gave us quite some headaches to get config > back working at all. The typical response from glib2 upstream was that > what we called regressions were fixes to wrong behaviour in glib2 and > that we should not rely on established behaviour (my words) but only > on the documentation, the latter exposing a sense of humour which I do > appreciate at times. > > In particular, glib2's read and write results are the authoritative > answer. And probably the older glib2 was "wrong" in what it accepted > leniently ... Does notmuch even get the chance to read partially? GError are intended to be used as recoverable errors - we have every right to refactor the logic to ignore keys that are unreadable while still moving forward with the rest of the file, if we want to do partial reads. But that's more invasive than the patch I just proposed, which merely prints the GError message to the user (serves as a warning to a user that their hand-written invalid escapes are being accepted anyways with 2.76.1, and gives the user an explanation why notmuch isn't working with 2.76.5). -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] config: Inform user if config file is broken
glib 2.76.1 silently treats invalid escape sequences as two
characters, even though it is willing to set a GError warning about
it. While 'notmuch config set' never produces such sequences in the
config file, the fact that the config file is human-readable lends
itself to hand-written edits, where the person making the edits can
introduce things such as:
[query]
foo = from:/example\.org/
instead of the correct
[query]
foo = from:/example\\.org/
glib 2.76.5 turned this into a hard error, but nothing higher in the
call stack outputs anything to the user in the case of
NOTMUCH_STATUS_FILE_ERROR to let the user know the problem ('notmuch
new' and 'notmuch count' silently fail with no output; 'notmuch config
list' outputs nothing and reports success). While glib will be fixing
their regression before 2.78 [1], it is likely that future glib will
restore the hard error. Thus, we should inform the user any time
their config file cannot be parsed; this gives a warning when using
glib 2.76.1 where the parse is still successful, and gives an
explanation why nothing happens during 'notmuch count' or 'notmuch
config list' when using glib 2.76.5 where the parse fails.
The added output message may still not be the most obvious:
$ notmuch --config=$PWD/.notmuch-config count
Key file contains key “foo” which has a value that cannot be interpreted.
but it is better than silence.
[1] https://gitlab.gnome.org/GNOME/glib/-/merge_requests/3565
Fixes:
https://nmbug.notmuchmail.org/nmweb/show/5a7paaqa2dvdo5lmnxvaeacfwhdytfnkr4gfh6mtlotdviki2s%40ro4gz4m2aqsw
---
I'm not sure if this is the best approach (as this is my first ever
patch to notmuch), but it's better than nothing.
[And if Carl Worth still reads the list - thanks for introducing me to
emacs back in 2000 when we worked together as students at BYU]
lib/config.cc | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/lib/config.cc b/lib/config.cc
index 2323860d..afe8f429 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -435,6 +435,7 @@ _notmuch_config_load_from_file (notmuch_database_t *notmuch,
for (gchar **keys_p = keys; *keys_p; keys_p++) {
char *absolute_key = talloc_asprintf (notmuch, "%s.%s", *grp,
*keys_p);
char *normalized_val;
+ GError *gerr = NULL;
/* If we opened from a given path, do not overwrite it */
if (strcmp (absolute_key, "database.path") == 0 &&
@@ -442,7 +443,11 @@ _notmuch_config_load_from_file (notmuch_database_t
*notmuch,
notmuch->xapian_db)
continue;
- val = g_key_file_get_string (file, *grp, *keys_p, NULL);
+ val = g_key_file_get_string (file, *grp, *keys_p, );
+ if (gerr) {
+ fprintf (stderr, "%s\n", gerr->message);
+ g_error_free (gerr);
+ }
if (! val) {
status = NOTMUCH_STATUS_FILE_ERROR;
goto DONE;
base-commit: 5303e35089e1a8ffcdb1d5891bc85d3f6c401a8f
prerequisite-patch-id: e81473c9dc7ffd8b3b9bf64b3f3edb84bfb99bbb
--
2.41.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] .gitignore: ignore __pycache__
Python likes to leave behind cache files; noticeable when doing an in-tree build. --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index f94d1480..eda6d9cf 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,4 @@ /bindings/python-cffi/_notmuch_config.py TAGS tags +__pycache__ base-commit: 5303e35089e1a8ffcdb1d5891bc85d3f6c401a8f -- 2.41.0 ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: notmuch breaks on \. in config file with upgrade from glib2 2.76.1 to 2.76.5
Hi there [snip] > Last night, I filed > https://bugzilla.redhat.com/show_bug.cgi?id=2237562. Later, I found > this about glib 2.77 introducing regressions: > https://bugzilla.redhat.com/show_bug.cgi?id=2225257; looks like Fedora > backported enough of that into 2.76.5 to cause similar issues in > relation to 2.76.1, even though a patchlevel release of glib shouldn't > be changing behaviors. Fedora has no related patches in 2.76.5-1 at all (only hmac/eperm). So, if that's the same regression as in 2.77 it was introduced earlier, and purely in upstream. > I presume that 'notmuch config set' should be the preferred way to > modify the config file - but since it IS a human-readable file, > notmuch should do a much better job of reporting errors whenever > glib's gkeyfile API cannot parse the file (even if that failure to > parse is caused by an unintended regression in glib behavior for > rejecting something it used to accept). Yes. This round of glib2 gave us quite some headaches to get config back working at all. The typical response from glib2 upstream was that what we called regressions were fixes to wrong behaviour in glib2 and that we should not rely on established behaviour (my words) but only on the documentation, the latter exposing a sense of humour which I do appreciate at times. In particular, glib2's read and write results are the authoritative answer. And probably the older glib2 was "wrong" in what it accepted leniently ... Does notmuch even get the chance to read partially? Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
notmuch breaks on \. in config file with upgrade from glib2 2.76.1 to 2.76.5
Summarizing a regression bug I first reported on IRC, which I hit on
Fedora 38. These steps reproduce it without impacting anything else
you may have on your system:
$ rpm -q notmuch
notmuch-0.37-5.fc38.x86_64
$ sudo dnf downgrade -y glib2*.x86_64 glib2*.i686
<...>
$ rpm -q glib2
glib2-2.76.1-1.fc38.x86_64
glib2-2.76.1-1.fc38.i686
$ mkdir /tmp/foo
$ cd /tmp/foo
$ cat >.notmuch-config <
$ notmuch --config=$PWD/.notmuch-config new
Found 1 total files (that's not much mail).
Note: Ignoring non-mail file: /tmp/foo/.notmuch-config
Processed 1 file in almost no time.
No new mail.
$ notmuch --config=$PWD/.notmuch-config config set query.q1
'from:/.*\.example\.org/'
$ printf 'q2=from:"/.*\\.example\\.org/"\n' >> .notmuch-config
$ tail -n3 .notmuch-config
[query]
q1=from:/.*\\.example\\.org/
q2=from:"/.*\.example\.org/"
$ notmuch --config=$PWD/.notmuch-config config get query.q1
from:/.*\.example\.org/
$ notmuch --config=$PWD/.notmuch-config config get query.q2
from:"/.*\.example\.org/"
$ notmuch --config=$PWD/.notmuch-config count
0
$ sudo dnf upgrade -y
$ rpm -q glib2
glib2-2.76.5-1.fc38.x86_64
glib2-2.76.5-1.fc38.i686
$ notmuch --config=$PWD/.notmuch-config config list
$ echo $?
0
$ notmuch --config=$PWD/.notmuch-config count
$ echo $?
1
Yikes - any configuration I wrote using 'notmuch config set' is stored
by older glib in a way that newer glib can still read. But any
configuration that I manually added directly into the config file
might contain data that older glib can parse but newer glib rejects.
In this particular case, it looks like older glib happily treats "\."
as a string with two literal characters, while newer glib is trying to
treat it as an escape sequence and failing. This failure is then
poorly handled by notmuch, which has knock-on odd effects ('config
list' producing no output but exiting successfully!, 'count' producing
no output but giving no error message why, etc.).
Last night, I filed
https://bugzilla.redhat.com/show_bug.cgi?id=2237562. Later, I found
this about glib 2.77 introducing regressions:
https://bugzilla.redhat.com/show_bug.cgi?id=2225257; looks like Fedora
backported enough of that into 2.76.5 to cause similar issues in
relation to 2.76.1, even though a patchlevel release of glib shouldn't
be changing behaviors.
I presume that 'notmuch config set' should be the preferred way to
modify the config file - but since it IS a human-readable file,
notmuch should do a much better job of reporting errors whenever
glib's gkeyfile API cannot parse the file (even if that failure to
parse is caused by an unintended regression in glib behavior for
rejecting something it used to accept).
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
