Re: Possible threading issues in nm 0.32
... just a guess: Could it be that
a9f74aee ("CLI/new: drop the write lock to run the pre-new hook.", 2021-03-18)
was not enough?
notmuch_database_reopen() only reopens the xapian db but does not update
other members in notmuch_database_t *notmuch, such as the last doc id
and thread id.
If a pre-merge hook changes the database then values in that struct will
be out of date.
Before the config changes in nm 0.32, there was no such struct to begin
with. After that, notmuch holds the struct just to be able to run the
hook (from the proper dir).
So I would think that reopen()ing to MODE_READ_ONLY is not a problem.
But after the hook run, are full close() then open() to
MODE_READ_WRITE() is necessary so that the values in the struct are
correct (or change reopen() to do that).
Indeed, if you open in MODE_READ_ONLY and don't hold a write lock you
cannot trust any cached values such as those stored in the struct, can
you?
Michael
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Possible threading issues in nm 0.32
Alexander Adolf venit, vidit, dixit 2021-05-11 16:32:22: > Alexander Adolf writes: > > > Michael J Gruber writes: > > > >> [...] > >> So it seems: > >> > >> - The mis-threading happens during `notmuch new`, not with `notmuch > >> reindex`. > >> - In this new case (and if I remember correctly also in the others), > >> it's always a new message getting worngly put into an existing thread, > >> and if I'm not mistaken, the existing thread was tagged as trash > >> before in all cases. > >> [...] > > > > I can confirm both observations. > > [...] > > p.s.: Just got the weird threading with `notmuch reindex`, too. Oh my gosh... This is getting interesting. I'm delving (literally) into the xapian db now. "Put into an existing thread" (what I had wiritten) was not correct in terms of thread IDs. What's happening is the following: I have an existing message A which is tagged as trash. A is the only message in thread 00021144. A new message B is put in the db by "notmuch new". Notmuch correctly creates a new thread 00021148 (the next avalaible id) and puts B in this new thread. G00021148 is the only thread term in the db for the document belonging to message B. So far so good, but: The document for message A has three thread terms now: G00021144 G00021148 G00021149 Note that neither A nor B have any in-reply-to or references header. AFAIK multiple thread terms on a single message document are a complete no-go and indicate a problem, especially when an unrelated existing message's entry is touched. notmuch search --exclude=false thread:00021148 lists both A and B now, of course. The third one, G00021149, is completely weird. It leads to yet another message document with multiple thread entries. Looking at a few of the most recent messages, my suspicion is: - A new message with in-reply-to/references get's a single (existing) thread term correctly. - A new message without in-reply-to/references get's the correct new thread term; in addition, this get's assigned to some random existing message by *adding* it to the list of terms, thereby making that message part of multiple threads. I have not checked systematically yet whether it (the multi-G-terms) indeed affects Ktrash ones only. Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Possible threading issues in nm 0.32
Michael J Gruber writes: > [...] > So it seems: > > - The mis-threading happens during `notmuch new`, not with `notmuch > reindex`. > - In this new case (and if I remember correctly also in the others), > it's always a new message getting worngly put into an existing thread, > and if I'm not mistaken, the existing thread was tagged as trash > before in all cases. > [...] I can confirm both observations. --alexander ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Possible threading issues in nm 0.32
David Bremner venit, vidit, dixit 2021-05-05 14:29:38: > Michael J Gruber writes: > > > After updating to notmuch 0.32 I noticed a few strange threadings that > > went away after downgrading to 0.31.3 and reindexing. I can't pin-point > > that yet but wanted to give an early warning so you can look out for > > that, too. > > I'm not aware of any threading relevant changes in 0.32, but there was a > lot of churn in the library. If you could bisect or give a reproducer > that would be great. I could not reproduce by reindexing with nm 0.32, so I started using nm 0.32 again. Indeed, I got another mis-threading. I reindexed with nm 0.32 (without downgrading) and the thread got separated correctly again. That explains why I was not able to create a reproducer. So it seems: - The mis-threading happens during `notmuch new`, not with `notmuch reindex`. - In this new case (and if I remember correctly also in the others), it's always a new message getting worngly put into an existing thread, and if I'm not mistaken, the existing thread was tagged as trash before in all cases. Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Possible threading issues in nm 0.32
Michael J Gruber writes: > After updating to notmuch 0.32 I noticed a few strange threadings that > went away after downgrading to 0.31.3 and reindexing. I can't pin-point > that yet but wanted to give an early warning so you can look out for > that, too. I'm not aware of any threading relevant changes in 0.32, but there was a lot of churn in the library. If you could bisect or give a reproducer that would be great. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
