Re: bug: wrong order of messages in notmuch-show
Hi David, notmuch developers, * David Bremner [2018-09-08; 07:35]: > Gregor Zattler writes: >> notmuch sometimes shows emails of one thread in wrong order. Now >> I found an example in a public mailing list so it is possible to >> share this example. It is a small thread from the notmuch >> mailing list, you might find it by searching for >> id:87zhzccrwg@tethera.net >> >> While this email is the last written and sent in this thread it >> is shown as first email in the notmuch show output, as shown in >> this "screenshot" of the corresponding notmuch-emacs buffer: > > As far as I know the problems discussed in this message should be fixed > in commit 87934c432c4bee9df09f268a3f05933c59c2caf1, so I'm marking it > fixed. Yes, for me these problems are fixed now, as the ones described in id:87fu05p2iu.fsf@len.workgroup Thanks, it's much easier now for me to follow long email threads. Ciao; Gregor -- -... --- .-. . -.. ..--.. ...-.- ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
Gregor Zattler writes: > Dear notmuch developers, > > notmuch sometimes shows emails of one thread in wrong order. Now > I found an example in a public mailing list so it is possible to > share this example. It is a small thread from the notmuch > mailing list, you might find it by searching for id:87zhzccrwg@tethera.net > > While this email is the last written and sent in this thread it > is shown as first email in the notmuch show output, as shown in > this "screenshot" of the corresponding notmuch-emacs buffer: As far as I know the problems discussed in this message should be fixed in commit 87934c432c4bee9df09f268a3f05933c59c2caf1, so I'm marking it fixed. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
Hi David, * David Bremner [2018-07-29; 17:47]: > Gregor Zattler writes: >> * Gregor Zattler [2018-07-01; 23:27]: > I find it helpful to look at these things in tree-view. > > July 09 Gregor Zattler┬►xxx-x-server > (inbox) > July 09 via RT > ╰┬►[support.xxx-x-x.de #33712] AutoReply: > xxx-x-server (inbox replied) > July 09 Gregor Zattler ╰┬► ... > (inbox replied) > July 10 Gregor Zattler > ├─►[support.xxx-x-x.de #33712] > xxx-x-server (inbox) > Tue. 20:03 Gregor Zattler > ├─►[support.xxx-x-x.de #33712] AutoReply: > xxx-x-server (inbox) > July 09 via RT > ╰─►[support.xxx-x-x.de #33712] > xxx-x-server (inbox) > > The place notmuch is not doing as nice a job as you would like is in > ordering the children of your July 9 message. I think this is because it > is added into the tree in the second pass. It ends up in this situation > because its In-Reply-To field points to a non-existent (in the local > database). In the previous version of threading, I'm pretty sure it > would have been a root message, which might have looked a bit better > with respect to dates, but would have been just as confusing. Yes it would or was. Your poatches are surely an improvement. > [enter thinking out loud mode] > > I agree there is still room for improvement here. Looking at the code, > it seems like it might make sense to order the children by date in > _notmuch_message_add_reply. Currently this uses a generic linked list > insert at tail, which is O(1). It would be more expensive to insert by > date, but it would depend how many replies to a given message there are. Thanks for considering this. FWIW: This is what mutt does (the '=>' marks duplicates): 1| | 0|Di-19.06.18-18:26|Gregor Zattler | 0K|FYI: xxx xxx 2| | 0|Di-19.06.18-18:26| via RT |0,1K|`->[support.xxx-x-x.de #33575] AutoReply: FYI: xxx xxx 3| | 0|Di-26.06.18-14:44| via RT |0,1K|[support.xxx-x-x.de #33575] Resolved: FYI: xxx xxx 4| | 0|Mo-09.07.18-10:03|Gregor Zattler |0,1K|xxx-x-server 5|r | 0|Mo-09.07.18-10:04|via RT |0,1K|`->[support.xxx-x-x.de #33712] AutoReply: xxx-x-server 6|r | 0|Mo-09.07.18-10:24|Gregor Zattler |0,1K| `-> 7|r | 0|Mo-09.07.18-10:24|Gregor Zattler |0,1K||=> 8| | 0|Mo-09.07.18-10:37|via RT |0,1K| |->[support.xxx-x-x.de #33712] xxx-x-server 9| | 0|Di-10.07.18-12:18|Gregor Zattler |0,1K||->Re: [support.xxx-x-x.de #33712] xxx-x-server 10| | 0|Di-10.07.18-12:18|Gregor Zattler |0,1K|| `=> 11| | 0|Di-24.07.18-14:03|Gregor Zattler |0,1K|`->Re: [support.xxx-x-x.de #33712] AutoReply: xxx-x-server 12| | 0|Di-24.07.18-14:03|Gregor Zattler |0,1K| `=> - The treads are ordered according to date of arrival. - The "resolved" email (No 3) is not part of the 33575-Thread (which is a pity). Thanks, Gregor -- -... --- .-. . -.. ..--.. ...-.- ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
Gregor Zattler writes: > Hi David, dear notmuch developers, > * Gregor Zattler [2018-07-01; 23:27]: >> This happens often to me with emails from a ticket system which I >> cannot disclose. In this kind of technical threads order is >> necessary to understand current status of affairs. > > I finally isolated two such threads, anonymized them and indexed > them. tar'ed and compressed the maildir is small enough to attach, > therefore I attached it. > > > With the first thread (Ticket 33712) notmuch shows the "-->" > marked email last in thread, instead (as according to date sent) > before position marked "+++": I find it helpful to look at these things in tree-view. July 09 Gregor Zattler┬►xxx-x-server (inbox) July 09 via RT ╰┬►[support.xxx-x-x.de #33712] AutoReply: xxx-x-server (inbox replied) July 09 Gregor Zattler ╰┬► ... (inbox replied) July 10 Gregor Zattler ├─►[support.xxx-x-x.de #33712] xxx-x-server (inbox) Tue. 20:03 Gregor Zattler ├─►[support.xxx-x-x.de #33712] AutoReply: xxx-x-server (inbox) July 09 via RT ╰─►[support.xxx-x-x.de #33712] xxx-x-server (inbox) The place notmuch is not doing as nice a job as you would like is in ordering the children of your July 9 message. I think this is because it is added into the tree in the second pass. It ends up in this situation because its In-Reply-To field points to a non-existent (in the local database). In the previous version of threading, I'm pretty sure it would have been a root message, which might have looked a bit better with respect to dates, but would have been just as confusing. [enter thinking out loud mode] I agree there is still room for improvement here. Looking at the code, it seems like it might make sense to order the children by date in _notmuch_message_add_reply. Currently this uses a generic linked list insert at tail, which is O(1). It would be more expensive to insert by date, but it would depend how many replies to a given message there are. ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
Hi David, dear notmuch developers, * Gregor Zattler [2018-07-01; 23:27]: > This happens often to me with emails from a ticket system which I > cannot disclose. In this kind of technical threads order is > necessary to understand current status of affairs. I finally isolated two such threads, anonymized them and indexed them. tar'ed and compressed the maildir is small enough to attach, therefore I attached it. With the first thread (Ticket 33712) notmuch shows the "-->" marked email last in thread, instead (as according to date sent) before position marked "+++": [support.xxx-x-x.de #33712] xxx-x-server Gregor Zattler (Mon, 09 Jul 2018 10:03:54 +0200) (inbox) Subject: xxx-x-server via RT (Mon, 09 Jul 2018 10:04:03 +0200) (inbox replied) Subject: [support.xxx-x-x.de #33712] AutoReply: xxx-x-server Gregor Zattler (Mon, 09 Jul 2018 10:24:26 +0200) (inbox replied) Gregor Zattler (Tue, 10 Jul 2018 12:18:39 +0200) (inbox) Subject: Re: [support.xxx-x-x.de #33712] xxx-x-server +++ Gregor Zattler (Tue, 24 Jul 2018 14:03:22 +0200) (inbox) Subject: Re: [support.xxx-x-x.de #33712] AutoReply: xxx-x-server --> via RT (Mon, 09 Jul 2018 10:37:43 +0200) (inbox) Subject: [support.xxx-x-x.de #33712] xxx-x-server With the second thread the email marked "-->" should be at the end, IMHO. I don'nt know how this Resolved-Message is produced, perhaps it is not a reply to the last message in thread: [support.xxx-x-x.de #33575] AutoReply: FYI: xxx xxx via RT (Tue, 26 Jun 2018 14:44:24 +0200) (inbox) --> Subject: [support.xxx-x-x.de #33575] Resolved: FYI: xxx xxx Gregor Zattler (Tue, 19 Jun 2018 18:26:26 +0200) (inbox) Subject: FYI: xxx xxx via RT (Tue, 19 Jun 2018 18:26:36 +0200) (inbox) Subject: [support.xxx-x-x.de #33575] AutoReply: FYI: xxx xxx Thanks again for your attention. Ciao; Gregor -- -... --- .-. . -.. ..--.. ...-.- test-email-order-in-threads.tar.xz Description: application/xz ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [Oliver Runge] Re: bug: wrong order of messages in notmuch-show
Oliver Runge writes: > > Hello, David. > > I came across this bug report for notmuch when I wanted to report it myself: > https://www.mail-archive.com/notmuch@notmuchmail.org/msg46633.html > > I didn't see an obvious way to respond on the mailing list (tho I now > subscribed for next time), so I hope a direct response is ok. I took the liberty of forwarding the response to the list. > > You've already reproduced it, but I wanted to give additional > information. I used a test case in my emails and threw it at git > bisect. The commit introducing this behaviour is > 491b1f4b4082bee18418942846ec6508856be7b4, where you fixed the order > for a loop. Reverting this results in the correct order for threads > with an unknown Message-ID in In-Reply-To (which is the case in my > thread as well). I suspected something along those lines; good to know for sure. I don't think reverting the commit is an option in general, since the bug it fixes is much worse than the one it introduces. > Not having an email with the Message-ID probably is quite a normal > situation, for instance when people reply to someone and add someone > else to CC. I guess the right behaviour would be to simply go by date. Did you try the series id:20180720233746.2844-1-da...@tethera.net [1]? As far as I know it should fix the cases where in-reply-to is missing, but references is sensible. > > I tried to build a test case myself and maybe a patch, but I'm rather > lost with the tests... and about 200 of them fail for me. No idea > whether they should pass or not. > The tests should definitely all pass (except the 5 or so marked BROKEN). Perhaps you can start a seperate thread about that. It would be useful to know the OS, and anything peculiar about your setup. [1]: https://www.mail-archive.com/notmuch@notmuchmail.org/msg46660.html ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[Oliver Runge] Re: bug: wrong order of messages in notmuch-show
--- Begin Message --- Hello, David. I came across this bug report for notmuch when I wanted to report it myself: https://www.mail-archive.com/notmuch@notmuchmail.org/msg46633.html I didn't see an obvious way to respond on the mailing list (tho I now subscribed for next time), so I hope a direct response is ok. You've already reproduced it, but I wanted to give additional information. I used a test case in my emails and threw it at git bisect. The commit introducing this behaviour is 491b1f4b4082bee18418942846ec6508856be7b4, where you fixed the order for a loop. Reverting this results in the correct order for threads with an unknown Message-ID in In-Reply-To (which is the case in my thread as well). Not having an email with the Message-ID probably is quite a normal situation, for instance when people reply to someone and add someone else to CC. I guess the right behaviour would be to simply go by date. I tried to build a test case myself and maybe a patch, but I'm rather lost with the tests... and about 200 of them fail for me. No idea whether they should pass or not. Ciao Oliver --- End Message --- ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
Hi David, * David Bremner [2018-07-01; 21:23]: > Gregor Zattler writes: > The thread is not ordered that way here. Can you run the "draw-thread" > tool in id:20180410014539.24717-1-da...@tethera.net and send the output > (either dot or PDF) as an attachement to the list? sure, I produced two sets of files: the ones with "for-real" in their names are done with my working notmuch installation, the ones with "test-order-notmuch" in their names are done with ta maildir which only contains the relevant messages. > Yes, I'll probably need the tar file to reproduce it. If it's small > (<50k), send it to the list, otherwise let me know how I can pick it up. This tar file also contains a tar file with the xapian database for these 8 messages etc.: https://giku.de/4iczfdzgnedth8ni--notmuch-show-order-problem.tar.7z Thanks for looking into this. Ciao; Gregor ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
Gregor Zattler writes: > Dear notmuch developers, > > notmuch sometimes shows emails of one thread in wrong order. Now > I found an example in a public mailing list so it is possible to > share this example. It is a small thread from the notmuch > mailing list, you might find it by searching for id:87zhzccrwg@tethera.net > > While this email is the last written and sent in this thread it > is shown as first email in the notmuch show output, as shown in > this "screenshot" of the corresponding notmuch-emacs buffer: > > File Edit Options Buffers Tools Help > 'notmuch search thread:<>' lists multiple threads > --> David Bremner (Yest. 15:42) (inbox new unread) > Subject: Re: 'notmuch search thread:<>' lists multiple threads > Naveen N. Rao (April 06) (inbox new > unread) >Naveen N. Rao (April 06) (inbox new > unread) >David Bremner (April 08) (inbox new unread) > David Bremner (April 09) (inbox new unread) > David Bremner (April 10) (inbox new unread) > Subject: [PATCH] devel: add new tool to draw thread structure > Naveen N. Rao (April 18) (inbox > new unread) > Subject: Re: 'notmuch search thread:<>' lists multiple threads > David Bremner (April 22) (inbox new unread) > Hi Gregor; The thread is not ordered that way here. Can you run the "draw-thread" tool in id:20180410014539.24717-1-da...@tethera.net and send the output (either dot or PDF) as an attachement to the list? > > This happens often to me with emails from a ticket system which I > cannot disclose. In this kind of technical threads order is > necessary to understand current status of affairs. Unfortunately ticket systems seem to be among the worse offenders for generating nonsense headers. > I did a test with notmuch 0.27+7~gfd3c936 and .notmuch-config > pointing to a test maildir with this 8 messages as they arrived > at my email account. I saved a tar archive of this in case this > is important to debug the problem. > Yes, I'll probably need the tar file to reproduce it. If it's small (<50k), send it to the list, otherwise let me know how I can pick it up. ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: bug: wrong order of messages in notmuch-show
For what it's worth, I've also noticed a similar behaviour for some threads on some of the lists.gnu.org lists lately. -amin ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
bug: wrong order of messages in notmuch-show
Dear notmuch developers, notmuch sometimes shows emails of one thread in wrong order. Now I found an example in a public mailing list so it is possible to share this example. It is a small thread from the notmuch mailing list, you might find it by searching for id:87zhzccrwg@tethera.net While this email is the last written and sent in this thread it is shown as first email in the notmuch show output, as shown in this "screenshot" of the corresponding notmuch-emacs buffer: File Edit Options Buffers Tools Help 'notmuch search thread:<>' lists multiple threads --> David Bremner (Yest. 15:42) (inbox new unread) Subject: Re: 'notmuch search thread:<>' lists multiple threads Naveen N. Rao (April 06) (inbox new unread) Naveen N. Rao (April 06) (inbox new unread) David Bremner (April 08) (inbox new unread) David Bremner (April 09) (inbox new unread) David Bremner (April 10) (inbox new unread) Subject: [PATCH] devel: add new tool to draw thread structure Naveen N. Rao (April 18) (inbox new unread) Subject: Re: 'notmuch search thread:<>' lists multiple threads David Bremner (April 22) (inbox new unread) This happens often to me with emails from a ticket system which I cannot disclose. In this kind of technical threads order is necessary to understand current status of affairs. I did a test with notmuch 0.27+7~gfd3c936 and .notmuch-config pointing to a test maildir with this 8 messages as they arrived at my email account. I saved a tar archive of this in case this is important to debug the problem. Thanks, for your attention, Gregor ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
