Re: Forcing a sync of maildir flags?
Hello, On Tue 03 May 2022 at 09:14AM -03, David Bremner wrote: > I have the same intuition. Unfortunately it is not as simple as adding > in a couple calls to this function. The complications I am aware of so > far are > > 1) We need to distinguish between when a newly discovered file should > update tags, and when the newly discovered file should have it's flags > updated. > > 2) Renaming files in the middle of notmuch new needs to be done > carefully in order that notmuch doesn't lose track of the files. They > are only lost until the next run of notmuch-new, but it's not ideal. Okay, thanks for the info. Nice to have it all recorded in this thread. -- Sean Whitton ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Sean Whitton writes: > Hello David, >> # next line is a no-op, because it already doesn't have the unread tag >> notmuch tag -unread folder:sent > > Seems also worth noting that to my mind it ultimately shouldn't be > necessary to run that command -- notmuch should notice that one copy of > the message has different maildir flags to the other in a way that's out > of sync with the notmuch tags it has. > Right, that's probably a useful way to think about the problem. >> The key point is that from notmuch's point of view the message never has >> the unread tag, so there is no change for "notmuch tag" to sync with >> maildir flags. >> >> It doesn't seem to make a difference if I put the copy in inbox/new or >> inbox/cur, so I don't think it is related to the previous efforts not to >> prematurely move files out of new/. >> >> As far as I can tell, notmuch-new (unlike notmuch-insert) does not call >> notmuch_message_tags_to_maildir_flags, so the maildir flags on the newly >> discovered copy are not updated. Perhaps it should, but that seems like >> a pretty big change, so I want to proceed with caution. > > It makes sense to me for notmuch-new to call that function too, fwiw. I have the same intuition. Unfortunately it is not as simple as adding in a couple calls to this function. The complications I am aware of so far are 1) We need to distinguish between when a newly discovered file should update tags, and when the newly discovered file should have it's flags updated. 2) Renaming files in the middle of notmuch new needs to be done carefully in order that notmuch doesn't lose track of the files. They are only lost until the next run of notmuch-new, but it's not ideal. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Hello David, On Sun 01 May 2022 at 08:23PM -03, David Bremner wrote: > Sean Whitton writes: >> >> Thanks. Let me record in this thread what I will believe it will take a >> reproduce this in a test: >> >> 1) inbox and sent are Maildirs >> >> 1) Compose mail to a mailing list which will return copies of >>submissions, with `Fcc: sent -unread` > > Since you mention Fcc, are you using notmuch insert? notmuch-maildir-use-notmuch-insert is t so I think I am? > # next line is a no-op, because it already doesn't have the unread tag > notmuch tag -unread folder:sent Seems also worth noting that to my mind it ultimately shouldn't be necessary to run that command -- notmuch should notice that one copy of the message has different maildir flags to the other in a way that's out of sync with the notmuch tags it has. > The key point is that from notmuch's point of view the message never has > the unread tag, so there is no change for "notmuch tag" to sync with > maildir flags. > > It doesn't seem to make a difference if I put the copy in inbox/new or > inbox/cur, so I don't think it is related to the previous efforts not to > prematurely move files out of new/. > > As far as I can tell, notmuch-new (unlike notmuch-insert) does not call > notmuch_message_tags_to_maildir_flags, so the maildir flags on the newly > discovered copy are not updated. Perhaps it should, but that seems like > a pretty big change, so I want to proceed with caution. It makes sense to me for notmuch-new to call that function too, fwiw. -- Sean Whitton ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Sean Whitton writes:
>
> Thanks. Let me record in this thread what I will believe it will take a
> reproduce this in a test:
>
> 1) inbox and sent are Maildirs
>
> 1) Compose mail to a mailing list which will return copies of
>submissions, with `Fcc: sent -unread`
Since you mention Fcc, are you using notmuch insert?
>
> 2) notmuch new, and in post-new hook, notmuch tag -unread -- folder:sent
>
> Expected result: copy of message in both inbox and sent has Seen flag
>
> Actual result: only copy of message in sent has Seen flag
I tried to simulate this in the form of a test. I'm not using notmuch
insert here, but I manage to trigger similar looking behaviour:
test_begin_subtest "duplicate file with seen flag"
test_subtest_known_broken
mkdir -p "${MAIL_DIR}"/sent/{cur,new,tmp}
notmuch config set maildir.synchronize_flags true
add_message '[subject]="seen message"' '[filename]=fcc-file:2,S'
'[dir]=sent/cur'
notmuch new
# for debugging
notmuch search subject:"seen message"
mkdir -p "${MAIL_DIR}"/inbox/{cur,new,tmp}
cp "${gen_msg_filename}" "${MAIL_DIR}"/inbox/new/copied-file:2,
notmuch new
# for debugging
notmuch search subject:"seen message"
# next line is a no-op, because it already doesn't have the unread tag
notmuch tag -unread folder:sent
count=$(notmuch search --output=files subject:"seen message" | grep -c ',S$')
# for debugging:
notmuch search --output=files subject:"seen message"
test_expect_equal "${count}" "2"
This has the output
BROKEN duplicate file with seen flag
--- T050-new.41.expected2022-05-01 22:51:45.644747316 +
+++ T050-new.41.output 2022-05-01 22:51:45.644747316 +
@@ -1 +1 @@
-2
+1
No new mail.
thread:0011 2001-01-05 [1/1] Notmuch Test Suite; seen message
(inbox)
Processed 1 file in almost no time.
No new mail.
thread:0011 2001-01-05 [1/1(2)] Notmuch Test Suite; seen
message (inbox)
/home/bremner/software/upstream/notmuch/test/tmp.T050-new/mail/sent/cur/fcc-file:2,S
/home/bremner/software/upstream/notmuch/test/tmp.T050-new/mail/inbox/new/copied-file:2,
The key point is that from notmuch's point of view the message never has
the unread tag, so there is no change for "notmuch tag" to sync with
maildir flags.
It doesn't seem to make a difference if I put the copy in inbox/new or
inbox/cur, so I don't think it is related to the previous efforts not to
prematurely move files out of new/.
As far as I can tell, notmuch-new (unlike notmuch-insert) does not call
notmuch_message_tags_to_maildir_flags, so the maildir flags on the newly
discovered copy are not updated. Perhaps it should, but that seems like
a pretty big change, so I want to proceed with caution.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Hi Sean, * Sean Whitton [2022-04-04; 21:48]: > On Tue 22 Mar 2022 at 12:44pm -07, Sean Whitton wrote: > >> I am seeing this bug, or a closely related one, a whole lot right now. >> Messages are coming back as unread over and over again. me too. And it's not only messages from me, but also messages from other people appear as unread of which I'm quite certain I already read them. >> I recently made some changes to my notmuch cronjobs, so >> that probably has something to do with it, but I have no >> guesses as to what the problem is. >> >> As a first thing to try, I am going to add something to my pre-new hook >> to perform the new/ -> cur/ move as specified by maildir(5) on all my >> synced maildirs, so that notmuch never sees messages in new/ except when >> it writes new drafts and sent mail there (and they'll get moved on the >> next sync). Would you please disclose your workaround? I'm very much interested. Ciao; Gregor -- -... --- .-. . -.. ..--.. ...-.- ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Hello, On Tue 22 Mar 2022 at 12:44pm -07, Sean Whitton wrote: > I am seeing this bug, or a closely related one, a whole lot right now. > Messages are coming back as unread over and over again. I recently made > some changes to my notmuch cronjobs, so that probably has something to > do with it, but I have no guesses as to what the problem is. > > As a first thing to try, I am going to add something to my pre-new hook > to perform the new/ -> cur/ move as specified by maildir(5) on all my > synced maildirs, so that notmuch never sees messages in new/ except when > it writes new drafts and sent mail there (and they'll get moved on the > next sync). > > I'll drop the 'notmuch tag -unread -- folder:sent' workaround at the > same time to see what happens. Seems to be working well. -- Sean Whitton ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Hello David, On Thu 20 Feb 2020 at 08:22AM -04, David Bremner wrote: > Sean Whitton writes: > >> I have this in my post-new hook: >> >> notmuch tag -unread -- folder:sent >> >> The idea is that copies of my sent mail which get returned to me and >> stored in my inbox, e.g. by mailing lists, would get maildir Seen tags >> added to them. But that does not happen. > > I guess this is most likely a bug. It would be nice to have a test in > T340-maildir-sync.sh that duplicated it. I suspect the problem is > related to messages in the new/ subdirectory. Some non-notmuch clients > (prominently mutt) interpret being in new/ has having user-visible > semantics, so notmuch tries not to move files out of there > unnecessarily, On the other hand the maildir spec says that files in > new/ cannot have flags, so they cannot have their unread tag removed > without moving the file to cur/ I am seeing this bug, or a closely related one, a whole lot right now. Messages are coming back as unread over and over again. I recently made some changes to my notmuch cronjobs, so that probably has something to do with it, but I have no guesses as to what the problem is. As a first thing to try, I am going to add something to my pre-new hook to perform the new/ -> cur/ move as specified by maildir(5) on all my synced maildirs, so that notmuch never sees messages in new/ except when it writes new drafts and sent mail there (and they'll get moved on the next sync). I'll drop the 'notmuch tag -unread -- folder:sent' workaround at the same time to see what happens. -- Sean Whitton ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Forcing a sync of maildir flags?
Hello, On Thu 20 Feb 2020 at 08:06PM -04, David Bremner wrote: > Sean Whitton writes: > >> So, your hypothesis is that notmuch is opting not to add the Seen flag >> in order to avoid having to move the message from new to cur, for the >> sake of mutt, but that's a bug because I've explicitly requested adding >> Seen, so it ought to go ahead and move it from new to cur? > > Yeah, I think that's the case. This is based only vague memories of when > I wrote 0082a557. Thanks. Let me record in this thread what I will believe it will take a reproduce this in a test: 1) inbox and sent are Maildirs 1) Compose mail to a mailing list which will return copies of submissions, with `Fcc: sent -unread` 2) notmuch new, and in post-new hook, notmuch tag -unread -- folder:sent Expected result: copy of message in both inbox and sent has Seen flag Actual result: only copy of message in sent has Seen flag -- Sean Whitton ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Forcing a sync of maildir flags?
Sean Whitton writes: > So, your hypothesis is that notmuch is opting not to add the Seen flag > in order to avoid having to move the message from new to cur, for the > sake of mutt, but that's a bug because I've explicitly requested adding > Seen, so it ought to go ahead and move it from new to cur? Yeah, I think that's the case. This is based only vague memories of when I wrote 0082a557. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Forcing a sync of maildir flags?
Hello David, Thank you for your reply. On Thu 20 Feb 2020 at 08:22AM -04, David Bremner wrote: > Sean Whitton writes: > >> I have this in my post-new hook: >> >> notmuch tag -unread -- folder:sent >> >> The idea is that copies of my sent mail which get returned to me and >> stored in my inbox, e.g. by mailing lists, would get maildir Seen tags >> added to them. But that does not happen. > > I guess this is most likely a bug. It would be nice to have a test in > T340-maildir-sync.sh that duplicated it. I suspect the problem is > related to messages in the new/ subdirectory. Some non-notmuch clients > (prominently mutt) interpret being in new/ has having user-visible > semantics, so notmuch tries not to move files out of there > unnecessarily, On the other hand the maildir spec says that files in > new/ cannot have flags, so they cannot have their unread tag removed > without moving the file to cur/ So, your hypothesis is that notmuch is opting not to add the Seen flag in order to avoid having to move the message from new to cur, for the sake of mutt, but that's a bug because I've explicitly requested adding Seen, so it ought to go ahead and move it from new to cur? >> This works: >> >> notmuch tag +unread -- folder:sent >> notmuch tag -unread -- folder:sent >> >> Is there some more elegant way to force a sync of maildir flags? Am I >> the only one that has copies of my own mail in my inbox folder, not >> marked as read from the point of view of anything except notmuch? > > I guess my workflow doesn't expose this problem as I leave many messages > unread (relying on the inbox flag). I'm not sure about the locking > implications, but having sent messages delivered to cur/ instead of to > new/ would probably workaround the problem. In this case it's mbsync downloading messages from IMAP, and there doesn't seem to be a way to configure it to deliver to cur instead of new. On Thu 20 Feb 2020 at 09:21AM -04, David Bremner wrote: > I guess the other thing is that I have > > [new] > tags=unread;inbox;new > > in .notmuch-config, which means that "notmuch new" moves things into > cur/. Maybe you can use that effect? To be honest that doesn't seem really better than the workaround I'm currently using. -- Sean Whitton ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Forcing a sync of maildir flags?
David Bremner writes: >> This works: >> >> notmuch tag +unread -- folder:sent >> notmuch tag -unread -- folder:sent >> >> Is there some more elegant way to force a sync of maildir flags? Am I >> the only one that has copies of my own mail in my inbox folder, not >> marked as read from the point of view of anything except notmuch? > > I guess my workflow doesn't expose this problem as I leave many messages > unread (relying on the inbox flag). I'm not sure about the locking > implications, but having sent messages delivered to cur/ instead of to > new/ would probably workaround the problem. I guess the other thing is that I have [new] tags=unread;inbox;new in .notmuch-config, which means that "notmuch new" moves things into cur/. Maybe you can use that effect? d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Forcing a sync of maildir flags?
Sean Whitton writes: > I have this in my post-new hook: > > notmuch tag -unread -- folder:sent > > The idea is that copies of my sent mail which get returned to me and > stored in my inbox, e.g. by mailing lists, would get maildir Seen tags > added to them. But that does not happen. I guess this is most likely a bug. It would be nice to have a test in T340-maildir-sync.sh that duplicated it. I suspect the problem is related to messages in the new/ subdirectory. Some non-notmuch clients (prominently mutt) interpret being in new/ has having user-visible semantics, so notmuch tries not to move files out of there unnecessarily, On the other hand the maildir spec says that files in new/ cannot have flags, so they cannot have their unread tag removed without moving the file to cur/ > This works: > > notmuch tag +unread -- folder:sent > notmuch tag -unread -- folder:sent > > Is there some more elegant way to force a sync of maildir flags? Am I > the only one that has copies of my own mail in my inbox folder, not > marked as read from the point of view of anything except notmuch? I guess my workflow doesn't expose this problem as I leave many messages unread (relying on the inbox flag). I'm not sure about the locking implications, but having sent messages delivered to cur/ instead of to new/ would probably workaround the problem. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
