Re: Mutt losing folders with new mails with additional session

2020-07-23 Thread Josef 'Jeff' Sipek
On Thu, Jul 23, 2020 at 00:01:18 -0500, Derek Martin wrote:
> Oops, if I'd seen this message before I sent my last post, I probably
> wouldn't have bothered to post it.
> 
> That said, I will take issue with the notion that mbox is a terrible
> format:  It isn't.  It does, however, have usage patterns for which
> it is not well suited...

I'd like to point out that I said that mbox was somehow bad only twice in my
email and in both instances there were qualifiers about the workload:

"mbox is a horrible format for any sort of mailbox that sees
modifications other than just appending"

and:

"[updating the status info] may require rewriting the whole mbox
file - which is the big reason mbox is a terrible format :)"

(The second is a bit more implicit about it than I wish, but having to
rewrite the whole file *is* a major flaw with mbox.)

> just like maildir does.  If you've read the
> famous comparison of the two on the Courier website, you should note
> that while many of its claims are true, it nevertheless is in no small
> part bunk.

I may have seen it years ago but I didn't remember anything about it.  I
looked at it just now, and I have to say that I wouldn't trust the results
for anything today.  17 years ago they *may have* indicated something
useful, but I'm not even convinced of that.  The only thing I truly believe
from its final analysis is:

These benchmarks show that a big factor is the underlying hardware
and the operating system.

Which more or less invalidates its other results.

...
> I'll grant you that one or two of the points I made may be outdated,
> depending on what hardware and file system you're using and what
> options you're mounting it with.  But mostly... probably not.

There are definitely use cases where mbox is a decent choice.  I used to use
maildir for all "regular" mailboxes and mbox for the spam mailbox, but I
both to more efficient formats in Dovecot (sdbox and mdbox, respectively).
These more efficient formats roughly correspond to maildir and mbox, but use
external index files to keep track of status and other often modified data.
This reduces the number of fs operations quite a bit.

Jeff.

-- 
Intellectuals solve problems; geniuses prevent them
- Albert Einstein


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-23 Thread Josef 'Jeff' Sipek
On Wed, Jul 22, 2020 at 23:25:18 -0500, Derek Martin wrote:
> On Fri, Jul 17, 2020 at 08:37:50AM +0200, Sebastian Stein wrote:
...
> > When I open a folder, I can mark a mail as new. How does mutt keep track of
> > this flag? Is this stored inside mbox file?
> 
> Yes.  It's stored in a message header in the individual message.  This
> fact is unfortunately one of the things that slows mbox down, in cases
> when it is slower than maildir (namely updating status, specifically,
> in this case).  It did not take much thought to imagine that making
> status updates faster for mbox only required invariably storing
> messages with a fixed-length status header, where status updates could
> simply overwrite the existing status field in place.  Sadly no one did
> that.

Dovecot tries to make this possible by padding the heading with extra
spaces.  Then if the update is minor enough and there is enough space in
the header, the update is performed in-place.  There might be other caveats
that I don't remember; it's been a long while since I looked at that code.

Of course changing the status is not the only operation that may end up
rewriting the entire mbox.

Jeff.


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-22 Thread Derek Martin
On Fri, Jul 17, 2020 at 08:37:50AM +0200, Sebastian Stein wrote:
> Kevin J. McCarthy  [200716 18:18]:
> > If the access time is earlier than the modification time, it notifies for
> > new mail.
> 
> Wow, what a bad algorithm. I mean, this was probably perfect 20 years ago,
> but in times of desktop search engines, cloud backup services, etc.
> something more advanced is needed?!

Right, it's so terrible that users complain about it constantly. Oh
wait... no, they don't.

What makes it actually rather good is that it relies only on the
existing file metadata built into the POSIX file store, making it
pretty efficient (basically one stat() call).  Pretty much anything
else you can imagine will be significantly more expensive, and quite
possibly monumentally more expensive.

> When I open a folder, I can mark a mail as new. How does mutt keep track of
> this flag? Is this stored inside mbox file?

Yes.  It's stored in a message header in the individual message.  This
fact is unfortunately one of the things that slows mbox down, in cases
when it is slower than maildir (namely updating status, specifically,
in this case).  It did not take much thought to imagine that making
status updates faster for mbox only required invariably storing
messages with a fixed-length status header, where status updates could
simply overwrite the existing status field in place.  Sadly no one did
that.

> Why not have a sidecar like .L-file1.mutt, where some essential stuff like
> last access time, checksum, etc. are stored?

Well for starters you just made new mail detection AT LEAST 2x more
expensive, probably more. Instead of just a stat call you need to open
the "sidecar" and read it. And then you need to update it on every
message update.  This MAY be a small cost for the average user, but
Mutt's users are not generally average e-mail users, and it can add
up if you have a large number of mailboxes or large folders and/or
have a workflow that does lots of message status updates.

What you really want for something like this is a database that can
optimize for e-mail's usage patterns, which has other benefits as well
(like making virtual folders easy to implement).  But that has a
variety of down sides, most obviously quite a lot more complexity,
lack of any reasonable standard for cross-platform/mailer
compatibility (as far as I'm aware, at least) and that you can't
generally use text-oriented command line tools to manipulate it.  In
other words, it requires Mutt to abandon some of its fundamental
design principles.  Not that that's necessarily a bad thing... but if
you want something that doesn't work like Mutt, there are plenty of
other mail clients that don't work like Mutt.

I think the solution you're looking for is to fix or properly
configure your file system index service.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-22 Thread Derek Martin
Oops, if I'd seen this message before I sent my last post, I probably
wouldn't have bothered to post it.

That said, I will take issue with the notion that mbox is a terrible
format:  It isn't.  It does, however, have usage patterns for which
it is not well suited... just like maildir does.  If you've read the
famous comparison of the two on the Courier website, you should note
that while many of its claims are true, it nevertheless is in no small
part bunk.  And if you feel the need to argue the point, I would urge
you to read this thread before you do so:

https://mutt-users.mutt.narkive.com/OE3ugjuM/is-it-safe-to-use-mbox

And this one:

http://blu.org/pipermail/discuss/2003-September/017386.html

I'll grant you that one or two of the points I made may be outdated,
depending on what hardware and file system you're using and what
options you're mounting it with.  But mostly... probably not.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.


On Fri, Jul 17, 2020 at 10:31:07AM -0400, Josef 'Jeff' Sipek wrote:
> On Fri, Jul 17, 2020 at 08:37:50 +0200, Sebastian Stein wrote:
> > Kevin J. McCarthy  [200716 18:18]:
> > > If the access time is earlier than the modification time, it notifies for
> > > new mail.
> > 
> > Wow, what a bad algorithm.
> 
> It is a super cheap test with zero storage overhead.
> 
> > I mean, this was probably perfect 20 years ago,
> > but in times of desktop search engines, cloud backup services, etc.
> > something more advanced is needed?!
> 
> The atime/ctime comparison isn't perfect, but it is a well known algorithm
> that's been around for a long time.  One could make the argument that the
> desktop search engine is buggy if it messes with the atime of every file it
> looks at given what unix file system semantics are.
> 
> Finally, I'm not sure most people care.  I think most do not use mbox on a
> desktop/laptop, or if they do the use is limited.  (mbox is a horrible
> format for any sort of mailbox that sees modifications other than just
> appending)
> 
> > When I open a folder, I can mark a mail as new. How does mutt keep track of
> > this flag? Is this stored inside mbox file?
> 
> Yes, a header with this information is stuffed into the corresponding mail
> in the mbox.  This may require rewriting the whole mbox file - which is the
> big reason mbox is a terrible format :)
> 
> > Why not have a sidecar like .L-file1.mutt, where some essential stuff like
> > last access time, checksum, etc. are stored?
> 
> Because that's extra overhead and well...it isn't mbox anymore.  There are
> other mailbox formats that don't use atime/ctime comparison.
> 
> Server software like Dovecot maintains index files for each mailbox much
> like what you are suggesting.  While some of the functionality is IMAP or
> POP3 specific, the index code is generic and quite complex.  I doubt that
> anyone will want to spend the effort to implement this client-side.
> 
> Mutt could of course have a setting that disables the atime/ctime behavior -
> and every time it wants to check it could scan the mailbox to see if
> anything appeared.
> 
> Ultimately, I think this is really a bug in the desktop search engine you're
> using.
> 
> Jeff.




signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-17 Thread Josef 'Jeff' Sipek
On Fri, Jul 17, 2020 at 08:37:50 +0200, Sebastian Stein wrote:
> Kevin J. McCarthy  [200716 18:18]:
> > If the access time is earlier than the modification time, it notifies for
> > new mail.
> 
> Wow, what a bad algorithm.

It is a super cheap test with zero storage overhead.

> I mean, this was probably perfect 20 years ago,
> but in times of desktop search engines, cloud backup services, etc.
> something more advanced is needed?!

The atime/ctime comparison isn't perfect, but it is a well known algorithm
that's been around for a long time.  One could make the argument that the
desktop search engine is buggy if it messes with the atime of every file it
looks at given what unix file system semantics are.

Finally, I'm not sure most people care.  I think most do not use mbox on a
desktop/laptop, or if they do the use is limited.  (mbox is a horrible
format for any sort of mailbox that sees modifications other than just
appending)

> When I open a folder, I can mark a mail as new. How does mutt keep track of
> this flag? Is this stored inside mbox file?

Yes, a header with this information is stuffed into the corresponding mail
in the mbox.  This may require rewriting the whole mbox file - which is the
big reason mbox is a terrible format :)

> Why not have a sidecar like .L-file1.mutt, where some essential stuff like
> last access time, checksum, etc. are stored?

Because that's extra overhead and well...it isn't mbox anymore.  There are
other mailbox formats that don't use atime/ctime comparison.

Server software like Dovecot maintains index files for each mailbox much
like what you are suggesting.  While some of the functionality is IMAP or
POP3 specific, the index code is generic and quite complex.  I doubt that
anyone will want to spend the effort to implement this client-side.

Mutt could of course have a setting that disables the atime/ctime behavior -
and every time it wants to check it could scan the mailbox to see if
anything appeared.

Ultimately, I think this is really a bug in the desktop search engine you're
using.

Jeff.


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-16 Thread Sebastian Stein
Kevin J. McCarthy  [200716 18:18]:
> If the access time is earlier than the modification time, it notifies for
> new mail.

Wow, what a bad algorithm. I mean, this was probably perfect 20 years ago,
but in times of desktop search engines, cloud backup services, etc.
something more advanced is needed?!

When I open a folder, I can mark a mail as new. How does mutt keep track of
this flag? Is this stored inside mbox file?

Why not have a sidecar like .L-file1.mutt, where some essential stuff like
last access time, checksum, etc. are stored?


Regards,
Sebastian


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-16 Thread Kevin J. McCarthy

On Thu, Jul 16, 2020 at 09:03:59AM +0200, Sebastian Stein wrote:

Kevin J. McCarthy  [200715 08:28]:
Have you checked the access time vs modification time of those 
mailboxes?  Capture them before and after procmail, and then 
launching Mutt.  See if this gives any clues.


Can you describe the algorithm how to determine if a mbox file has new 
mails? Is this only done based on timestamps?


There are several configuration variables that can change the behavior 
($mail_check_recent, $check_mbox_size).  However, by default it's based 
on access time (Zugriff) vs modification time (Modifiziert).


If the access time is earlier than the modification time, it notifies 
for new mail.



╰─$ stat Mail/L-file1
Zugriff: 2020-07-16 08:36:57.869739259 +0200
Modifiziert: 2020-07-16 08:36:37.909074114 +0200



╰─$ stat Mail/L-file2
Zugriff: 2020-07-16 08:36:57.805727843 +0200
Modifiziert: 2020-07-16 08:36:37.688061706 +0200


For both L-file1 and L-file2, the access time is *later* than the 
modification time.  So something is reading the files before Mutt even 
launches.


What happens if other programs access mbox files? For example, what if 
some Kubuntu desktop search also reads those files?


If they don't reset the access time, this will interfere with new mail 
detection in mbox files.


So my conclusion would be that just based on those timestamps, mutt 
can't determine anything. But I guess mutt is keeping it's own last 
access time somewhere in those .L-file1.index and .L-file.index.ids 
files.


No, those files aren't owned by mutt.

I've now disabled Kubuntu desktop search, but not sure if this is the 
cause.


It sounds likely this is the problem.  Please let me know if that fixes
the problem.

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-16 Thread Sebastian Stein
Kevin J. McCarthy  [200715 08:28]:
> Have you checked the access time vs modification time of those mailboxes?
> Capture them before and after procmail, and then launching Mutt.  See if
> this gives any clues.

Can you describe the algorithm how to determine if a mbox file has new
mails? Is this only done based on timestamps? What happens if other programs
access mbox files? For example, what if some Kubuntu desktop search also
reads those files?

Following situation this morning:

- machine freshly started
- mutt not running
- running procmail
- new mails in 3 files:
o spool inbox
o Mail/L-file1
o Mail/L-file2
- me waiting 10 seconds after procmail completed
- opening mutt (defaults to spool inbox)
- mutt doesn't show any other unread folders (expected: L-file1, L-file2)

Here the output of stat before opening mutt:

╭─steinchen@steinchen ~
╰─$ stat Mail/L-file1
  Datei: Mail/L-file1
  Größe: 45266944   Blöcke: 88424  EA Block: 4096   Normale Datei
Gerät: 801h/2049d   Inode: 1978935 Verknüpfungen: 1
Zugriff: (0600/-rw---)  Uid: ( 1000/steinchen)   Gid: ( 1000/steinchen)
Zugriff: 2020-07-16 08:36:57.869739259 +0200
Modifiziert: 2020-07-16 08:36:37.909074114 +0200
Geändert   : 2020-07-16 08:36:37.909074114 +0200
 Geburt: -
 
╭─steinchen@steinchen ~
╰─$ stat Mail/L-file2
  Datei: Mail/L-file2
  Größe: 10009245   Blöcke: 19560  EA Block: 4096   Normale Datei
Gerät: 801h/2049d   Inode: 1978682 Verknüpfungen: 1
Zugriff: (0600/-rw---)  Uid: ( 1000/steinchen)   Gid: ( 1000/steinchen)
Zugriff: 2020-07-16 08:36:57.805727843 +0200
Modifiziert: 2020-07-16 08:36:37.688061706 +0200
Geändert   : 2020-07-16 08:36:37.688061706 +0200
 Geburt: -


So we can see that modification date (Modifiziert and Geändert) are set, but
also access (Zugriff) time before opening mutt. For L-file1, access time is
BEFORE modification and in case of L-file2 the file was accessed AFTER it
was modified.

So my conclusion would be that just based on those timestamps, mutt can't
determine anything. But I guess mutt is keeping it's own last access time
somewhere in those .L-file1.index and .L-file.index.ids files.

I've now disabled Kubuntu desktop search, but not sure if this is the cause.


Regards,
Sebastian


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-14 Thread Kevin J. McCarthy

On Tue, Jul 14, 2020 at 11:07:10PM +0200, Sebastian Stein wrote:

set folder="$HOME/Mail"
mailboxes `cd $HOME/Mail; echo P-*|sed -e 's/P-/+&/g'`
mailboxes `cd $HOME/Mail; echo L-*|sed -e 's/L-/+&/g'`


Since you are prefixing your mailboxes with '+' (as you should!), this 
precludes the problem being related to the one fixed for 1.14.6.


There are 283 mbox files counting about 4.000 characters. Is this too 
much?


No, it shouldn't be.

Is it expected that value of "mailboxes" variable isn't shown when 
running:


mutt -D


Mailboxes are not stored in a configuration variable, so that won't be 
dumped out by mutt -D.  Mutt doesn't provide a way to print out the 
current mailboxes list.  However the 'y' keybinding () 
will list them in the mailbox browser.



So far, I haven't found any clear way to reproduce it.


This is the important part though.  I've been staring at the code 
looking into Mike's problem.  I saw a few (old) race conditions I need 
to address eventually but not much else.  Without a simplified 
configuration and instructions to reproduce, I doubt I'll be able to 
find the problem.


I "think" it most often happens if several emails are added by procmail 
to several different mbox files. Only 1 or 2 of such folder files are 
marked as having new mails, all others are lost.


Have you checked the access time vs modification time of those 
mailboxes?  Capture them before and after procmail, and then launching 
Mutt.  See if this gives any clues.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-14 Thread Sebastian Stein
Kevin J. McCarthy  [200714 21:46]:
> Mike's reported bug was triggered by relative path mailboxes.  Instead of:
>   set folder = "~/Mail"
>   mailboxes =a =b =c
> he had:
>   set folder = "~/Mail"
>   mailboxes a b c
> where a, b, and c were then resolved relative to the mutt starting
> directory, instead of under $folder.  I made a mistake with the system call,
> forgetting a flag to ensure relative path behavior when resetting the
> access-time.
 
I have the following:

set folder="$HOME/Mail"
mailboxes `cd $HOME/Mail; echo P-*|sed -e 's/P-/+&/g'`
mailboxes `cd $HOME/Mail; echo L-*|sed -e 's/L-/+&/g'`

There are 283 mbox files counting about 4.000 characters. Is this too much?

Is it expected that value of "mailboxes" variable isn't shown when running:

mutt -D

Also inside mutt, running

:echo $mailboxes

doesn't output anything.

Some other values of (maybe) interest (via mutt -D):

mail_check=5
mail_check_recent is set
mail_check_stats is unset
mail_check_stats_interval=60

mbox="~/mbox"
mbox_type=mbox
move=no

browser_abbreviate_mailboxes is set
sort_browser=alpha

So far, I haven't found any clear way to reproduce it. I "think" it most
often happens if several emails are added by procmail to several different
mbox files. Only 1 or 2 of such folder files are marked as having new mails,
all others are lost.

I compiled 1.14.6 from source.

Official Ubuntu is still on mutt 1.13.2, but I first reported the problem
with version 1.10.1. So I think the problem might have been introduced with
version 1.10, because before Ubuntu had mutt 1.9, where I didn't encounter
this problem.

Please let me know for any additional information. Of course, you can also
contact me directly and I'm happy to share my config with you.


Regards,
Sebastian


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-14 Thread Kevin J. McCarthy

On Tue, Jul 14, 2020 at 08:26:07AM +0200, Sebastian Stein wrote:

Kevin J. McCarthy  [200708 08:59]:

> Perhaps some combination of settings is causing this.  Would you mind
> forwarding your configuration to me, so I can try to reproduce with
> that?

Thanks to Mike's help paring down his configuration to a minimal reproduce,
I was able to find the problem.

I'll get a patch to stable tomorrow and a release out later this week.


Yesterday I compiled 1.14.6 from source code, but still see it happening.
Today, I had several mails in 3 different folders (Mail/folder-1-file,
Mail/folder-2-file), but only one of them showed up as being new.


Your original mail (<20200106150335.ga10...@hpfsc.de>) reported you were 
seeing the problem with 1.10.1, but then it stopped with 1.13.2.


Unfortunately, the problem Mike reported first showed up in Mutt 1.11.0. 
If you are still having the problem with 1.14.6 it may be a different 
issue.


Mike's reported bug was triggered by relative path mailboxes.  Instead 
of:

  set folder = "~/Mail"
  mailboxes =a =b =c
he had:
  set folder = "~/Mail"
  mailboxes a b c
where a, b, and c were then resolved relative to the mutt starting 
directory, instead of under $folder.  I made a mistake with the system 
call, forgetting a flag to ensure relative path behavior when resetting 
the access-time.



Do you have any minimal mutt config I could try to see if the problem goes
away? My config was created like 20 years ago and not sure if there is
anything weird in.


If the above isn't the problem, I'm afraid I would need your help in
paring down to a minimal reproduce.

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-14 Thread Sebastian Stein
Kevin J. McCarthy  [200708 08:59]:
> > Perhaps some combination of settings is causing this.  Would you mind
> > forwarding your configuration to me, so I can try to reproduce with
> > that?
> 
> Thanks to Mike's help paring down his configuration to a minimal reproduce,
> I was able to find the problem.
> 
> I'll get a patch to stable tomorrow and a release out later this week.

Yesterday I compiled 1.14.6 from source code, but still see it happening.
Today, I had several mails in 3 different folders (Mail/folder-1-file,
Mail/folder-2-file), but only one of them showed up as being new.

I ran first fetchmail and procmail, before launching mutt.

Do you have any minimal mutt config I could try to see if the problem goes
away? My config was created like 20 years ago and not sure if there is
anything weird in.

Regards,
Sebastian


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-07 Thread Kevin J. McCarthy

On Tue, Jul 07, 2020 at 12:18:40PM -0700, Kevin J. McCarthy wrote:

On Tue, Jul 07, 2020 at 08:34:54PM +0200, mi...@posteo.nl wrote:

* On 07.07. Kevin J. McCarthy muttered:

Do you have $mail_check_stats set?


No.


Perhaps some combination of settings is causing this.  Would you mind 
forwarding your configuration to me, so I can try to reproduce with 
that?


Thanks to Mike's help paring down his configuration to a minimal 
reproduce, I was able to find the problem.


I'll get a patch to stable tomorrow and a release out later this week.

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-07 Thread Kevin J. McCarthy

On Tue, Jul 07, 2020 at 08:34:54PM +0200, mi...@posteo.nl wrote:

* On 07.07. Kevin J. McCarthy muttered:

Do you have $mail_check_stats set?


No.


Perhaps some combination of settings is causing this.  Would you mind 
forwarding your configuration to me, so I can try to reproduce with 
that?


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-07 Thread mike9
* On 07.07. Kevin J. McCarthy muttered:

> On Mon, Jul 06, 2020 at 08:39:49PM -0700, Kevin J. McCarthy wrote:
> >Also, I wonder if you would mind trying a vanilla mutt 1.14.5 release.

Here vanilla mutt 1.14.5 also looses the information of the
new-mail-folder like stated for Gentoo-mutt 1.13.5 or 1.14.4-r1.

> >A quick test on my side wasn't able to duplicate this,

Hm. Why do I encounter the issue with the same version? Can I
provide more information?

> Do you have $mail_check_stats set?

No.

Mike


Re: Mutt losing folders with new mails with additional session

2020-07-07 Thread Kevin J. McCarthy

On Mon, Jul 06, 2020 at 08:39:49PM -0700, Kevin J. McCarthy wrote:

Also, I wonder if you would mind trying a vanilla mutt 1.14.5 release.

A quick test on my side wasn't able to duplicate this, so it would be 
nice to make sure it isn't a Gentoo patch causing the problem.


After taking a closer look, I do have one theory about this, but it 
would be hard to trigger reliably.


Do you have $mail_check_stats set?

The sidebar code (merged back in 1.7.0) introduced a race condition for 
$mail_check_stats, because it resets the atime after scanning the mbox 
for message counts.  That works fine with just one Mutt running, but not 
so well if there are more than one.


I think the only way around it is to lock the mailbox, but I'll ponder 
it a bit more.  It would be nice to know if that's your problem though.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-06 Thread Kevin J. McCarthy

On Mon, Jul 06, 2020 at 08:12:38PM -0700, Kevin J. McCarthy wrote:

On Tue, Jul 07, 2020 at 04:49:37AM +0200, mi...@posteo.nl wrote:

My issue seems to be similar to this:
http://lists.mutt.org/pipermail/mutt-users/Week-of-Mon-20200106/001471.html


Mike, would you mind opening a ticket on gitlab with all these details?

I'll take a closer look and see what I can find.


Also, I wonder if you would mind trying a vanilla mutt 1.14.5 release.

A quick test on my side wasn't able to duplicate this, so it would be 
nice to make sure it isn't a Gentoo patch causing the problem.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: Mutt losing folders with new mails with additional session

2020-07-06 Thread Kevin J. McCarthy

On Tue, Jul 07, 2020 at 04:49:37AM +0200, mi...@posteo.nl wrote:

My issue seems to be similar to this:
http://lists.mutt.org/pipermail/mutt-users/Week-of-Mon-20200106/001471.html


Mike, would you mind opening a ticket on gitlab with all these details?

I'll take a closer look and see what I can find.

Thank you!

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Mutt losing folders with new mails with additional session

2020-07-06 Thread mike9
Hi,

My issue seems to be similar to this:
http://lists.mutt.org/pipermail/mutt-users/Week-of-Mon-20200106/001471.html

I'm running procmail and mbox folders on Gentoo. mutt-1.10.1 works fine.

When using mutt 1.13.5 or 1.14.4-r1 opening an additional session makes the
first session forget about folders with new mails.

Example:
- Folder a defined by 'mailboxes' command
- Start 'mutt -y' (session 1), switch to foder x
- Incoming mail in folder a,
  1.14.4-r1 shows new mail notice for folder a at once,
  1.10.1 shows new mail notice for folder a after key press (e.g. some
  cursor key)
- Hitting 'c' suggests switching to folder a, abort by Ctrl-g
- Start additional 'mutt -y' session.
  When running 1.10.1 session 2 shows 'N' flag for newest folder a but
  1.14.4-r1 does not
- Hitting 'c' in session 1 again (1.14.4-r1) does not suggest any folder
  anymore, 1.10.1 still does

Don't know if this is important: Without having changed the use flags
manually the ebuilds' IUSE variables of 1.14.4-r1 and 1.10.1 differ:
< autocrypt
> crypt
> gpg
> nntp
> notmuch
> smime

Any idea what causes the first session to forget about the folder
with new mail in versions >1.10.1?

Thank you.

Mike