On Tue, Nov 16, 2021 at 11:41 (-0500), Kris Deugau wrote:
> Jim wrote:
>> On Mon, Nov 15, 2021 at 12:25 (-0500), Wietse Venema wrote:
>>> Instead, use Maildir format with one message per file,
>> I thought about that once, but I decided I have too many e-mail
>> messages for that. (I don't want to run out of inodes, nor do I want to
>> make file accesses too slow because of the number of files in the
>> directory.)
> I converted "local"[*] storage from mbox to maildir a number of
> years ago - IIRC I was starting to see performance issues with mbox
> in part due to the way I manage my mail and in part simply due to
> the number of messages I keep around.
> This account has ~13G of mail on my PC, with over 100K messages each
> in two folders, several in the tens of thousands, and most dedicated
> mailing list folders holding somewhere between about 5K and 8K
> messages each.
Thanks for the specifics.
> The only performance issues I have are:
> a) something sucks in the the IMAP protocol such that my mail client keeps
> having to create a new connection and reauthenticate - it's not strictly a
> timeout, because it's not on anything remotely resembling a predictable
> timing
At first glance I wouldn't see that related to mbox vs. maildir, but
I've been surprised before.
> Local storage is ext4 on a SATA SSD, although I wouldn't expect a noticeable
> performance difference if it were on a conventional hard drive.
I am surprised that accessing files in a directory with 100K entries
is not slow, since (according to what I read) ext4 stores entries in
an "almost linear" list, and thus to find a director entry you might
have to chew through (on average) 50K entries. Of course, file system
caching will speed things up immensely, assuming one has enough RAM
(given the other activity on the system) to keep the contents of those
maildirs (that is, the directory contents, not the contents of the
files) in RAM.
> [*] Due to some legacy mail flow that would be painful to convert, I
> pull mail with fetchmail, deliver locally with procmail (sorry),
> then expose it to my mail client with a local Dovecot instance.
Again, thanks for your specifics. Maybe I should give maildir a try
some time and see what happens. (Or maybe I should just delete a bunch
of email and forget that I ever got it.)
I see that I have lots on inodes on the file system where I keep my
email: although the file system is 48% full, I have used only 3% of
the inodes, so I'm in no danger of running out. If I used another
160,000 for mail messages I'd still be less than 6% inodes used, so
that turns out not to be a concern for me. (The last time I
considered doing this I don't think I had such a surplus of inodes.)
Cheers.
Jim
