Markus Stumpf <[EMAIL PROTECTED]> writes:
> I think the benefit is with OSs having poor directory access routines.
Such as Solaris/UFS (and, to my knowledge, all the *BSDs that use UFS
have not improved this much either.)
> As directories are scanned on a linear basis access is faster scanning
> two small directories than one very big one. Also modifications in
> the small directories (adding/removing files) will be faster.
UFS code, when looking for a filename, checks for a matching size,
then a matching first letter, second letter, and the rest: so it's
sub-optimal to have all your files in a single directory with the same
filename length and starting letters (as everyone running INN
eventually found out.) Each stat() or open() is a linear scan through
the directory to find the file.
> And (not sure about that, though) the first level directory is held
> in the filesystem cache as it has lots of accesses but does (usually)
> not change.
Whilst qmail-queue is scanning over todo/ I'd expect it to remain in
the various caches, with or without splitting up the directories.
Indeed, qmail already uses a split queue/mess/ directory structure and
it was a bit of an omission to assume that there would never be a
surge of mail in one go (VERP list expansion is definitely good for
creating this situation) and thus many messages in todo/ at once.
Further more, when there are many messages in todo/ the processing
overwhelmes the rest of the work that qmail-queue normally does and
slows down outbound deliveries to a crawl. All it takes is one of the
first people on a list to reply quickly and then you get another list
delivery, and yet more messages in todo/. Very messy business.
I've just started working on trying to collect (and thereby graph) as
much data as I can on qmail's operation - but one thing that isn't
recorded in the logs is the time between todo/ creation by
qmail-inject and 'new msg'.
If anyone has previously made changes to the code to accomplish this,
could they let me know and/or give me some pointers? Otherwise I'll
have to go digging. (Any patches to qmail-smtpd to log useful things
would also be of help to me: I've seen, and am looking at Maex's code.)
James.
