On Mon, Mar 21, 2011 at 01:29:15PM -0400, Wietse Venema wrote:
> I thought of that. This is not a problem with today's default
> configuration where the high-traffic queues (i.e. incoming, active)
> are not hashed.
>
> Do you expect that there will be configurations that do hash their
> high-traffic queues?
Scattering of queue files in the active queue is largely cost-less,
it is rarely walked. The queue manager knows where all the files are.
Essentially all "active" queue access is "random access". The active
queue should perhaps be hashed by default, this costs little and
can help with large active queues.
For the incoming queue, the queue-manager "walks" the queue pulling in
the first file found (modulo a few incomplete files) and after re-start
some files time-stamped into the future. So the queue manager is better
off with an unhashed incoming queue, which also improves fairness, with
traditional sequential directories. Of course cleanup adding files to
a long incoming queue may be slowed down, this is perhaps not entirely a
bad thing.
When directories are btrees, files sort by name, and the "locally
constant" hash directory + lexically monotone increasing queue ids lead
to better "fairness" if "incoming" is hashed, but perhaps hashing of
"incoming" should be discouraged. If so, perhaps we don't need to optimize
for "locally constant" sub-directories.
> > Also, with a little-endian base 52 queue-id, and hash depth of "2",
> > we have 2704 directories to search, while big-endian 32 x 32 takes us
> > to 977 directories as compared to 245 directories for big-endian base 16.
>
> Base 52 requires fewer levels of hashing than smaller bases, and
> 52 with depth 3 or more seems to make little sense. So that is a
> limitation in usability.
>
> I could just forget about lexicographical hashing and simply hash
> the hexadecimal representation of the microseconds (extracted from
> the queue file name and converted from base 52). With this there
> would be no change in file distribution compared to Postfix 2.8.
This is an interesting idea, also, no new sub-directories in existing
queues. Is it worth the effort though? The code gets complex when there
is a mixture of 2.8 and 2.9 style files in the queue.
--
Viktor.
