I've been thinking about this for a while and I keep coming back to the
same answer.
seen_local is legacy and I wouldn't expect to find this in the wild
anymore. I don't think we should waste cycles doing anything with it.
I don't recall why seen_bigdb was created by one of my predecesors, but
its not used in production at CMU. I don't think its the way to go,
even with your Seen state changes.
The reason is that I think the distributed Seen state offered by seen_db
is the best for sites with a large number of shared mailboxes, such as
CMU. We currently have over 14,000 shared mailboxes that are called
bulletin boards on campus (used to be a lot more when we also had
non-binary newsgroups). And we have 10's of thousands of users reading
these mailboxes and maintaining their own Seen state. Using the current
divide and conquer approach where we keep each user's Seen state in a
separate database seems the most sane to me, rather than having several
hundred or thousand handles open to a single database.
Any change that will effect the performance or stability of CMU's
current environment would not be a good thing.
Bron Gondwana wrote:
At the moment Cyrus appears to support 3 seen backends:
* seen_local:
stores all the seen data for all users in a file in
the spool directory. Legacy.
* seen_db:
as far as I can see, everyone uses this. It's the only
one that replication's SETSEEN_ALL command works with
for sure.
* seen_bigdb:
one single database for ALL users seen data.
Now - I'm in two minds. I've already made one HUGE change
to how seen is handled, in that it's a system_flag in the
index record for the owner of the mailbox for user.*
mailboxes now. Also recentuid is in the index header for
the owner. This catches 99% of cases, reducing IO, since
compulsory CONDSTORE means we're always updating the
record for seen changes anyway.
So - in most cases there will be no $user.seen file any
more. I'm wondering if there is actually any benefit in
supporting three different operating modes for seen, or
if we should standardise on one. The choices are either
seen_db (advantage - less can go corrupt if anything
goes wrong) or seen_bigdb (advantage - only one file,
reduces the "stat" call and inode caching cost)
For that matter - if we standardised all $user.sub files
into a subscription.db, we'd cut yet another bunch of
tiny files. I'll probably leave that one alone for now,
since otherwise these changes will get totally out of
hand...
Speaking of which, I'm probably due to write another
update on how my future branch work is going!
Anyway - the reason I'm writing this is: I can see
that I'm going to need to provide a "seen_user_foreach"
API which calls a function with each given seen record
name... and I'm wondering if I should write 3 or just
not bother and standardise on one.
Bron.
--
Kenneth Murchison
Systems Programmer
Carnegie Mellon University
