-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I set the Reply-To to mailman-developers, since that's where we should move this discussion to.
On Feb 22, 2008, at 11:14 PM, Brad Knowles wrote: > On 2/22/08, Barry Warsaw wrote: > >>> Can we at least use a hashed mail directory solution that doesn't >>> have massive scalability problems? >> >> For the digests it probably doesn't matter because they'll never get >> that big. I'm still planning on making this change for the queue >> directories (though I haven't yet). > > I thought we were talking about replacing the 7th edition mbox file > format for the raw or the similar mbox-like format for the cooked > archives. Using some sort of a hashed directory structure would > allow a lot more flexibility in terms of going in and deleting or > editing messages in the archive, along with many other benefits it > might bring. > > However, if you imagine python-list with hundreds of thousands of > messages in the archive, there's just no way that could possibly > scale with Maildir, Maildir+, or any other solution that does not > enforce a good hashed directory scheme that is kept invisible to the > higher-level applications. The archives is an entirely different subsystem, and without a volunteer stepping forward (for which I've been practically begging for years ;), I don't see us being able to do any architectural changes to Pipermail, at least until after MM3 comes out, if ever. That's the unfortunate reality, but I do have thoughts about our archiver situation, which I'll outline at some future date. Actually, what I was talking about was the temporary digest mailbox. When Mailman receives messages destined for the list's digest, it holds these messages in a mailbox, currently mbox format, because that's all that older Python's supported. Then when Mailman is building the digest, it reads messages from this mbox, and chucks it when its done. So they never get that big, or live for that long. Still, mbox presents its own problems, such as what Mark outlined. One corrupted message, or a broken mbox delimiter horks the whole digest. I think we can do better with maildir and for the digest application, we don't have to worry about scalability. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkfAILwACgkQ2YZpQepbvXGm2ACggEuyXvTC7iV/32tK7y435VyL lNsAn2UPZg9QNKZUkWwZHDzBmkPYbz02 =L7kZ -----END PGP SIGNATURE----- ------------------------------------------------------ Mailman-Users mailing list [email protected] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp
