At 11:52 AM -0800 2005-02-08, Carl Zwanzig wrote:
3. Any general advice for handling ridiculously large lists?
You might want to look at http://www.python.org/doc/faq/library.html#id35, otherwise, I think you're hacking MM to default to the marshal method. The code is there, but as the backup method. (Or really hacking it to use mysql...)
Using Berkeley db is pretty blindingly fast, and if done right, should result in an extremely reliable database. MySQL uses BerkeleyDB as their way of achieving full ACID compliance in their high-end MaxSQL database product. Rock-damn-bloody-solid stuff, if done right. But also easier to screw up.
If you can replace the marshal method with one based on Berkeley DB, I imagine that will probably solve most of the performance problems right there. Assuming you choose the right type of file format, that is. I imagine that a btree is likely to perform better in this application than a hash table.
Alternatively, a Berkeley DB-based member adaptor should not be too excessively difficult to code. However, if such a beast doesn't already exist, you might want to first try the LDAP member adaptor.
-- Brad Knowles, <[EMAIL PROTECTED]>
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755SAGE member since 1995. See <http://www.sage.org/> for more info. ------------------------------------------------------ Mailman-Users mailing list [email protected] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
