thanks to all the responses. I will test with BTreeFolder2, and subfoldering with first character of id as another subfolder2, so that working with ZMI will also be possible without further scripting or customized views. i will report the results back to the list.
just an additional info: I run into this problem recently...
I've got something around 40k objects to insert into ZODB.
I started doing some tests with normal Folders, but I discarded it completely in favor of BTreeFolders. I got information that BTreeFolder was already tested with more than 400k objects... and that made me happy! :-)
But (there's always a 'but'!)...
Even using BTreeFolder to store all objects I was getting >300s delay to show a single object (ok, it's an archetypes-based one, containing >50 fields, splitted into 7 schematas, with lots of fancy stuff...). So I made a directory hash structure based on UID from each object (an AT UID is md5, so we have a hex base).
Using this hash structure, with 16 divisions, I got a better performance (but still far from acceptable): 'only' ~100s to show a single object... :-(
So I decided to create another level for the hash structure: now each folder has another 16 folder inside it. This time I was getting a delay of ~20s...
As you're already thinking... it's time for another hash level. Now, with 16^3 additional BTreeFolders to split all my objects I got an acceptable performance: ~3s...
The path is uggly as hell, f/f0/f0a/f0a5aac38aeff101b3168f2592dd879b, but at least the system is usable...
Resuming what I've learned: don't abuse of BTreeFolder, hash your content and live happy forever... ;-)
PS: The server is a modest PIII 1.2GHz, 1GB RAM with a 160MB/s SCSI controller running only one instance of Zope 2.7.1 without ZEO.
Dorneles Treméa Caxias do Sul - RS - Brasil +55 54 9114 9312 - UIN: 2413568 X3ng Web Technology <http://www.x3ng.com.br>
-----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS/IT d- s:->: a25 C+++ UBL++++$ P--- L++ E-- W+++ N++ o? K? w+ O M+ V-- PS+ PE- Y-- PGP++ t+ 5 X++ R+ tv+ b(++) DI+ D++ G+>+++ e++>++++ h---- r+++ y+++** ------END GEEK CODE BLOCK------
Zope-Dev maillist - [EMAIL PROTECTED]
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce