Re: [ZODB-Dev] large database files
On Wed, Jul 23, 2008 at 2:01 AM, Sean Allen [EMAIL PROTECTED] wrote: step 1, move about 10% of our data over to get an idea about speed of searching 1.5 million entries etc. standard file storage... the csv file i dumped all the info to is about 190 megs, the resulting file storage file is 750 megs ( 1.8 gigs before packing ). Does this mean, that I should expect the entire database to close in at something like 7.5 gigs? Not a bad guess. If yes, are there problems that can arise for a database of that size? You should be fine. That's a small database by some standards. -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] large database files
On Wed, Jul 23, 2008 at 2:01 AM, Sean Allen [EMAIL PROTECTED] wrote: i'm starting to do some serious playing around with zodb. step 1, move about 10% of our data over to get an idea about speed of searching 1.5 million entries etc. standard file storage... the csv file i dumped all the info to is about 190 megs, the resulting file storage file is 750 megs ( 1.8 gigs before packing ). Does this mean, that I should expect the entire database (After reading your other message about indexing); if you aren't doing any indexing yet, then you should expect a larger database once the indexes are added. Some advocate keeping indexes in a separate database, especially if they are a significant fraction of the total database size. That way you can have different packing schedules, backup plans, storage devices, etc. -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] indexing
Sean Allen [EMAIL PROTECTED] wrote on 07/22/2008 11:44:11 PM: googling and the main zodb website seem to take me to a bunch of what appears to be severely dated websites, most not updated in at least 2 years ( some upwards of 6 ). this is probably because i simply don't know what to google for, so, could someone be so kind as to shoot some links to different indexing options for use with zodb/zeo? I've just started playing around with ZODB myself. We have a database of ~650,000 items, with dates and multiple tags. For indexing, I'm using the zc.catalog package from PyPi, with a value index for the dates and a set index for the tags. It seems good enough. There were other indexing classes, but I don't remember any of them. I've found the ZODB website to be very disorganized an not nearly as helpful as repeated googlings. -- Anthony Foglia Princeton Consultants (609) 987-8787 x233___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
