Re: [Zope-dev] 60GB Data.fs?
On Wed, 6 Jun 2001, Matt Hamilton wrote: > Out of interest, is this startup time avoided when using BerkeleyDB as > the storage? I know that it has its own indexes etc. so I am > wondering if it no longer needs to load an index into memory. I'm not sure, but as I said in a previous oops-correcting-myself-mail it actually only uses 40 secs - FileStorage, that is. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 60GB Data.fs?
On Wed, 6 Jun 2001, Erik Enge wrote: > I'm running a 1GB Data.fs with CompressedStorage here and that takes > probably about 3-5 minutes on a 1GHz with 1GB RAM. I keep banging my head > against it, but it just won't run faster. Out of interest, is this startup time avoided when using BerkeleyDB as the storage? I know that it has its own indexes etc. so I am wondering if it no longer needs to load an index into memory. Also how are the disks layed out? Is it possible to have, say, 5 disks each on their own (no RAID) and then split the data.fs over them using PartitionedFileStorage or similar. -Matt -- Matt Hamilton [EMAIL PROTECTED] Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Hosting | Web Design | Domain Names | Co-location | DB Integration ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 60GB Data.fs?
On Wed, 6 Jun 2001, Erik Enge wrote: > I'm running a 1GB Data.fs with CompressedStorage here and that takes > probably about 3-5 minutes on a 1GHz with 1GB RAM. I keep banging my > head against it, but it just won't run faster. Oops, misleading you there. Actually, FileStorage uses about 40 seconds to "initialize" the Data.fs. Can't complain about that. (Although 60GB might not be too fun ;) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 60GB Data.fs?
[no cross-posting, please] On Wed, 6 Jun 2001, Bjorn Stabell wrote: > We're planning a Yahoo! Clubs like system that should scale to about > 30, 000 users. Assuming about 3,000 groups and 20MB per group (group > functionality includes photo albums), gives a database size of 60GB. > Assuming on average 3,000 users per day, 20 page views per users, > gives about 60,000 page views (not a lot, but if it's all dynamically > generated?). You're going to need some serious hardware for that. You could do a lot with your setup though (ZEO, RDBMS, distributed application-programming) but I don't have much experience to share on that. In a scenario where each box (if you have several) has its own 60GB Data.fs I'd be worried about disk-activity for one. It seems to me (with my petty 1GB Data.fs) that it is the disks rather than ZODB itself that slows things down. > At this scale, how would ZODB hold up with respect to memory use and > speed? I've heard rumors that it loads an index into memory on > start-up. I'm running a 1GB Data.fs with CompressedStorage here and that takes probably about 3-5 minutes on a 1GHz with 1GB RAM. I keep banging my head against it, but it just won't run faster. Let us know how that project progress, will you? :) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
