Re: [Zope-dev] Specialist/Rack scalability

2001-01-22 Thread Michael Bernstein
"Phillip J. Eby" wrote: Just to expand a little on the abov... Racks should scale at least as well, if not larger than a ZCatalog, given the same storage backing for the ZODB. This is because ZCatalog has to manage a minimum of one forward and reverse BTree for *each* index, plus another

[Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Michael Bernstein
After comsidering the fedback I got from the previous 'Massive scalability thread, I decided to split my queries into two areas: Rack scalability and ZCatalog scalability. This email deals with the former. It seems clear that indexing and searching are more of a botleneck than storage/retreival.

Re: [Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Steve Alexander
Hi Michael, Michael Bernstein wrote: It seems clear that indexing and searching are more of a botleneck than storage/retreival. Nevertheless, so far I have not heard of anyone trying to store more than 60,000 objects in a rack. I need to know if there is any reason to suspect that storage

Re: [Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Michael Bernstein
Steve Alexander wrote: Hi Michael, Michael Bernstein wrote: It seems clear that indexing and searching are more of a botleneck than storage/retreival. Nevertheless, so far I have not heard of anyone trying to store more than 60,000 objects in a rack. I need to know if there is

Re: [Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Steve Alexander
Michael Bernstein wrote: There is some question in my mind if accessing any attribute (such as the thumbnail version) causes all attributes to be loaded into memory. If so, displaying a list of images with thumbnails may result in many large objects being loaded into memory. Make sure

Re: [Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Michael Bernstein
Steve Alexander wrote: Michael Bernstein wrote: There is some question in my mind if accessing any attribute (such as the thumbnail version) causes all attributes to be loaded into memory. If so, displaying a list of images with thumbnails may result in many large objects being

Re: [Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Steve Alexander
Michael Bernstein wrote: Make sure that each large attribute is an instance of a class that derives from Persistent. Ok, I'll give that a try. Since Photo is a Python Product, what will happen to current instances if I make this (and only this) change? I don't know. I can think of

Re: [Zope-dev] Specialist/Rack scalability

2001-01-21 Thread Phillip J. Eby
At 07:12 PM 1/21/01 +, Steve Alexander wrote: So, storing things in a Rack happens in a number of stages: Your application interacts with the Rack The Rack (perhaps) stores the object persistently in its BTree The BTree is a collection of persistent ZODB objects The ZODB objects