Re: [Zope] For the record
Peter Bengtsson wrote: So, to rewrite my benchmark I'd have to flush the cache between each fetch? I don't mind because the app itself has little traffic but individual pages are slow. If this is the case, you're looking in the wrong place for the source of the slowness ;-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] For the record
On 19/03/2008, Dieter Maurer <[EMAIL PROTECTED]> wrote: > Peter Bengtsson wrote at 2008-3-18 17:38 +: > > >I wanted to see how much faster brain.getObject() would be if the > >returned object was located in a BTreeFolder versus a normal folder so > >I wrote a benchmark that created about 10,000 light simple objects in > >two distinctly different containers and each with a ZCatalog that has > >a PathIndex. > > > >I would expect the BTreeFolder parenting calls of brain.getObject() to > >be much faster but I found that there was NO DIFFERENCE. wtf?! > > > If you access many (most) objects, these accesses may be even > more expensive with a "BTreeFolder" than with a standard folder. > This is not that difficult to explain: > > When you access a normal folder, with the folder all child > references are loaded. When you later access the folder > again (and it has not yet been flushed from the ZODB cache), > no loading is necessary. > > When you access a "BTreeFolder", no child references are loaded. > Thus, elementary access to a "BTreeFolder" is much more > efficient. > When you access a child in a "BTreeFolder", then all child > references are loaded that are in the same bucket as the > access child. In addition, all nodes on the path > from the "BTreeFolder" to this bucket are loaded. > When you access many children, you have a good chance, > that you have accessed at least on child in each bucket. > In this case, you have loaded all child references (as > in the standard folder case). In addition, you have > loaded the complete access structure. > > To summarize: an isolated access to a child of a large "BTreeFolder" > is usually much more efficient than an isolated access to a standard > folder with the same children. However, the advantage of the > "BTreeFolder" decreases when lots of children are accessed. > ...because they're cached in the normal folder. So, to rewrite my benchmark I'd have to flush the cache between each fetch? I don't mind because the app itself has little traffic but individual pages are slow. > > > -- > > Dieter > -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] For the record
Peter Bengtsson wrote at 2008-3-18 17:38 +: >I wanted to see how much faster brain.getObject() would be if the >returned object was located in a BTreeFolder versus a normal folder so >I wrote a benchmark that created about 10,000 light simple objects in >two distinctly different containers and each with a ZCatalog that has >a PathIndex. > >I would expect the BTreeFolder parenting calls of brain.getObject() to >be much faster but I found that there was NO DIFFERENCE. wtf?! If you access many (most) objects, these accesses may be even more expensive with a "BTreeFolder" than with a standard folder. This is not that difficult to explain: When you access a normal folder, with the folder all child references are loaded. When you later access the folder again (and it has not yet been flushed from the ZODB cache), no loading is necessary. When you access a "BTreeFolder", no child references are loaded. Thus, elementary access to a "BTreeFolder" is much more efficient. When you access a child in a "BTreeFolder", then all child references are loaded that are in the same bucket as the access child. In addition, all nodes on the path from the "BTreeFolder" to this bucket are loaded. When you access many children, you have a good chance, that you have accessed at least on child in each bucket. In this case, you have loaded all child references (as in the standard folder case). In addition, you have loaded the complete access structure. To summarize: an isolated access to a child of a large "BTreeFolder" is usually much more efficient than an isolated access to a standard folder with the same children. However, the advantage of the "BTreeFolder" decreases when lots of children are accessed. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] For the record
I wanted to see how much faster brain.getObject() would be if the returned object was located in a BTreeFolder versus a normal folder so I wrote a benchmark that created about 10,000 light simple objects in two distinctly different containers and each with a ZCatalog that has a PathIndex. I would expect the BTreeFolder parenting calls of brain.getObject() to be much faster but I found that there was NO DIFFERENCE. wtf?! The benchmark is too messy to share right now but how come this could have happened? getObject() basically does a parent.restrictedTraverse(path_splitted[-1]) which should be some sort of getattr() on the container right? Is there something obvious I've missed? -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
