On Sun, 2008-08-24 at 08:13 +0200, Dieter Maurer wrote: > Roché Compaan wrote at 2008-8-23 19:31 +0200: > >On Sat, 2008-08-23 at 14:09 +0200, Dieter Maurer wrote: > >> Roché Compaan wrote at 2008-8-22 14:49 +0200: > >> >I've been doing some benchmarks on Plone and got some surprising stats > >> >on the pickle size of btrees and their buckets that are persisted with > >> >each transaction. Surprising in the sense that they are very big in > >> >relation to the actual data indexed. I would appreciate it if somebody > >> >can help me understand what is going on, or just take a look to see if > >> >the sizes look normal. > >> > > >> >In the benchmark I add and index 10000 ATDocuments. I commit after each > >> >document to simulate a transaction per request environment. Each > >> >document has a 100 byte long description and 100 bytes in it's body. The > >> >total transaction size however is 40K in the beginning. The transaction > >> >sizes grow linearly to about 350K when reaching 10000 documents. > >> > >> The "Bucket" nodes store usually between 22 ("OOBucket") and 90 > >> ("IIBucket") > >> objects in a single bucket. > >> > >> With any change, the transaction will contain unmodified data > >> for several dozens other objects. > > > >Are you saying *all* 22 OOBuckets and 90 IIBuckets will be persisted > >again whether they are modified or not? > > I did not speak of "22 OOBuckets" but of typically 22 entries in an > "OOBucket" (similarly for "IIBucket"). > > And indeed, when a single entry in an "OOBucket" is changed, then all > entries are rewritten even if the other entries did not change. > > That is because the ZODB load/store granularity is the persistent object > (without persistent subobjects). An "OOBucket" is a persistent object -- > it is loaded/stored always as a whole (all entries together).
Yes, that's how I understand it. I misunderstood your original statement and thought that there is something else at play here. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev