Quick note...
Smaller buckets, fewer conflicts, more overhead on reading and writing.
Larger buckets, more conflicts, less overhead on reading and writing.
One bucket ... constant conflicts.
I'd bet that the additional tree with tiny buckets would be best.
Transfer them into the normal tree once the overhead starts rising.
How to transfer them? You would not want a single transaction to
take the hit for the whole transfer, so have a low and high water
mark. When hitting HWM, transfer only until LWM is reached. Or, just
focus on transferring some items out of a single tree, to avoid the
cost of tree rebalancing on the additional tree at the same time as
rebalancing on the main tree.
Sounds like a fin project,
--r.
On 1 Nov 2007, at 21:00, David Binger wrote:
On Nov 1, 2007, at 4:25 PM, Matt Hamilton wrote:
David Binger <dbinger <at> mems-exchange.org> writes:
On Nov 1, 2007, at 7:05 AM, Matt Hamilton wrote:
Ie we perhaps look at a catalog data structure
in which writes are initially done to some kind of queue then moved
to the
BTrees at a later point.
A suggestion: use a pair of BTrees, one with a high branching factor
(bucket size)
and one with a very low branching factor. Force all writes into the
tree with little
buckets. Make every search look in both trees. Consolidate
occasionally.
An interesting idea. Surely we need the opposite though, and that
is an
additional BTree with a very large bucket size, as we want to
minimize the
chance of a bucket split when inserting? Then we occasionally
consolidate and
move the items in the original BTree with the regular bucket size/
branch factor.
You may be right about that. Conflict resolution makes it harder for
me to predict which way is better. If you don't have conflict
resolution
for insertions, then I think the smaller buckets are definitely better
for avoiding conflicts. In either case, smaller buckets reduce the
size and serialization time of the insertion transactions, and that
alone
*might* be a reason to favor them. I think I'd still bet on smaller
buckets, but tests would expose the trade-offs.
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Russ Ferriday
[EMAIL PROTECTED]
office: +44 118 3217026
mobile: +44 7789 338868
skype: ferriday
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
