Alan Runyan wrote:
Could you provide a bit more guidance?   Maybe an example.

Here's one: in certain circumstances people want to aggregate lots of incoming data, but doing the obvious thing of updating a data structure as new data comes in turns out to be too slow. One way to handle that situation is to buffer incoming data and do a single aggregate write periodically (the approach taken by QueueCatalog).

Another simple one is wanting to keep up with a count, the naive way will get you in trouble (incrementing an attribute on an object), but using a the BTree.Length object helps a lot.

Most people come at ZODB with previous experience in RDBMS.

How do they map SQL INSERT/UPDATE activities to ZODB data structures?
In a way that does not create hotspot.

A few years ago people would say "pick a BTree implementation that maps
to your data structures and use that."  But that is naive and pushes the
hotspot into the BTree where concurrent updates are happening.

Naivety is relative. Most people don't push their RDBMS or ZODB to their limits, so a straight-forward approach is often best.

Inserting object references into an IOBTree, for example, seems reasonable, being careful to choose good integer keys so there's little chance of keys conflicting.

If your insert rate is high enough that bucket splits cause too many conflicts, then you'll have to deal with that. As I understand it one option is to choose your keys such that it is unlikely that two insertions happen in the same bucket.
--
Benji York
Senior Software Engineer
Zope Corporation
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev

Reply via email to