On Mar 23, 2008, at 3:59 AM, Roché Compaan wrote:

On Sat, 2008-03-22 at 15:49 -0400, Sean Allen wrote:
Ha! Ok.

Questions then from there.

Why not store each object type,

customer, order etc in their own folders?

You could, and I understand that you want to do so coming from a RDBMS world, but if you are working with objects then you have the luxury of
organising content in a way that is closer to reality. I think it is
rather convenient to store a customer's orders inside it.

I can see the convenience. Ok, so here is a big question...
Assuming you did store each object in its own folder and you had

    has many orders

At what point do you have to hook into zodb so that if you say did this:

customer = Customer( ... )
customer.addOrder( Order( ... ) )

that the order in question when customer is saved, gets saved to its
own folder
and then customer gets a reference to that now saved persistent order
before it itself is saved.

You first create the order inside the orders folder, then you reference
it from the customer:

order = Order()
orderfolder['order123'] = order
# assuming you have a folder for orders in each customer, the following
# will create a reference since the above code already persisted the
# object.
customer.orders['order123'] = order

I don't see why you need the reference though. Why not just create the
order directly in the order folder for the relevant customer:

order = Order()
customer.orders['order123'] = order

Can that be wrapped in a transaction so that if something goes wrong
with customer creation, the order gets rolled back as well?

Or put higher level, do transactions only work within a single folder
or across folders?

There is another good reason - it makes a lot more sense to distribute
object creation to different containers in the ZODB to avoid write
conflict errors.

I think I understand this but can you elaborate?

By 'distribute object creation to different containers' do you mean
store each
object in own folders?

No that would lead to an unnecessary proliferation of folders and it
wouldn't actually remove conflict errors.

Conflicts errors occur when two concurrent transactions modify the same attribute. When you are putting objects in a folder you are essentially
modifying the same attribute (unless you are using a btree). A good
application design strategy for the ZODB is to ensure that concurrent
transactions are modifying attributes on different objects. If you are
putting all your orders in one folder you are increasing the chances of
conflict errors.

If you dont put all orders in a single folder, when it comes time to
search orders, how would you know where to find?

Do you sequentially search the multiple folders for a given object?

Like I mentioned above you could use a btree folder since it has a
fairly good conflict resolution strategy. But given a high enough
insertion rate even a btree will not protect you against conflict
errors. I would recommend you create orders inside a btree-based order
folder inside each customer. This will significantly reduce the
likeliness of conflict errors.

So if this occurs, you get an exception ( basically optimistic locking )
which you then deal with on a higher level in your application?


the database i am looking at moving over has

1.2 million entries in a transactions table
980,000 orders
775,000 customers
1.5 million order items

I think these numbers are quite manageable. But think carefully what
kind of queries you want to do on the data. You have a very rich query language in SQL that allows the construction of complex queries and it
will come naturally to you. To do the same in the ZODB will take

Are there any mistakes that people usually make when doing this sort
of mental context switch that you can make me aware of now?

Any good reading on the subject?

Either there isnt a ton of information on this on the net or my google
are slipping.

The most valuable resource is the mailing list. Unfortunately nobody has
documented their experience elsewhere before.

Well, if I go with zeo/zodb and python, I'll be sure to document the entire thing.

For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to