On 3/30/07, Jim Fulton <[EMAIL PROTECTED]> wrote:
> What can be done to avoid that?

By "that" I assume you mean dangling referenced.

I really meant avoiding the PosKeyError. ;)

You could create extra references in the source database.  For
example, if you make a cross-database reference, you could also add a
corresponding reference in the source database.

Right. And removing this corresponding difference when the object in
the target database goes away.

> Not packing the databases at all?

That will work too. :)

Not an option!

> Would it be possible to make the loading of a cross-database reference
> that is gone return some sort of 'BrokenObject' instead of a
> PosKeyError, to work around the problem temporarily?

Sure.  I suppose the same could and perhaps should be done whenever
you would get a POSKeyError.

That sounds like a candidate for a ZODB proposal.

> Specially if the databases happen to be on different ZEO servers
> *wink*.

That is a complication, yes.  You would need some sort of distributed
GC protocol.

Ouch. :)

>> - A non-GC pack that got rid of old records but didn't bother with
>> GC.
>>    This would be advantagious for lots of folks independent of cross-
>> database reference issues.
> Sounds like this would be the easiest way to solve the above issue?

It is a fairly straightforward way, depending on the application.

The application is named Zope 2. You might have heard about it. :)

Unfortunately, it's probably non-trivial, as the FileStorage packing
code is fairly intense. :)

I've been wanting to redo this code for some time to reduce the
amount of disk I/O.  That would certainly be an opportunity to make
GC optional. But maybe it wouldn't be too hard to disable GC in the
current implementation -- I don't know.

Let's see. GC right now is done when the object has 0 references
pointing to it. A quick workaround would be making sure the code that
computes the references (referencesf?) always returns at least one
reference pointing to it?

>> Cross database references are pretty transparent and automatic.
>> Maybe there should be an option to make them less so.
> Yes, such an option would be great.

Maybe.  It depends on what semantics you want.  In fact, I don't know
how Zope 2 is setting up multi-databases.  If you don't want cross-
database references at all, you could change the startup code to open
separate databases without making them part of a multi-database.

Hm, brainstorming:

I suppose there could be could be an option that says you can't have
a cross reference unless the source database has a root key equal to
the object id of the source object with the value being the source
object.  Then, an intentional cross reference would be made like this::

   # Make a reference to foo, which is in another DB:
   foo._p_jar.root()[foo._p_oid] = foo
   self.x = foo

The code that creates cross-database references would, if this option
is available, do a check something like:

  if self.explicit_cross_database_references:
       if other._p_jar.root().get(other._p_oid) is not other:
           raise InvalidCrosDatabaseReference(...)

That looks good. If not for solving the problem completely, at least
for catching where in Zope implicit references might be created, and
possibly fixing those places.

I suspect you know what the patch is at this point -- at least to get
the pack done.  Perhaps you can try it and let us know how it goes.

Packing worked fine, and packing the other database didn't generate a
PosKeyError... yet. That makes me really nervous though. I will try to
come up with a patch that creates a BrokenObject where a PosKeyError
would occur in the case of multi databases.

Sidnei da Silva
Enfold Systems                http://enfoldsystems.com
Fax +1 832 201 8856     Office +1 713 942 2377 Ext 214
For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to