On Mar 30, 2007, at 10:10 AM, Sidnei da Silva wrote:

On 3/30/07, Jim Fulton <[EMAIL PROTECTED]> wrote:
> Say there's two databases, and the
> object in database '1' is the only one referencing some object in
> database '2'. Since those references would be treated like weak
> references, wouldn't the object in database '2' go away in a 'pack', > thus causing a PosKeyError when loading the object from database '1'
> that still points to it?

Yes. That is a limitation of cross-database references.

What can be done to avoid that?

By "that" I assume you mean dangling referenced.

You could create extra references in the source database. For example, if you make a cross-database reference, you could also add a corresponding reference in the source database.

Not packing the databases at all?

That will work too. :)

Would it be possible to make the loading of a cross-database reference
that is gone return some sort of 'BrokenObject' instead of a
PosKeyError, to work around the problem temporarily?

Sure. I suppose the same could and perhaps should be done whenever you would get a POSKeyError.

Note that treating the references differently in a pack can't help
this as the references are in the referencing database, not the
referenced database.

It would be cool to have:

- A multi-database pack that took multiple databases into account.
   Such a feature is doable, but obviously, non-trivial.

Specially if the databases happen to be on different ZEO servers *wink*.

That is a complication, yes. You would need some sort of distributed GC protocol.

- A non-GC pack that got rid of old records but didn't bother with GC.
   This would be advantagious for lots of folks independent of cross-
database reference issues.

Sounds like this would be the easiest way to solve the above issue?

It is a fairly straightforward way, depending on the application. Unfortunately, it's probably non-trivial, as the FileStorage packing code is fairly intense. :)

I've been wanting to redo this code for some time to reduce the amount of disk I/O. That would certainly be an opportunity to make GC optional. But maybe it wouldn't be too hard to disable GC in the current implementation -- I don't know.

> Right. I'm wondering how I did end up with a cross-database reference. > Seems like a copy/paste through the ZMI caused it. My question is if
> that's expected/correct.

Cross database references are pretty transparent and automatic.

Maybe there should be an option to make them less so.

Yes, such an option would be great.

Maybe. It depends on what semantics you want. In fact, I don't know how Zope 2 is setting up multi-databases. If you don't want cross- database references at all, you could change the startup code to open separate databases without making them part of a multi-database.

Hm, brainstorming:

I suppose there could be could be an option that says you can't have a cross reference unless the source database has a root key equal to the object id of the source object with the value being the source object. Then, an intentional cross reference would be made like this::

  # Make a reference to foo, which is in another DB:
  foo._p_jar.root()[foo._p_oid] = foo
  self.x = foo

The code that creates cross-database references would, if this option is available, do a check something like:

 if self.explicit_cross_database_references:
      if other._p_jar.root().get(other._p_oid) is not other:
          raise InvalidCrosDatabaseReference(...)

>> What version of ZODB are you using?
> The one included with Zope 2.9.6.

Hm, are you using ZEO?

I'm wondering what sort of release would be needed to help you out.

Yes, I'm using ZEO. I don't need any sort of release particularly, a
patch would be fine. As long as there's a Zope 2.9.x release sometime
in the future for the people that can't live with patches, it should
be ok.

I suspect you know what the patch is at this point -- at least to get the pack done. Perhaps you can try it and let us know how it goes.


Jim Fulton                      mailto:[EMAIL PROTECTED]                Python 
CTO                             (540) 361-1714                  
Zope Corporation        http://www.zope.com             http://www.zope.org

For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to