On 3/30/07, Jim Fulton <[EMAIL PROTECTED]> wrote:
> What can be done to avoid that?
By "that" I assume you mean dangling referenced.
I really meant avoiding the PosKeyError. ;)
You could create extra references in the source database. For
example, if you make a cross-database reference, you could also add a
corresponding reference in the source database.
Right. And removing this corresponding difference when the object in
the target database goes away.
> Not packing the databases at all?
That will work too. :)
Not an option!
> Would it be possible to make the loading of a cross-database reference
> that is gone return some sort of 'BrokenObject' instead of a
> PosKeyError, to work around the problem temporarily?
Sure. I suppose the same could and perhaps should be done whenever
you would get a POSKeyError.
That sounds like a candidate for a ZODB proposal.
> Specially if the databases happen to be on different ZEO servers
> *wink*.
That is a complication, yes. You would need some sort of distributed
GC protocol.
Ouch. :)
>> - A non-GC pack that got rid of old records but didn't bother with
>> GC.
>> This would be advantagious for lots of folks independent of cross-
>> database reference issues.
>
> Sounds like this would be the easiest way to solve the above issue?
It is a fairly straightforward way, depending on the application.
The application is named Zope 2. You might have heard about it. :)
Unfortunately, it's probably non-trivial, as the FileStorage packing
code is fairly intense. :)
I've been wanting to redo this code for some time to reduce the
amount of disk I/O. That would certainly be an opportunity to make
GC optional. But maybe it wouldn't be too hard to disable GC in the
current implementation -- I don't know.
Let's see. GC right now is done when the object has 0 references
pointing to it. A quick workaround would be making sure the code that
computes the references (referencesf?) always returns at least one
reference pointing to it?
>> Cross database references are pretty transparent and automatic.
>>
>> Maybe there should be an option to make them less so.
>
> Yes, such an option would be great.
Maybe. It depends on what semantics you want. In fact, I don't know
how Zope 2 is setting up multi-databases. If you don't want cross-
database references at all, you could change the startup code to open
separate databases without making them part of a multi-database.
Hm, brainstorming:
I suppose there could be could be an option that says you can't have
a cross reference unless the source database has a root key equal to
the object id of the source object with the value being the source
object. Then, an intentional cross reference would be made like this::
# Make a reference to foo, which is in another DB:
foo._p_jar.root()[foo._p_oid] = foo
self.x = foo
The code that creates cross-database references would, if this option
is available, do a check something like:
if self.explicit_cross_database_references:
if other._p_jar.root().get(other._p_oid) is not other:
raise InvalidCrosDatabaseReference(...)
That looks good. If not for solving the problem completely, at least
for catching where in Zope implicit references might be created, and
possibly fixing those places.
I suspect you know what the patch is at this point -- at least to get
the pack done. Perhaps you can try it and let us know how it goes.
Packing worked fine, and packing the other database didn't generate a
PosKeyError... yet. That makes me really nervous though. I will try to
come up with a patch that creates a BrokenObject where a PosKeyError
would occur in the case of multi databases.
--
Sidnei da Silva
Enfold Systems http://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
