Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-20 Thread Leonardo Rochael
Since we're talking about x-db POSKeyErrors, suppose I have a system with mounted databases an I want to move it into using a single database to avoid exactly the kinds of errors we've been talking here? More concretely, suppose a mounted database at /mountpoint that I want to get rid of. I was

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-18 Thread Christian Theune
Chris Withers schrieb: Jim Fulton wrote: It's a shame ZODB doesn't turn POSKeyErrors into proper Broken objects as it does when the class can no longer be imported. The problem with POSKeyErrors is that they prevent you accessing *any object that refers to the broken object* not just the

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-18 Thread Christian Theune
Hi, Chris Withers schrieb: Jim Fulton wrote: If someone can make something like this work without modifying ZODB, I won't object. The only thing I'd like is to be able to: - specify in policy that cross storage references should result in the target object being copied to the storage I

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-18 Thread Chris Withers
Christian Theune wrote: Okay, so I count two issues: - packing and multiple mounted storages - POSKeyErrors resulting in failure to load referring object rather than creation of a broken referred object Where would you like me to file these bug reports? How about the bugtracker? What's

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-18 Thread Christian Theune
Chris Withers schrieb: Christian Theune wrote: Okay, so I count two issues: - packing and multiple mounted storages - POSKeyErrors resulting in failure to load referring object rather than creation of a broken referred object Where would you like me to file these bug reports? How

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Binger David
On Feb 13, 2008, at 11:28 PM, Paul Winkler wrote: (Jim wrote) Catalogs and session data change much faster than content. It makes sense to pack these more frequently. On large system, it provides the ability to spread load over multiple servers, if necessary. Also, as I understand

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Christian Theune
Binger David schrieb: On Feb 13, 2008, at 11:28 PM, Paul Winkler wrote: (Jim wrote) Catalogs and session data change much faster than content. It makes sense to pack these more frequently. On large system, it provides the ability to spread load over multiple servers, if necessary.

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Jim Fulton
On Feb 13, 2008, at 11:28 PM, Paul Winkler wrote: On Wed, Feb 13, 2008 at 07:34:20PM -0500, Jim Fulton wrote: On Feb 13, 2008, at 6:14 PM, Binger David wrote: On Feb 13, 2008, at 3:58 PM, Jim Fulton wrote: Note that, IMO, some of the best use cases for multi databases are separating

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Jim Fulton
On Feb 14, 2008, at 7:21 AM, Binger David wrote: On Feb 13, 2008, at 11:28 PM, Paul Winkler wrote: (Jim wrote) Catalogs and session data change much faster than content. It makes sense to pack these more frequently. On large system, it provides the ability to spread load over

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Binger David
On Feb 14, 2008, at 9:43 AM, Jim Fulton wrote: On the other hand, it seems clear that cross-storage references make the system as a whole (software + people) less reliable. I don't agree in general. Uh. I thought we were talking about reported POSKeyErrors that would not exist if there

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Binger David wrote: It seems like all of these potential advantages are available through the use of multiple storages, but none of them really require database-level support for cross-storage references.On the other hand, it seems clear that cross-storage references make the system as a

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Binger David wrote: Uh. I thought we were talking about reported POSKeyErrors that would not exist if there were no such thing as cross-storage references. I don't understand what there is to disagree with here. Are you saying that cross-storage references reduce the risk of other

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Shane Hathaway wrote: Chris Withers wrote: I did try this: data._p_jar = app._p_jar data._p_oid = oid app.x = data import transaction transaction.get().note('Fix POSKeyError') transaction.commit() ...but it didn't work. I don't know if the differences are significant. app.x =

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Jim Fulton wrote: There's also a problem that we made cross-database references so easy to make that they are made accidentally. There's also no way to say actually, I want to *move* this object to this other storage. Semantically, that's what my customer was trying to do... make them

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Jim Fulton wrote: If someone can make something like this work without modifying ZODB, I won't object. The only thing I'd like is to be able to: - specify in policy that cross storage references should result in the target object being copied to the storage or - have an api to say where

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Binger David wrote: On Feb 13, 2008, at 3:58 PM, Jim Fulton wrote: Note that, IMO, some of the best use cases for multi databases are separating catalog and session data from regular content. Could you say more about what the benefit of this separation is, and why cross-storage references

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Jim Fulton
On Feb 12, 2008, at 7:43 PM, Shane Hathaway wrote: Chris Withers wrote: Anyway, lets get on and insert our 'broken' marker so that we don't get POSKeyErrors raised: import cStringIO import cPickle import transaction s = app._p_jar.db()._storage file = cStringIO.StringIO() p =

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Shane Hathaway
Chris Withers wrote: I did try this: data._p_jar = app._p_jar data._p_oid = oid app.x = data import transaction transaction.get().note('Fix POSKeyError') transaction.commit() ...but it didn't work. I don't know if the differences are significant. app.x = data won't work because

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Binger David
On Feb 13, 2008, at 11:49 AM, Marius Gedminas wrote: Data-loss bugs are *bad* Do all of these unfortunate conditions recently described go away if we don't have references to external persistent instances? Are there benefits of cross-storage references that justify this risk?

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Jim Fulton
On Feb 13, 2008, at 1:45 PM, Binger David wrote: On Feb 13, 2008, at 11:49 AM, Marius Gedminas wrote: Data-loss bugs are *bad* Do all of these unfortunate conditions recently described go away if we don't have references to external persistent instances? Are there benefits of

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Shane Hathaway
Jim Fulton wrote: Also, in the medium term, you can use zc.FileStorage to disable GC during packing. My plan is that this will become a built-in (and cleaner) FileStorage feature in ZODB 3.9. By disabling GC, you mean that packing retains at least one version of every object, correct?

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Jim Fulton
On Feb 13, 2008, at 5:40 PM, Shane Hathaway wrote: Jim Fulton wrote: Also, in the medium term, you can use zc.FileStorage to disable GC during packing. My plan is that this will become a built-in (and cleaner) FileStorage feature in ZODB 3.9. By disabling GC, you mean that packing retains

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Jim Fulton
On Feb 13, 2008, at 6:14 PM, Binger David wrote: On Feb 13, 2008, at 3:58 PM, Jim Fulton wrote: Note that, IMO, some of the best use cases for multi databases are separating catalog and session data from regular content. Could you say more about what the benefit of this separation is,

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Paul Winkler
On Wed, Feb 13, 2008 at 07:34:20PM -0500, Jim Fulton wrote: On Feb 13, 2008, at 6:14 PM, Binger David wrote: On Feb 13, 2008, at 3:58 PM, Jim Fulton wrote: Note that, IMO, some of the best use cases for multi databases are separating catalog and session data from regular content. Could

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-13 Thread Shane Hathaway
Paul Winkler wrote: We've been discussing doing this for openplans.org... one question that came up that none of us knew the answer to: If I undo some changes to content, what happens to the index values for that content? Does undo fire an ObjectModifiedEvent? Last time I looked it didn't.