Caveat to this below.
On Mon, Jul 15, 2013 at 7:54 AM, Marius Gedminas <mar...@gedmin.as> wrote:
> On Mon, Jul 15, 2013 at 03:36:21PM +0200, Pedro Ferreira wrote:
> > We need to move a considerable number of persistent objects (~40K) from
> > one module to another, and we were wondering what is the best strategy
> > to follow.
> ...
> > Has anyone ever done anything like this? Which approach have you
> > followed? Any suggestions?
>
> 1. Make sure all the classes are still importable from the old location
> ('from newmodule import MyPersistentSomething # BBB')
>
> That is sufficient to make it work. If you also want to eradicate all
> references to the old module name from your ZODB (e.g. because you'd
> like to remove the BBB import), then proceed to step 2:
>
> 2. Write a script that loads every instance of this class and does a
>
> obj._p_activate() # actually not sure this is required, but won't hurt
> obj._p_changed = True
>
> and then commit the transaction. Do the commit multiple times, after
> each batch of several hundred objects, to avoid excessive memory usage.
> Also be sure to handle conflict errors and retry that batch if you're
> running this script on the live system. Finding all instances is left
> as an exercise for the reader (sometimes findObjectsProviding() helps,
> if you use nested containers everywhere; sometimes application-specific
> logic works best; sometimes you end up having to use ZODB iterators to
> loop through every single object in the DB -- I believe zodbupdate does
> that.)
>
This works only partially, AFAICT. It will update the stored class name of
a persistent object. It will not seek out and change the class name of the
object in a reference. Your step 3 (removing the BBB code) can break
things (broken objects) as a result of this. Fixing persistent objects is
half the battle if you get broken references to them.
You can verify that this is incomplete by creating a PersistentMapping of
some simple objects, perform the trick above, commit, then pack. You will
still have the old BBB classname stored in the database in the references,
unless you do a _p_changed=1 on the mapping containing/referencing the
items as well (then commit).
I am not sure how to walk/iterate all oids for all transactions to get all
possible referencing objects (I assume this is storage-specific, maybe
building a reference map like Lawrence Rowe has done [1]). This might be
necessary to update the referencing objects?
[1] http://plone.org/documentation/kb/debug-zodb-bloat/inspectZodbUtils.py
Sean
_______________________________________________
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
