Re: [Zope-dev] Re: Converting from old-style BTreess
On Mon, Dec 15, 2003 at 02:47:01AM -0500, Jim Fulton wrote: shrug Who knows. I'd rather be safe. It's not that hard. In any case, we would have needed the fix to handle old pickles correctly. Asking a possibly redundant question... not being much of a ZODB hacker, I didn't follow this thread in detail... Q: Will Zope = 2.8 be able to import .zexp's exported from zope = 2.6.2? From Jim's comment above I would guess yes, but I need to be sure. Some of us out here in the wild use import / export machinery pretty heavily (see e.g. ZSyncer), and we typically like to burn in a new zope version on a dev system for some time before upgrading the production systems... but during that time we need to keep syncing stuff from dev to production. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's FOOD-COVERED STRAIGHT FROM UNCLE! (random hero from isometric.spaceninja.com) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Converting from old-style BTreess
Chris McDonough wrote: On Sun, 2003-12-14 at 23:02, Chris McDonough wrote: Here's an idea: create a simple script that manufactured a module which set up the Zope configuration, filled in for BTree in sys.modules, and got a hold of Zope.app(), causing ZGlobals to be replaced by a new-style BTree due to the above magic. People upgrading to 2.8 with databases created by Zope versions older than 2.3 would still need to run convertBTrees inside a different Zope version, but people upgrading databases created in versions equalling or later than 2.3 wouldn't because we stopped using the old BTree module for anything except ZGlobals after 2.3 AFAICT. Actually, given my findings later in this email, this isn't even necessary. Right. Whew. I checked in the necessary fix for PersistentMapping. There was a change in the data structure and I had to add some logic to convert some old state. I also had to check in a %$#@ travesty because Data.fs.in still has references to BoboPOS. With these changes, I can create a database with 2.6 and open it with the head. Thanks Chris for helping us figure out that this was easier than we thought. :) Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Converting from old-style BTreess
Chris McDonough wrote: ... Out of curiosity, could you explain why it matters in this context whether it's an extensionclass or not? Is it because there might be a set of people using PersistentMapping objects for whom its important that they be able to use extenionclass semantics against them? I'd just never thought of using __of__ or inheritedAttribute or any of the other EC-specific stuff on a PersistentMapping. shrug Who knows. I'd rather be safe. It's not that hard. In any case, we would have needed the fix to handle old pickles correctly. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Converting from old-style BTreess
On Mon, 2003-12-15 at 02:47, Jim Fulton wrote: Chris McDonough wrote: ... Out of curiosity, could you explain why it matters in this context whether it's an extensionclass or not? Is it because there might be a set of people using PersistentMapping objects for whom its important that they be able to use extenionclass semantics against them? I'd just never thought of using __of__ or inheritedAttribute or any of the other EC-specific stuff on a PersistentMapping. shrug Who knows. I'd rather be safe. It's not that hard. In any case, we would have needed the fix to handle old pickles correctly. Gotcha. Can you answer a question about the features we're allowed now? If we don't use extensionclass, is it possible to persist instances of new-style classes in ZODB now (ie. is persistent.Persistent really a new-style type/class)? - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Converting from old-style BTreess
On Mon, 2003-12-15 at 02:39, Jim Fulton wrote: I checked in the necessary fix for PersistentMapping. There was a change in the data structure and I had to add some logic to convert some old state. Oh, geez, I remember this setstate/getstate business now. FWIW, I think Jeremy added it at my insistence after he changed PersistentMapping to inherit from UserDict in order to maintain backwards compatibility for versions that didn't. The next panic will start when UserDict becomes a new-style class. ;-) - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Converting from old-style BTreess
On Mon, Dec 15, 2003 at 05:21:55AM -0500, Jim Fulton wrote: Chris McDonough wrote: Woot! ;-) Descriptors, yum... But you can use descriptors in extension classes now too. I wondered what the heck you guys were on about... google turned up this: http://users.rcn.com/python/download/Descriptor.htm very very cool stuff. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's ZORRO SLUDGE! (random hero from isometric.spaceninja.com) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Converting from old-style BTreess
Chris McDonough wrote: I think this problem is actually easier to solve in the common case than requring that people run a conversion script. The ZGlobals data structure is a cache, and can be blown away and recreated indiscriminately. It's recreated whenever a Product is added or removed, and Zope already does this as a matter of course if anything about the data structure is goofy. Hm, OK. Then the question is how to decide that it's goofy. I'll note that I certainly did not intend it to be a cache when I added it a long time ago. I'm a bit uncomfortable assuming that it's always used that way, but you know more than I about the current usage. I think the problem Sidnei has run into isn't so much that the BTree module can't be imported, it's instead that the Persistence.PersistentMapping module redefines PersistentMapping in order to do something I don't quite understand, and the redefinition is what appears to be causing the AttributeError, 'data' to appear. Hm, I'll add a comment explaining what's going on. If it's not used, everything works fine. Here's the module: import Persistence import persistent from persistent.mapping import PersistentMapping if Persistence.Persistent is not persistent.Persistent: class PersistentMapping(Persistence.Persistent, PersistentMapping): Legacy persistent mapping class This class mixes in ExtensionClass Base if it is present. Unless you actually want ExtensionClass semantics, use persistent.mapping.PersistentMapping instead. If you comment out the if clause and everything underneath of it, make a dummy BTree.py module with the following and put in in the software home: from Globals import Persistent class BTree(Persistent): pass ... and restart Zope with an old filestorage file, all is well: There needs to be a cleaner solution than this. A Dummy BTree module is not acceptable. ... I'm not sure what the redefinition is supposed to achieve? persistent.Persistent is not an extension class. It's a pure new-style persistent base class. Persistent.Persistent is an extension class if ExtensionClass is installed. If ExtensionClass is not installed, then a warning is issued and it is set tp persistent.Persistent. This is to ease the transition for people who use ZODB outside of Zope. The check above is to cause an extension-class-based PersistentMapping is ExtensionClass is installed. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Converting from old-style BTreess
On Sun, 2003-12-14 at 22:08, Jim Fulton wrote:
Chris McDonough wrote:
I think this problem is actually easier to solve in the common case than
requring that people run a conversion script. The ZGlobals data
structure is a cache, and can be blown away and recreated
indiscriminately. It's recreated whenever a Product is added or
removed, and Zope already does this as a matter of course if anything
about the data structure is goofy.
Hm, OK. Then the question is how to decide that it's goofy.
The current way it's done is in OFS.Application.Application via:
def checkGlobalRegistry(self):
Check the global (zclass) registry for problems, which can
be caused by things like disk-based products being deleted.
Return true if a problem is found
try:
keys=list(self._p_jar.root()['ZGlobals'].keys())
except:
LOG('Zope', ERROR,
'A problem was found when checking the global product '\
'registry. This is probably due to a Product being '\
'uninstalled or renamed. The traceback follows.',
error=sys.exc_info())
return 1
return 0
If the check fails, the registry is blown away and rebuilt in the same
class' fixupZClassDependencies method which does a bunch of sniffing
around in Products for zclass ids. I admit I have no idea whether this
check is valid and haven't had to care in a long time. It's been there
since before 2.3, however.
I'll note that I certainly did not intend it to be a cache when
I added it a long time ago. I'm a bit uncomfortable assuming that
it's always used that way, but you know more than I about the current usage.
Probably not, but I do know that there haven't been many complaints
about the rebuilding code.
If you comment out the if clause and everything underneath of it, make
a dummy BTree.py module with the following and put in in the software
home:
from Globals import Persistent
class BTree(Persistent):
pass
... and restart Zope with an old filestorage file, all is well:
There needs to be a cleaner solution than this. A Dummy BTree module is
not acceptable.
Here's an idea: create a simple script that manufactured a module which
set up the Zope configuration, filled in for BTree in sys.modules, and
got a hold of Zope.app(), causing ZGlobals to be replaced by a new-style
BTree due to the above magic. People upgrading to 2.8 with databases
created by Zope versions older than 2.3 would still need to run
convertBTrees inside a different Zope version, but people upgrading
databases created in versions equalling or later than 2.3 wouldn't
because we stopped using the old BTree module for anything except
ZGlobals after 2.3 AFAICT.
I'm not sure what the redefinition is supposed to achieve?
persistent.Persistent is not an extension class. It's a pure
new-style persistent base class.
Out of curiosity, could you explain why it matters in this context
whether it's an extensionclass or not? Is it because there might be a
set of people using PersistentMapping objects for whom its important
that they be able to use extenionclass semantics against them? I'd just
never thought of using __of__ or inheritedAttribute or any of the other
EC-specific stuff on a PersistentMapping.
Persistent.Persistent is an extension class if ExtensionClass is
installed. If ExtensionClass is not installed, then a warning is issued and
it is set tp persistent.Persistent. This is to ease the transition for
people who use ZODB outside of Zope.
I see.. thanks.
Well, even in the case that a (dummy or real) BTree module doesn't
exist, if Zope is told to use the persistent.mapping.PersistentMapping
class instead of the one that inherits from Persistence.Persistent, it
does not fail in the way Sidnei reported. Additionally, if you switch
the redefined PersistentMapping's base class order, it also begins to
work. The fact it can't import BTree.BTree causes it to complain, but
the ZGlobals rebuilder does begin work under both circumstances and Zope
starts.
--
2003-12-14T22:45:49 ERROR(200) ZODB Couldn't load state for
000a
Traceback (most recent call last):
File /home/chrism/software/Trunk/lib/python/ZODB/Connection.py, line
428, in setstate
self._reader.setGhostState(obj, p)
File /home/chrism/software/Trunk/lib/python/ZODB/serialize.py, line
203, in setGhostState
obj.__setstate__(state)
TypeError: function takes exactly 2 arguments (0 given)
--
2003-12-14T22:33:35 ERROR(200) Zope A problem was found when checking
the global product registry. This is probably due to a Product being
uninstalled or renamed. The traceback follows.
Traceback (most recent call last):
File /home/chrism/software/Trunk/lib/python/OFS/Application.py, line
233, in checkGlobalRegistry
keys=list(self._p_jar.root()['ZGlobals'].keys())
File /home/chrism/software/Trunk/lib/python/ZODB/Connection.py,
Re: [Zope-dev] Re: Converting from old-style BTreess
On Sun, 2003-12-14 at 23:02, Chris McDonough wrote: Here's an idea: create a simple script that manufactured a module which set up the Zope configuration, filled in for BTree in sys.modules, and got a hold of Zope.app(), causing ZGlobals to be replaced by a new-style BTree due to the above magic. People upgrading to 2.8 with databases created by Zope versions older than 2.3 would still need to run convertBTrees inside a different Zope version, but people upgrading databases created in versions equalling or later than 2.3 wouldn't because we stopped using the old BTree module for anything except ZGlobals after 2.3 AFAICT. Actually, given my findings later in this email, this isn't even necessary. If it's sensible to turn the inheritance ordering around for the EC-based persistent mapping, it appears that no conversion script needs be run at all. We might catch the error and report it more nicely in checkGlobalRegistry, but that's all that needs be done by programmers and users need do nothing except copy their database over. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
