Re: [ZODB-Dev] Disappearing volatile attributes in ZODB 3.6
Terry Jones wrote at 2007-1-12 20:58 +0100: I have a top-level class (call it ABC) that I only ever want one instance of. In __init__ of that class, it uses a couple of _v_ attributes to open a ZODB store and get the root dictionary. If the root is empty it puts a few things into it. It then uses setattr to set some attributes of self to the things in the ZODB root. Then it just acts like any other Python class. Is that normal? It sounds a bit strange: When the class (more precisely, its singleton instance) opens a ZODB store (assuming it is the only one), then this instance is apparently not persistent (from what ZODB store should it come). But when it is not persistent, _v_ attributes are just normal attributes. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Disappearing volatile attributes in ZODB 3.6
In trying to debug some code last night and today, I ran across what looks to be a problem with ZODB 3.6. I'm not 100% sure, and I'm also not running the SVN code (see below). I'm on Mac OS X 10.4.8 with Python 2.4.3 from the Darwin Ports collection. Briefly, I have a class that puts a volatile _v_xxx attribute on instances via a method called from its __init__ method. I make 4 instances of this class, and immediately assert(hasattr(instance, '_v_xxx')) on return. All these assertions succeed. A little later in my program though, another set of asserts shows that two of the four volatile attributes are gone. So of course I went into the debugger to try to see where/when the attributes were disappearing. But in one long trip through the code, the attribute I was looking for was still present. Then I discovered that if I simply moved a copy of the assert statements up two lines in my code, the attributes do not disappear! E.g., with this: # assert(hasattr(instance, '_v_xxx')) func1(x) func2(x) assert(hasattr(instance, '_v_xxx')) the assertion fails. But with this assert(hasattr(instance, '_v_xxx')) func1(x) func2(x) assert(hasattr(instance, '_v_xxx')) everything runs just fine. I.e., the presence or absence of a single assert(hasattr(...)) call causes a volatile attribute to remain or to disappear. That seemed pretty odd. I hesitate to send mail here, having learned the hard way that to resist the idea that the system is probably at fault. But I've spent probably 5 hours trying to find and analyze this - and I'm wondering if it may be a known issue. I should add that my class is a subclass of Persistent, but that I remove the ZODB file storage file before each test run, so the new instances are not being brought out of a persisted state. Seeing as I'm running the stock 3.6 ZODB, I went looking for the latest ZODB, and grabbed svn://svn.zope.org/repos/main/ZODB/trunk. The svn log includes: r66125 | tseaver | 2006-03-22 16:43:22 +0100 (Wed, 22 Mar 2006) | 3 lines PersistentMapping was inadvertently pickling volatile attributes (http://www.zope.org/Collectors/Zope/2052). Which sounded promising, especially seeing as my class does contain a PersistentMapping, and this fix is not in the 3.6 tarball. So I fixed that by hand in my local 3.6 version and made sure a new created mapping.pyc was created in the right place, etc. But after making this change, the problem continues. So I went to install the latest ZODB. I found 3.7.0b3 at http://cheeseshop.python.org/pypi/ZODB3/3.7.0b3 and I've made 3 attempts to install this and none of them worked: - using python setup.py install, which gets me raise TypeError, dist must be a Distribution instance, and there's not much help for this to be found via google. - using easy_install, but I couldn't figure out what argument to give it (I tried all of ZODB, ZODB3.7, ZODB3-3.7.0b3, ZODB3-3.7.0b3.xml) - using zc.buildout, which I installed via easy_install but when I tried reading how to use it at http://www.python.org/pypi/zc.buildout I ended up feeling a little overwhelmed. I also grabbed svn://svn.zope.org/repos/main/ZODB/trunk but doing the normal python setup.py install in their gets me the raise TypeError, dist must be a Distribution instance, as above. As a result I haven't yet managed to install or test under the latest ZODB3-3.7.0b3 So I have 2 questions: - Does the error I'm seeing ring any bells with ZODB developers? - Can someone tell me what I'm doing wrong in trying to install the latest ZODB? If no-one has any suggestions, I'll try to make a small example where it goes wrong. Apologies for not including real code above, I'm seeing this in the middle of something much bigger and it's not easy to extract anything digestible. Thanks for any help / suggestions, Terry ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Disappearing volatile attributes in ZODB 3.6
Terry Jones wrote at 2007-1-12 12:53 +0100: ... Briefly, I have a class that puts a volatile _v_xxx attribute on instances via a method called from its __init__ method. I make 4 instances of this class, and immediately assert(hasattr(instance, '_v_xxx')) on return. All these assertions succeed. A little later in my program though, another set of asserts shows that two of the four volatile attributes are gone. The volatile attributes of an object disappear as soon as the the object is flushed from the ZODB cache. In principle, this can happen at any time. However, the current ZODB implementation is quite conservative in this respect. It flushes objects only at transaction or savepoint boundaries and when explicitly told by the application. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Disappearing volatile attributes in ZODB 3.6
Hi Jim and Dieter Thanks for the helpful answers - I'm sure you guys are right. I did understand about _v_* attributes being volatile, but I had no idea my class instances were being periodically stored and restored - I suppose only in the case that they are reachable from whatever has changed during the transaction. I'll re-think the need for those volatile attributes. --- While I (hopefully) still have your attention, I have a question about overall code design when using ZODB. I have a top-level class (call it ABC) that I only ever want one instance of. In __init__ of that class, it uses a couple of _v_ attributes to open a ZODB store and get the root dictionary. If the root is empty it puts a few things into it. It then uses setattr to set some attributes of self to the things in the ZODB root. Then it just acts like any other Python class. Is that normal? By that I mean that you just have a single class that knows how to construct an instance from scratch, or re-populate one from a persisted root? The alternative would seem to be some kind of helper class, and not using the volatile attributes in ABC.__init__ to deal with pulling in the persisted attributes. Anyway, it works just fine, though it feels a little convoluted at times. My concern is the following: The root contains various data structures and variables that hold class instances, which contain others, etc etc. Many of the class instances that are stored are created by classes that need to be passed an instance of ABC to their __init__ and which store that instance in their own self.abc for later use. I.e., I have a top-level single ABC instance in a class that subclasses Persistent, and some methods in that class pass self to the __init__ of other classes whose instances then store the ABC reference as self.abc As a result, when I re-start my application and say mainABC = ABC() to get an instance of the top-level class, I can follow the data structures it holds and get to a previously stored (in ZODB) instance of ABC (from the last run). What I've seen is that these two ABC instances can be different (this can be seen just by printing them and looking at their addresses) and when a class instance that got pulled out of ZODB uses its persisted self.abc instance, this difference becomes apparent - it is old, it doesn't have anything newly added to the top-level ABC instance. I hope that's making some sense. I believe I can solve this by arranging for the new instance to walk its relevant stored data in ABC.__init__, updating any stored self.abc references to the current (just created) instance. I wrote code to do this (putting the reference into a _v_abc attribute in the stored class instances, which was my original error). But, maybe my setup is just plain stupid due to lack of experience in using ZODB? This does seem convoluted. I could also store the single top-level ABC instance into a global variable (ugh?) - then stored class instances could just go through that global variable to get to what they need. Or maybe there's another way to do what I'm attempting. If I've managed to make myself clear, does my approach sound reasonable? If I'm not being clear, it's easy for me to whip up a small example of how I've done things. I'd really appreciate any comments. Thanks again for clearing up the _v_ question. Terry ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
