Re: [Zope-dev] Zope Head (2.8) breaks refresh
On Mon, 2003-12-15 at 07:34, Dieter Maurer wrote: I don't think it makes a lot of sense to put an __del__ method on a Persistent object. I just read in cPersistence.c:ghostify: /* We remove the reference to the just ghosted object that the ring * holds. Note that the dictionary of oids-objects has an uncounted * reference, so if the ring's reference was the only one, this frees * the ghost object. Note further that the object's dealloc knows to * inform the dictionary that it is going away. */ This means: *all* persistent objects have a special dealloc function. Hope, this dealloc function plays well with the cyclic garbage collector. All Python objects have a dealloc function -- whatever is in the tp_dealloc slot. The tp_dealloc function at the C level is completely different from an __del__ method at the Python level. Jeremy ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
On Sun, 2003-12-14 at 16:53, Chris McDonough wrote: We still use ZODB 3.1 and at least there, the cache verification protocol seems quite stupid. We will soon switch to ZODB 3.2 and when cache validation still needs minutes, I will need to look into this... Apparently, the ZEO in 2.7 and the HEAD is better about doing as little work as possible for cache verification at startup than older versions were. Specifically, the server saves the list of invalidated objects for the last N transactions and the client keeps track of the last transaction id it received an invalidation for. One restart, if the last txn the client saw was within N of the current transaction, the server just sends the list of invalidated objects. For a large cache, this is much cheaper than having the client send the oid and serialno of every object in the cache to the server. N is configurable. I think the default value is 100. Jeremy ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
On Fri, 2003-12-12 at 23:43, Stuart Bishop wrote: I've just clarified the issue in the collector. We have since found that manual product refresh is working, and it is just the automatic product refresh that gives this error. I don't mean to sound glib (really), but refresh has always caused more problems for me than it has solved, so when I see bug reports for refresh I'm always reminded of Doctor, doctor, it hurts when I do this... With ZEO, Zope typically restarts in under a second. I just do that and don't bother with refresh. Is there any commanding reason to use refresh other than startup/teardown time? - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
On Sat, 2003-12-13 at 22:10, Stuart Bishop wrote: Firstly, I've *never* had problems with refresh (as a user of) until Zope 2.7 (I only use it on my own products, so I must naturally code in a refresh friendly manner). I'm used to developing with autorefresh switched on for my products and it just *works*. If I make a change and don't see the expected behavior, I switch to the product refresh page to see where the syntax error is that caused the refresh to fail. I must code in a particularly refresh-unfriendly manner because often the globals I use in my code turn into Nones after a refresh, and it leaves me in a place where I need to try to figure out whether it's my problem or refresh's problem, and the first step in doing that is restarting. So I just cut out the middleman, because it's not much of a bother. I do agree that having autorefresh would be nice, though. Debugging it is just not problem I'm keen on tackling. It's an amazing piece of work by Shane, but it seems to have some limits that I don't quite understand. Either that or it works just fine but I just haven't used it in a while. I've occasionally also used manual refresh to reload products on production servers without nuking sessions, but apart from that it is startup/teardown time. This is a problem though because doc/INSTALL.txt tells us to: ./configure make instance Until following the steps in doc/INSTALL.txt fires up a single ZEO server bound to the loopback address and a single ZEO client (with authentication on), the vast majority of developer and production installations will be ZEO-less. This also makes ZEO-less installations the best tested and most stable environment, or at least people will assume so. Zope still starts in under a second with a very small index and in under 10 seconds with a large one (at least on my main development machines), so even this isn't so terrible. With the zopectl shell, it's even just a matter of arrowing up in a terminal window and choosing restart. But autorefresh would indeed kick the snot out of this any day if I felt like I could rely on it unconditionally or if the conditions under which it caused me problems were easily predictable. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/12/2003, at 5:50 AM, Dieter Maurer wrote: Stuart Bishop wrote at 2003-12-10 11:24 +1100: On 08/12/2003, at 12:15 AM, Dieter Maurer wrote: Playing with Zope Head (as of 2003-12-04) revealed problems with refresh. This is not just a 2.8 issue - the behavior is in the 2.7 betas (at least up to beta 2 - havn't tested autorefresh with beta 3 yet) as well: http://collector.zope.org/Zope/1010 Manually refreshing works fine, but automatic refreshes cause the ValueErrors. Then, try my patch... The patch is incompatible with the 2.7 branch - self._reader and ConnectionObjectReader don't seem to exist in 2.7 at all, so there is nothing to shuffle around. - -- Stuart Bishop [EMAIL PROTECTED] http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/2OoZAfqZj7rGN0oRAnWhAJoDezgxOKwswrhZEvAWeYrXyLNZ2ACcCmqG TVUNPb7Ts0pndeNvEWgtHP0= =1/Bz -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
Stuart Bishop wrote at 2003-12-10 11:24 +1100: On 08/12/2003, at 12:15 AM, Dieter Maurer wrote: Playing with Zope Head (as of 2003-12-04) revealed problems with refresh. This is not just a 2.8 issue - the behavior is in the 2.7 betas (at least up to beta 2 - havn't tested autorefresh with beta 3 yet) as well: http://collector.zope.org/Zope/1010 Manually refreshing works fine, but automatic refreshes cause the ValueErrors. Then, try my patch... -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/12/2003, at 12:15 AM, Dieter Maurer wrote: Playing with Zope Head (as of 2003-12-04) revealed problems with refresh. This is not just a 2.8 issue - the behavior is in the 2.7 betas (at least up to beta 2 - havn't tested autorefresh with beta 3 yet) as well: http://collector.zope.org/Zope/1010 Manually refreshing works fine, but automatic refreshes cause the ValueErrors. - -- Stuart Bishop [EMAIL PROTECTED] http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/1mfNAfqZj7rGN0oRAj9PAJ4+EUiHT9uI48ne5fRbO7+Tfa2fSQCdETcl ZF0IU3quFDv6QGlnzRdxSz4= =HIIJ -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
Jeremy Hylton wrote at 2003-12-7 23:01 -0500: On Sun, 2003-12-07 at 08:15, Dieter Maurer wrote: ATT: replacing the cache without clearing it can lead to huge memory leaks (everything in the old cache is leaked!). Without commenting on the rest of the bug report, I should mention that caches and persistent objects all participate in cyclic GC. They will not leak the way pre-2.8 caches did. Has the do not free when a cycle contains an object with destructor restriction been removed from the cyclic GC? It is not unlikely that the cache references an object with a __del__. This may keep the cache and all its content. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
On Mon, 2003-12-08 at 13:17, Dieter Maurer wrote: Jeremy Hylton wrote at 2003-12-7 23:01 -0500: On Sun, 2003-12-07 at 08:15, Dieter Maurer wrote: ATT: replacing the cache without clearing it can lead to huge memory leaks (everything in the old cache is leaked!). Without commenting on the rest of the bug report, I should mention that caches and persistent objects all participate in cyclic GC. They will not leak the way pre-2.8 caches did. Has the do not free when a cycle contains an object with destructor restriction been removed from the cyclic GC? It is not unlikely that the cache references an object with a __del__. This may keep the cache and all its content. I don't think it makes a lot of sense to put an __del__ method on a Persistent object. When would you expect it to be called? Every time the object is turned into a ghost? When the object is removed by pack? I can imagine some people would try to migrate non-persistent code to ZODB and have to deal with __del__ then. If I were doing that, I'd make sure to check gc.garbage when I tested and fix any code that caused leaks. Jeremy ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Head (2.8) breaks refresh
Playing with Zope Head (as of 2003-12-04) revealed problems with
refresh.
I see varying behaviour refreshing CMFCore:
* Refresh is not effective (objects continue the have the old
behavior)
* After a while, commit reports an exception
ValueError: Cache values may only be in one cache.
http://localhost:10080/Portal/portal_types/Document/manage_editProperties
Traceback (innermost last):
Module ZPublisher.Publish, line 104, in publish
Module Zope.App.startup, line 223, in commit
Module ZODB.Transaction, line 254, in commit
Module ZODB.Transaction, line 233, in commit
Module ZODB.Transaction, line 352, in _commit_objects
Module ZODB.Connection, line 286, in commit
ValueError: Cache values may only be in one cache.
The following transscript shows strange reproducible behaviour
which is almost surely related to the above:
linux: bin/zopectl debug
Starting debugger (the name app is bound to the top-level Zope object)
pt=getattr(app,'standard_template.pt')
pr=app.unrestrictedTraverse('Control_Panel/Products/PageTemplates/manage_performRefresh')
pt.getId()
'standard_template.pt'
pr()## refresh
pt.getId() ## apparently not deactivated
'standard_template.pt'
pt._p_deactivate()
pt.getId() ## obviously a wrong result
'Zope'
pt.aq_base.getId() ## may be explainable as pt is not refetched
Traceback (most recent call last):
File stdin, line 1, in ?
File /home/dieter/ZH/ZopeHead/lib/python/ZODB/Connection.py, line 424, in setstate
self._reader.setGhostState(obj, p)
File /home/dieter/ZH/ZopeHead/lib/python/ZODB/serialize.py, line 203, in
setGhostState
obj.__setstate__(state)
File
/home/dieter/ZH/ZopeHead/lib/python/Products/PageTemplates/ZopePageTemplate.py, line
301, in __setstate__
ZopePageTemplate.inheritedAttribute('__setstate__')(self, state)
AttributeError: 'NoneType' object has no attribute 'inheritedAttribute'
pt=getattr(app,'standard_template.pt') ## refetch
pt.getId() ## still wrong
'Zope'
Looks as if refresh did not flush the ZODB cache, which still
delivers objects referencing the old classes.
I now know why the above code does not work as I expected:
The new refresh no longer uses minimize to flush the ZODB
caches but updates the code_timestamp.
This causes the cache to be replaced by a new one when
the connection is opened for the next time.
ATT: replacing the cache without clearing it can lead to huge
memory leaks (everything in the old cache is leaked!).
I can not yet explain the original observations...
--
Dieter
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks refresh
On Sun, 2003-12-07 at 08:15, Dieter Maurer wrote: ATT: replacing the cache without clearing it can lead to huge memory leaks (everything in the old cache is leaked!). Without commenting on the rest of the bug report, I should mention that caches and persistent objects all participate in cyclic GC. They will not leak the way pre-2.8 caches did. Jeremy ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
