RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
Tim Peters wrote at 2004-2-24 17:30 -0500: > ... >I think I'd like to raise an exception when a connection is closed but >changes from that connection are pending in the current transaction. +1 -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
Thanks for the hints. It's an OpenBSD3.4 system, shipped with gcc 2.95, that's why I have that old gdb. I recompiled python according to the intructions (Step 2) found here: http://zope.org/Members/ensane/HowTo.2003-09-24.2929 I also bumped up the stack limit from 4M to 8M for the zope user (I have no idea if this has any effect to the problem): $ ulimit -a ... stack(kbytes)8192 Now I'm praying ;-) There have been no core dumps since the recompilation, but this issue somehow relates to the load, so I'm not sure if this helped. Sandor ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
[Dieter Maurer] > This means that the current transaction references a connection > that is meanwhile closed. > > Some earlier request forgot to commit or abort the transaction; > therefore, the transaction contains still references to the former > connection. > > I had to fix product installation code because Zope forgot an > abort when the installation of a product did not succeed. > Apparently, this is not your problem. When a connection is closed while modifications are pending for that connection, what *should* happen? Jeremy & I were just talking about that, and agree the current "benign neglect" sucks -- a subsequent attempt to commit the transaction can't succeed, and will fail in one of these obscure ways. It certainly wouldn't be right to commit the current transaction "by magic" when a connection is closed. It doesn't seem right to abort it by magic either, since it could, e.g., be that the connection being closed has no changes pending, but that other connections do have changes pending in the transaction, and silently aborting them could be a real surprise. I think I'd like to raise an exception when a connection is closed but changes from that connection are pending in the current transaction. Would anyone object to that? I doubt it was ever *intended* that transactions could be left in unusable states (or, if it was intended, the design has proven flawed in this respect). ZODB 4 kept track of enough stuff so that it was easy to do this there; ZODB 3 would require more work. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
>> #0 0x6257255 in select_select (self=0x0, args=0x3d641f7c) >>at >> /usr/local/src/python_release23-maint/Modules/selectmodule.c:171 171 >> } (gdb) bt #0 0x6257255 in select_select (self=0x0, args=0x3d641f7c) >>at >> /usr/local/src/python_release23-maint/Modules/selectmodule.c:171 [Dieter Maurer] > It is almost surely not this thread that got the SIGSEGV (unless > it is a C runtime stack overflow, quite common on BSD systems). I've got serious wonders about that: 1. The full stack trace had more than 350 frames, which is excessive even for Zope <0.9 wink>. 2. Line 171 of selectmodule.c in Python branch release23-maint is the closing curly brace of internal function set2list(). When a debugger dies pointing to a function's closing curly, it can mean that the generated return-from-function C code is blowing up due to stack corruption. But not enough info to say for sure ... ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
Sandor Palfy wrote at 2004-2-24 09:48 -0600: >I followed Jeremy's instructions to get the trace from python.core. >Here's the script: > ... >$ gdb /zope/python23/bin/python ./python.core GNU gdb 4.16.1 Almost surely, your "gdb" is far too old to analyse the core of a multi-threaded program (such as Zope). The indicator for this is that "gdb" reports just the main thread (which usually does not have any problem). You need a gdb version 5.x (with "x" as high as possible). >Copyright 1996 Free Software Foundation, Inc. >GDB is free software, covered by the GNU General Public License, and you >are Please do not send a complete script the next time! Remove unintesting parts. E.g. we are not interested which shared object were loaded and we are only interested in the first 20 to 40 stack frames and only of the thread that got the signal. >#0 0x6257255 in select_select (self=0x0, args=0x3d641f7c) >at /usr/local/src/python_release23-maint/Modules/selectmodule.c:171 >171 } >(gdb) bt >#0 0x6257255 in select_select (self=0x0, args=0x3d641f7c) >at /usr/local/src/python_release23-maint/Modules/selectmodule.c:171 It is almost surely not this thread that got the SIGSEGV (unless it is a C runtime stack overflow, quite common on BSD systems). -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
Sandor Palfy wrote at 2004-2-24 09:11 -0600: > ... >-- >2004-02-23T19:34:42 ERROR(200) ZODB A storage error occured during >object abort. This shouldn't happen. >Traceback (most recent call last): > File "/zope/zopestable/lib/python/ZODB/Transaction.py", line 401, in >_commit_error >j.tpc_abort(self) # This should never fail > File "/zope/zopestable/lib/python/ZODB/Connection.py", line 675, in >tpc_abort >self._storage.tpc_abort(transaction) >AttributeError: 'NoneType' object has no attribute 'tpc_abort' This means that the current transaction references a connection that is meanwhile closed. Some earlier request forgot to commit or abort the transaction; therefore, the transaction contains still references to the former connection. I had to fix product installation code because Zope forgot an abort when the installation of a product did not succeed. Apparently, this is not your problem. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Zope 2.7 and objects turning into None
This is what I found in the event.log today:
There was a python.core around 19:33.
2004-02-23T19:33:34 INFO(0) ZServer HTTP server started at Mon Feb 23
19:33:34 2004
Hostname: localhost
Port: 8001
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 ClientStorage (pid=21613) created
RW/normal for storage: 'elms'
--
2004-02-23T19:33:35 INFO(0) ZEC:None ClientCache: storage='elms',
size=2000; file[0]=None
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 Testing connection
--
2004-02-23T19:33:35 INFO(0) zrpc-conn:127.0.0.1: received handshake
'Z201'
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 Server authentication protocol
None
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 Connected to storage:
('localhost', )
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 Verifying cache
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 Waiting for cache verification to
finish
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 Waiting for cache verification to
finish
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 endVerify finishing
--
2004-02-23T19:33:35 INFO(0) ZCS:21613 endVerify finished
--
2004-02-23T19:33:36 PROBLEM(100) Zope No such file or directory:
/zope/instances/elms/Products/RFFolder/help
--
2004-02-23T19:33:36 PROBLEM(100) Zope No such file or directory:
/zope/instances/elms/Products/RemoteUserFolder/help
--
2004-02-23T19:33:37 INFO(0) Zope Ready to handle requests
--
2004-02-23T19:34:04 INFO(0) ZServer HTTP server started at Mon Feb 23
19:34:04 2004
Hostname: localhost
Port: 8001
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 ClientStorage (pid=4368) created
RW/normal for storage: 'elms'
--
2004-02-23T19:34:05 INFO(0) ZEC:None ClientCache: storage='elms',
size=2000; file[0]=None
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 Testing connection
--
2004-02-23T19:34:05 INFO(0) zrpc-conn:127.0.0.1: received handshake
'Z201'
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 Server authentication protocol None
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 Connected to storage: ('localhost',
)
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 Verifying cache
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 Waiting for cache verification to
finish
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 Waiting for cache verification to
finish
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 endVerify finishing
--
2004-02-23T19:34:05 INFO(0) ZCS:4368 endVerify finished
--
2004-02-23T19:34:06 PROBLEM(100) Zope No such file or directory:
/zope/instances/elms/Products/RFFolder/help
--
2004-02-23T19:34:06 PROBLEM(100) Zope No such file or directory:
/zope/instances/elms/Products/RemoteUserFolder/help
--
2004-02-23T19:34:07 INFO(0) Zope Ready to handle requests
--
2004-02-23T19:34:39 INFO(0) ZODB conflict error at
/VirtualHostBase/https/elms.vanth.org:443/Instances/a/VirtualHostRoot/pa
gesFolder/cwmgrFolder/getMyDeliverer (1 conflicts since startup at
2004-02-23T19:34:07)
--
2004-02-23T19:34:42 ERROR(200) ZODB A storage error occured during
object abort. This shouldn't happen.
Traceback (most recent call last):
File "/zope/zopestable/lib/python/ZODB/Transaction.py", line 401, in
_commit_error
j.tpc_abort(self) # This should never fail
File "/zope/zopestable/lib/python/ZODB/Connection.py", line 675, in
tpc_abort
self._storage.tpc_abort(transaction)
AttributeError: 'NoneType' object has no attribute 'tpc_abort'
--
2004-02-23T19:34:42 ERROR(200) eLMS Site
===User=sliferdw_vu;
URL=https://elms.vanth.org/CourseWare/ncw_rosellr1_vuMuscle_Force/Delive
ries/sliferdw_vu_BME101_ncw_rosellr1_vuMuscle_Force/renderQTI;
form={'hi': '0', 'eLMSaction': 'render'};
Traceback (most recent call last):
File "/zope/zopestable/lib/python/ZPublisher/Publish.py", line 104, in
publish
if transactions_manager: transactions_manager.commit()
File "/zope/zopestable/lib/python/Zope/App/startup.py", line 221, in
commit
get_transaction().commit()
File "/zope/zopestable/lib/python/ZODB/Transaction.py", line 233, in
commit
ncommitted += self._commit_objects(objects)
File "/zope/zopestable/lib/python/ZODB/Transaction.py", line 348, in
_commit_objects
jar.commit(o, self)
File "/zope/zopestable/lib/python/ZODB/Connection.py", line 370, in
commit
dbstore=self._storage.store
AttributeError: 'NoneType' object has no attribute 'store'
--
2004-02-24T06:23:12 INFO(0) Z2 Caught signal SIGTERM
--
2004-02-24T06:23:12 INFO(0) Z2 Shutting down fast
--
2004-02-24T06:23:12 INFO(0) ZServer closing HTTP to new connections
--
2004-02-24T06:23:13 INFO(0) Zope Shutting down with exit code 0
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope 2.7 and objects turning into None
> >This is a guess, but perhaps the Catalog is masking some other low-level > >error and returning None instead? I'll see if that seems possible from > >the code. > > Some time ago, I found out that > "OFS.Traversable.Traversable.unrestrictedTraverse" > catched "ConflictError" and returned "default" if a "default" has been > specified. > > I filed a bug report but I am not sure whether this was fixed > (it is fixed in our Zope version, of course). It was fixed by Andreas at the end of last September. Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:[EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope 2.7 and objects turning into None
On Saturday 21 February 2004 06:04, Casey Duncan wrote: > This is indeed possible. getObject swallows all errors and returns None > when one occurs. This would make it return None on busy systems if the > traversal during getObject raised a read ConflictError. This would indeed result in the behaviour I was seeing. > I consider this a bug and I will look into what the *real* desired > semantics are. I've been bitten myself by this in applications which > naively assumed getObject would always return a Zope object. You say "naively", I say "rightly". I could be wrong though :) > For Zope 2.8 it might be reasonable to consider also making the case > where accessing objects that have been removed without uncataloging is > also an exception. I think this is a good policy. Richard pgp0.pgp Description: signature ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope 2.7 and objects turning into None
On Fri, 20 Feb 2004 22:18:33 +0100 Dieter Maurer <[EMAIL PROTECTED]> wrote: > Casey Duncan wrote at 2004-2-20 10:53 -0500: > > ... > >This is a guess, but perhaps the Catalog is masking some other > >low-level error and returning None instead? I'll see if that seems > >possible from the code. > > Some time ago, I found out that > "OFS.Traversable.Traversable.unrestrictedTraverse" > catched "ConflictError" and returned "default" if a "default" has been > specified. > > I filed a bug report but I am not sure whether this was fixed > (it is fixed in our Zope version, of course). I think so, I was just checking this code the other day working on similar ConflictError swallowing happening in the Catalog. -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope 2.7 and objects turning into None
Casey Duncan wrote at 2004-2-20 10:53 -0500: > ... >This is a guess, but perhaps the Catalog is masking some other low-level >error and returning None instead? I'll see if that seems possible from >the code. Some time ago, I found out that "OFS.Traversable.Traversable.unrestrictedTraverse" catched "ConflictError" and returned "default" if a "default" has been specified. I filed a bug report but I am not sure whether this was fixed (it is fixed in our Zope version, of course). -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope 2.7 and objects turning into None
On Fri, 20 Feb 2004 10:53:30 -0500 Casey Duncan <[EMAIL PROTECTED]> wrote: > On Fri, 20 Feb 2004 09:47:47 +1100 > Richard Jones <[EMAIL PROTECTED]> wrote: > > > On Friday 20 February 2004 05:52, Dieter Maurer wrote: > > > We have a report for Zope 2.7 about an object magically turning > > > into "None" (1 or 2 weeks ago). > > > > Sorry, I haven't been following this thread, but I will note that > > I've seen (repeatably) in my test Zope environment this behaviour. > > > > This is probably unrelated though... > > > > For me, the situation comes about when I am loading up my fresh (ie. > > clean ZODB) test Zope with a thousand users. While the load script > > is running (it runs on the command-line and goes TTW using urllib to > > register the users) if I access a page which summarises all users, I > > *sometimes* get None for some of the users. The page uses a Catalog > > to find basic user info, and sometimes uses getObject() to find the > > real user object. The getObject() appears to return None sometimes. > > This is a guess, but perhaps the Catalog is masking some other > low-level error and returning None instead? I'll see if that seems > possible from the code. This is indeed possible. getObject swallows all errors and returns None when one occurs. This would make it return None on busy systems if the traversal during getObject raised a read ConflictError. I consider this a bug and I will look into what the *real* desired semantics are. I've been bitten myself by this in applications which naively assumed getObject would always return a Zope object. It might be sane to return None in the case where a cataloged object has been deleted and not uncataloged. For Zope 2.7 at least we should probably preserve these semantics. I propose however that if an unusual exception (ConflictError, MemoryError, SystemExit ;^) occurs, it should be propagated to the client, since it indicates a true exceptional condition. ConflictError propagation especially would help not miss results since it would force a retry of the transaction. This is crucial to applications which rely on catalog results to access objects which require action (I'm thinking specifically of the reindexObjectSecurity method in CMFCatalogAware). For Zope 2.8 it might be reasonable to consider also making the case where accessing objects that have been removed without uncataloging is also an exception. What do people think? -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope 2.7 and objects turning into None
On Fri, 20 Feb 2004 09:47:47 +1100 Richard Jones <[EMAIL PROTECTED]> wrote: > On Friday 20 February 2004 05:52, Dieter Maurer wrote: > > We have a report for Zope 2.7 about an object magically turning > > into "None" (1 or 2 weeks ago). > > Sorry, I haven't been following this thread, but I will note that I've > seen (repeatably) in my test Zope environment this behaviour. > > This is probably unrelated though... > > For me, the situation comes about when I am loading up my fresh (ie. > clean ZODB) test Zope with a thousand users. While the load script is > running (it runs on the command-line and goes TTW using urllib to > register the users) if I access a page which summarises all users, I > *sometimes* get None for some of the users. The page uses a Catalog to > find basic user info, and sometimes uses getObject() to find the real > user object. The getObject() appears to return None sometimes. This is a guess, but perhaps the Catalog is masking some other low-level error and returning None instead? I'll see if that seems possible from the code. -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
