Re: [ZODB-Dev] Heads-up: Test failure on trunk/Python 2.4.2/OS X
Tim Peters wrote: [Dmitry Vasiliev] Oops, my bad. I couldn't imagine that misspelled text could been used in any doctests so I had omitted the testing phase because I was in some hurry. :-( And unfortunately I was far from any computers for past few days. Thanks for following up -- I was afraid you had vanished! Not so fast... ;-) Especially for newer code, the tests try to exercise error conditions as well as normal and edge cases, and it's natural in doctests to include the text of an exception message. So, ya, there's really nothing you can change outside of NEWS.txt anymore that doesn't require testing. Sure, but my reasonable assumption was that when you insert an exception message in a doctest you reread it another one time and fix typos if any. I'll be more careful in the future about such assumptions. -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] More exciting ZODB errors ;-)
Hi All,
We recently upgraded to Zope 2.8.4 and have been seeing some different
and special errors every so often (why does this always and only ever
happen to me? ;-) Nothing noticeably bad appears to result from these,
but, as always, I loose hair over them and would like ot know what's
going on :-S
Any clues?
Chris
08:55 - Traceback (most recent call last):
lib/python/transaction/_transaction.py,
line 422, in _commitResources
rm.tpc_begin(self)
File lib/python/ZODB/Connection.py, line
446, in tpc_begin
self._normal_storage.tpc_begin(transaction)
File lib/python/ZEO/ClientStorage.py,
line 914, in tpc_begin
txn._extension, tid, status)
File lib/python/ZEO/ServerStub.py, line
232, in tpc_begin
return self.rpc.call('tpc_begin', id, user, descr, ext, tid, status)
File lib/python/ZEO/zrpc/connection.py,
line 536, in call
raise inst # error raised by server
StorageTransactionError: Multiple simultaneous tpc_begin requests from one
client.
10:05 - Couldn't load state for 0x013872b7:
Traceback (most recent call last):
File lib/python/ZODB/Connection.py, line
704, in setstate
self._setstate(obj)
File lib/python/ZODB/Connection.py, line
757, in _setstate
self._load_before_or_conflict(obj)
File lib/python/ZODB/Connection.py, line
765, in _load_before_or_conflict
if not (self._mvcc and self._setstate_noncurrent(obj)):
File lib/python/ZODB/Connection.py, line
788, in _setstate_noncurrent
assert end is not None
AssertionError
10:05 - Error viewing a method on an object:
File lib/python/Products/PluginIndexes/DateIndex/DateIndex.py, line
135, in index_object
oldConvertedDate = self._unindex.get( documentId, _marker )
File lib/python/ZODB/Connection.py, line
704, in setstate
self._setstate(obj)
File lib/python/ZODB/Connection.py, line
757, in _setstate
self._load_before_or_conflict(obj)
File lib/python/ZODB/Connection.py, line
765, in _load_before_or_conflict
if not (self._mvcc and self._setstate_noncurrent(obj)):
File lib/python/ZODB/Connection.py, line
788, in _setstate_noncurrent
assert end is not None
AssertionError
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] More exciting ZODB errors ;-)
[Chris Withers]
We recently upgraded to Zope 2.8.4 and have been seeing some different
and special errors every so often (why does this always and only ever
happen to me? ;-)
I don't know, but if I were you I'd try running for a while on entirely
different hardware.
Nothing noticeably bad appears to result from these, but, as always, I
loose hair over them and would like ot know what's going on :-S
Any clues?
Chris
08:55 - Traceback (most recent call last):
lib/python/transaction/_transaction.py, line 422, in _commitResources
rm.tpc_begin(self)
File lib/python/ZODB/Connection.py, line 446, in tpc_begin
self._normal_storage.tpc_begin(transaction)
File lib/python/ZEO/ClientStorage.py, line 914, in tpc_begin
txn._extension, tid, status)
File lib/python/ZEO/ServerStub.py, line 232, in tpc_begin
return self.rpc.call('tpc_begin', id, user, descr, ext, tid, status)
File lib/python/ZEO/zrpc/connection.py, line 536, in call
raise inst # error raised by server
StorageTransactionError: Multiple simultaneous tpc_begin requests from one
client.
Found only one previous report of this, against Zope 2.6.1, some ZODB 3.0.1
beta (unclear exactly which), and Berkeley DB:
http://mail.zope.org/pipermail/zodb-dev/2003-May/005195.html
Jeremy made some suggestions a few msgs down, but there was no followup from
the OP and no resolution.
10:05 - Couldn't load state for 0x013872b7: Traceback (most recent call
last):
File lib/python/ZODB/Connection.py, line 704, in setstate
self._setstate(obj)
File lib/python/ZODB/Connection.py, line 757, in _setstate
self._load_before_or_conflict(obj)
File lib/python/ZODB/Connection.py, line 765, in
_load_before_or_conflict
if not (self._mvcc and self._setstate_noncurrent(obj)):
File lib/python/ZODB/Connection.py, line 788, in _setstate_noncurrent
assert end is not None
AssertionError
10:05 - Error viewing a method on an object:
File lib/python/Products/PluginIndexes/DateIndex/DateIndex.py, line
135, in index_object
oldConvertedDate = self._unindex.get( documentId, _marker )
File lib/python/ZODB/Connection.py, line 704, in setstate
self._setstate(obj)
File lib/python/ZODB/Connection.py, line 757, in _setstate
self._load_before_or_conflict(obj)
File lib/python/ZODB/Connection.py, line 765, in
load_before_or_conflict
if not (self._mvcc and self._setstate_noncurrent(obj)):
File lib/python/ZODB/Connection.py, line 788, in _setstate_noncurrent
assert end is not None
AssertionError
Haven't seen a report of this before. It's trying to load state for a ghost
object, but the oid has been invalidated, so (because of MVCC) it's trying
to load state that _was_ current at the time the transaction began. The
assertion is griping because the non-current state it loaded doesn't appear
to have a sane validity range (that `end` is None is claiming that the old
state is still valid, but that can't be true if the oid has in fact been
invalidated).
Smells like an internal MVCC bug to me.
Jeremy, the code at the start of _setstate() looks fishy: any idea what the
intent was?:
# There is a harmless data race with self._invalidated. A
# dict update could go on in another thread, but we don't care
# because we have to check again after the load anyway.
if (obj._p_oid in self._invalidated
and not myhasattr(obj, _p_independent)
and not self._invalidated
):
# If the object has _p_independent(), we will handle it below.
self._load_before_or_conflict(obj)
return
Specifically, how could:
obj._p_oid in self._invalidated
and:
not self._invalidated
be true at the same time? The latter says _invalidated is empty, but the
former can't be true if _invalidated is empty. Offhand, then, unless we're
_counting_ on a thread race here ;-), it looks impossible that the body of
this `if` could execute.
I don't have a reason to believe it's directly relevant to Chris's problem,
it's just that I got baffled by the first thing I looked at ;-)
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] More exciting ZODB errors ;-)
On 12/14/05, Tim Peters [EMAIL PROTECTED] wrote: Jeremy, the code at the start of _setstate() looks fishy: any idea what the intent was?: # There is a harmless data race with self._invalidated. A # dict update could go on in another thread, but we don't care # because we have to check again after the load anyway. if (obj._p_oid in self._invalidated and not myhasattr(obj, _p_independent) and not self._invalidated ): # If the object has _p_independent(), we will handle it below. self._load_before_or_conflict(obj) return I don't have any idea what the intent of not self._invalidated was. I looked back at 3.3a2, which I think was the first MVCC release. It doesn't have the and not self._invalidated line , and I can't think of why it would. The code is just an optimization; it's trying to avoid loading the current version if it knows that it needs a non-current version. I don't have a checkout of ZODB, so I can't check the svn blame. Does it say who wrote the code and what change it was related to? Jeremy Specifically, how could: obj._p_oid in self._invalidated and: not self._invalidated be true at the same time? The latter says _invalidated is empty, but the former can't be true if _invalidated is empty. Offhand, then, unless we're _counting_ on a thread race here ;-), it looks impossible that the body of this `if` could execute. I don't have a reason to believe it's directly relevant to Chris's problem, it's just that I got baffled by the first thing I looked at ;-) ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] More exciting ZODB errors ;-)
[Timmy] Jeremy, the code at the start of _setstate() looks fishy: any idea what the intent was?: # There is a harmless data race with self._invalidated. A # dict update could go on in another thread, but we don't care # because we have to check again after the load anyway. if (obj._p_oid in self._invalidated and not myhasattr(obj, _p_independent) and not self._invalidated ): # If the object has _p_independent(), we will handle it below. self._load_before_or_conflict(obj) return [Jeremy] I don't have any idea what the intent of not self._invalidated was. I looked back at 3.3a2, which I think was the first MVCC release. It doesn't have the and not self._invalidated line , and I can't think of why it would. The code is just an optimization; it's trying to avoid loading the current version if it knows that it needs a non-current version. I don't have a checkout of ZODB, so I can't check the svn blame. Does it say who wrote the code and what change it was related to? Congratulations on weaseling out of another one ;-)! Turns out this all happened after your statutory blame period, as part of a large merge to fix ZClasses: http://svn.zope.org/ZODB/branches/3.4/src/ZODB/Connection.py?rev=29891view =rev That introduced the and not self._invalidated clause. We'll fix that one way or another (it doesn't make sense), but I'm pretty sure it's irrelevant to Chris's symptom (it stops an optimization rather than creates a problem). ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] More exciting ZODB errors ;-)
On 12/14/05, Tim Peters [EMAIL PROTECTED] wrote: I don't have a checkout of ZODB, so I can't check the svn blame. Does it say who wrote the code and what change it was related to? Congratulations on weaseling out of another one ;-)! It was much harder to weasel out of them when I actually had commit privileges. Turns out this all happened after your statutory blame period, as part of a large merge to fix ZClasses: http://svn.zope.org/ZODB/branches/3.4/src/ZODB/Connection.py?rev=29891view =rev That introduced the and not self._invalidated clause. We'll fix that one way or another (it doesn't make sense), but I'm pretty sure it's irrelevant to Chris's symptom (it stops an optimization rather than creates a problem). Right. It's certainly going to hurt the performance of ZODB in a high-write situation, because it will always read the current revision of an object, even when it needs a non-current revision. Jeremy ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: [Zope] sessions in the presence of conflicts
Hi Davids. Thanks for your note. I remember your post. No, I have not used HappySession although we looked at it when we started. When we started development, sessions had just become part of the standard release Zope and we decided that in the long run we'd be better off using the release product. As I've said in other posts, we use the session mechanism fairly heavily. We have O(200) simultaneous interactive users each of whom has 40 or so session variables which maintain the dynamic state of their instance. Nearly every substantive operation touches one or more session variables. Everything works most of the time, but the error frequency is high enough for it to be of concern. I'm a bit cautious about moving to another sessioning system on a sizable production system running 24x7. Dennis, Im only curious and this is no recommendation ... and I asked this on the list maybe a year ago (and got zero reponses) Have you given HappySession a try? Has anyone? I've never had a problem with it, but then its never been under much load and is no longer actively supported (least last time I checked). I converted back and forth between Zope SESSION and HappySession (for reasons of legacy code). Their interfaces have just few differences, e.g., HappySession['this'] returns None (if 'this' is not a key) and SESSION raises an exception. That type of thing. HappySession does not read/write to zodb (if I recall it stores session data in RAM (session) cookies). I just do not know if its otherwise up to your requirements. All best, David -- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Heads-up: Test failure on trunk/Python 2.4.2/OS X
Dmitry Vasiliev wrote: Sure, but my reasonable assumption was that when you insert an exception message in a doctest you reread it another one time and fix typos if any. I'll be more careful in the future about such assumptions. Without meaning to sound condescending (which certainly is not my intention): Rule number one of problem solving: Never assume anything Rule number two: Every assumption is wrong In fact, I have learned the very, very hard way over the last 20 years or so (I've got scars to show) that this applies to *everything*, and particularly to the field of computer problem solving, regardless of whether it is code, hw/sw or user interaction. /dario -- -- --- Dario Lopez-Kästen, IT Systems Services Chalmers University of Tech. Lyrics applied to programming application design: emancipate yourself from mental slavery - redemption song, b. marley ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
