Re: [ZODB-Dev] database conflict fail
Le 21/03/2012 23:54, Claudiu Saftoiu a écrit :
Hello ZODB List,
snip
key = ...
print Acquiring %s lock... % (key,)
KEY_LOCKS[key].acquire()
print %s lock acquired. % (key,)
def after_commit_hook(success):
KEY_LOCKS[key].release()
print (after %s commit): Released %s lock % (('failed',
'successful')[success], key)
transaction.get().addAfterCommitHook(after_commit_hook)
Hum, if I understood correctly, lock acquiring is done before any change
are made. So the only thing I would have done here is to have a
transaction.begin() after lock.acquire. So you know the other
transaction is committed and you won't have any conflict in you new
transaction.
Does it helps ?
Alex
--
Alexandre Garel
06 78 33 15 37
___
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] database conflict fail
Le jeudi 22 mars 2012 18:23:47, Claudiu Saftoiu a écrit : Ahh, now that looks promising. Are there any particularly good places to get documentation on that sort of thing? All I see when I google are mailing list archives. See ZODB/ConflictResolution.txt . Disclaimer: I didn't read it. I read the code - but I expect the text file to be easier to assimilate :) . BTrees/Length.py:Length class implements a simple _p_resolveConflict (yours will be even simpler). Also: is there any easier way to see which objects had a conflict when a ConflictError is raised? Currently I am doing a binary-search via commenting code, but I figure there must be a better way... There should be some details in the exceptoin itself. Like, oids, currently commited TID and the TID transaction started with. root()._p_jar[the_oid] will get you the object. Then the hard part starts: guess where in the object tree that object is. If you have varied classes, and significant data on your persistent instances, it will be easy. Ragards, -- Vincent Pelletier ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] database conflict fail
On Thu, Mar 22, 2012 at 2:45 PM, Vincent Pelletier plr.vinc...@gmail.comwrote:
Le jeudi 22 mars 2012 18:23:47, Claudiu Saftoiu a écrit :
Ahh, now that looks promising. Are there any particularly good places to
get documentation on that sort of thing? All I see when I google are
mailing list archives.
See ZODB/ConflictResolution.txt . Disclaimer: I didn't read it. I read the
code - but I expect the text file to be easier to assimilate :) .
BTrees/Length.py:Length class implements a simple _p_resolveConflict (yours
will be even simpler).
Thanks, I will take a look.
There should be some details in the exceptoin itself. Like, oids, currently
commited TID and the TID transaction started with. root()._p_jar[the_oid]
will
get you the object. Then the hard part starts: guess where in the object
tree
that object is. If you have varied classes, and significant data on your
persistent instances, it will be easy.
Ok, I just got a ConflictError:
ConflictError: database conflict error (oid 0x139c35, class
BTrees.OOBTree.OOBucket, serial this txn started with 0x03954ed053c0ff88
2012-03-22 16:48:19.629820, serial currently committed 0x03954f996d61c944
2012-03-22 20:09:25.636401)
in my paster shell I do:
In [14]: root._p_jar[0x139c35]
However, this causes:
In [14]: root._p_jar[0x139c35]
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (85, 0))
---
TypeError Traceback (most recent call last)
/home/tsa/sports/ipython console in module()
/home/tsa/env/lib/python2.6/site-packages/ZODB/Connection.pyc in get(self,
oid)
246 return obj
247
-- 248 p, serial = self._storage.load(oid, '')
249 obj = self._reader.getGhost(p)
250
/home/tsa/env/lib/python2.6/site-packages/ZEO/ClientStorage.pyc in
load(self, oid, version)
813 self._lock.acquire()# for atomic processing of
invalidations
814 try:
-- 815 t = self._cache.load(oid)
816 if t:
817 return t
/home/tsa/env/lib/python2.6/site-packages/ZEO/cache.pyc in call(*args, **kw)
141 inst._lock.acquire()
142 try:
-- 143 return self.func(inst, *args, **kw)
144 finally:
145 inst._lock.release()
/home/tsa/env/lib/python2.6/site-packages/ZEO/cache.pyc in load(self, oid)
487 @locked
488 def load(self, oid):
-- 489 ofs = self.current.get(oid)
490 if ofs is None:
491 self._trace(0x20, oid)
/home/tsa/env/lib/python2.6/site-packages/ZODB/fsIndex.pyc in get(self,
key, default)
123
124 def get(self, key, default=None):
-- 125 tree = self._data.get(key[:6], default)
126 if tree is default:
127 return default
TypeError: 'int' object is unsubscriptable
What am I doing wrong?
Thanks again,
- Claudiu
___
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] database conflict fail
Le jeudi 22 mars 2012 21:13:34, Claudiu Saftoiu a écrit :
In [14]: root._p_jar[0x139c35]
Actually, you want to write:
root._p_jar['\x00\x00\x00\x00\x00\x13\x9c\x35']
ie, OIDs are 8-byte binary strings.
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (85, 0))
Looks like internal ipython error. Personally, I stay away of it, despite its
nice colors. It crashes and burns too easily.
/home/tsa/env/lib/python2.6/site-packages/ZODB/fsIndex.pyc in get(self,
key, default)
123
124 def get(self, key, default=None):
-- 125 tree = self._data.get(key[:6], default)
126 if tree is default:
127 return default
TypeError: 'int' object is unsubscriptable
This is the actual error, which will go away if you provide a string.
--
Vincent Pelletier
___
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] database conflict fail
On Thu, Mar 22, 2012 at 04:13:34PM -0400, Claudiu Saftoiu wrote:
On Thu, Mar 22, 2012 at 2:45 PM, Vincent Pelletier
plr.vinc...@gmail.comwrote:
Le jeudi 22 mars 2012 18:23:47, Claudiu Saftoiu a écrit :
Ahh, now that looks promising. Are there any particularly good places to
get documentation on that sort of thing? All I see when I google are
mailing list archives.
See ZODB/ConflictResolution.txt . Disclaimer: I didn't read it. I read the
code - but I expect the text file to be easier to assimilate :) .
BTrees/Length.py:Length class implements a simple _p_resolveConflict (yours
will be even simpler).
Thanks, I will take a look.
There should be some details in the exceptoin itself. Like, oids, currently
commited TID and the TID transaction started with. root()._p_jar[the_oid]
will
get you the object. Then the hard part starts: guess where in the object
tree
that object is. If you have varied classes, and significant data on your
persistent instances, it will be easy.
Ok, I just got a ConflictError:
ConflictError: database conflict error (oid 0x139c35, class
BTrees.OOBTree.OOBucket, serial this txn started with 0x03954ed053c0ff88
2012-03-22 16:48:19.629820, serial currently committed 0x03954f996d61c944
2012-03-22 20:09:25.636401)
in my paster shell I do:
In [14]: root._p_jar[0x139c35]
OIDs are 64-bit numbers internally encoded as 8-byte-long binary strings.
Use
!from ZODB.utils import p64
p root._p_jar.get(p64(0x139c35))
or http://pypi.python.org/pypi/zodbbrowser
However, this causes:
In [14]: root._p_jar[0x139c35]
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (85, 0))
Where did the EOF come from?
(This is why I don't trust ipython.)
---
TypeError Traceback (most recent call last)
/home/tsa/sports/ipython console in module()
/home/tsa/env/lib/python2.6/site-packages/ZODB/Connection.pyc in get(self,
oid)
246 return obj
247
-- 248 p, serial = self._storage.load(oid, '')
249 obj = self._reader.getGhost(p)
250
/home/tsa/env/lib/python2.6/site-packages/ZEO/ClientStorage.pyc in
load(self, oid, version)
813 self._lock.acquire()# for atomic processing of
invalidations
814 try:
-- 815 t = self._cache.load(oid)
816 if t:
817 return t
/home/tsa/env/lib/python2.6/site-packages/ZEO/cache.pyc in call(*args, **kw)
141 inst._lock.acquire()
142 try:
-- 143 return self.func(inst, *args, **kw)
144 finally:
145 inst._lock.release()
/home/tsa/env/lib/python2.6/site-packages/ZEO/cache.pyc in load(self, oid)
487 @locked
488 def load(self, oid):
-- 489 ofs = self.current.get(oid)
490 if ofs is None:
491 self._trace(0x20, oid)
/home/tsa/env/lib/python2.6/site-packages/ZODB/fsIndex.pyc in get(self,
key, default)
123
124 def get(self, key, default=None):
-- 125 tree = self._data.get(key[:6], default)
126 if tree is default:
127 return default
TypeError: 'int' object is unsubscriptable
This makes more sense than the EOF error above.
Marius Gedminas
--
I am monolingual and English is the ling I mono.
-- James Nicoll
signature.asc
Description: Digital signature
___
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] database conflict fail
On Thu, Mar 22, 2012 at 11:07:09PM +0200, Marius Gedminas wrote: On Thu, Mar 22, 2012 at 04:13:34PM -0400, Claudiu Saftoiu wrote: Ok, I just got a ConflictError: ConflictError: database conflict error (oid 0x139c35, class BTrees.OOBTree.OOBucket, serial this txn started with 0x03954ed053c0ff88 2012-03-22 16:48:19.629820, serial currently committed 0x03954f996d61c944 2012-03-22 20:09:25.636401) in my paster shell I do: In [14]: root._p_jar[0x139c35] OIDs are 64-bit numbers internally encoded as 8-byte-long binary strings. Use !from ZODB.utils import p64 p root._p_jar.get(p64(0x139c35)) Sorry, I shouldn't use pdb notation on the interactive Python prompt. from ZODB.utils import p64 root._p_jar.get(p64(0x139c35)) or http://pypi.python.org/pypi/zodbbrowser Marius Gedminas -- IBM motto: TEN vowels? Don't you know vowels are scrd? -- Linus Torvalds signature.asc Description: Digital signature ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] database conflict fail
Stop! Before you go down that road, application-level conflict resolution if an extremely advanced feature. It's on the same level as meta classes and custom import hooks in a Python. It's a last resort (or maybe the one after that). The first thing you need to ask yourself is why your application wants to update the same value from multiple threads. This is a hot spot. Hot spots: bad. Most applications will need to update the same value from multiple threads occasionally. Occasionally retrying is fine. If your application wants to update the same value from multiple threads often, then that's a design problem you should solve. Thanks for the advice, Jim. I was going for an all-or-nothing approach (leave all conflicts in, or try to eradicate all of them), but it does seem like some of them are quite easy to resolve, and others don't matter at all if they happen rarely. I'll take care to see if there are any other solutions, and that it's important, before going to the application-level conflict resolution. - Claudiu ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
