[ZODB-Dev] "Advanced ZODB for Python Programmers" article: volatile example
Hi. I think the example on volatile attributes given in "Advanced ZODB for Python Programmers" article[1] shows a bad practice. >From the article: if hasattr(self, '_v_image'): return self._v_image This should be rewritten as: try: return self._v_image except AttributeError: This solves cases where _v_ attribute gets garbage-collected between hasattr call and "return" line. A way to make this obvious is to artificially set object cache size to 0. Note that another broken pattern would be to use code like: self._v_image=expensive_calculation() return self._v_image the article advertises the right pattern (using a local variable for return): image=expensive_calculation() self._v_image=image return image I've been hitting problems with this code pattern many times in the past, and should have written about it earlier. [1] http://zodb.org/documentation/articles/ZODB2.html -- Vincent Pelletier ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] "Advanced ZODB for Python Programmers" article: volatile example
On 5/24/10 13:59 , Vincent Pelletier wrote: > Hi. > > I think the example on volatile attributes given in "Advanced ZODB for Python > Programmers" article[1] shows a bad practice. > >> From the article: > if hasattr(self, '_v_image'): > return self._v_image > > This should be rewritten as: > try: > return self._v_image > except AttributeError: or better: marker = [] value = getattr(self, "_v_image", marker) if value is not marker: return value which prevents the exception overhead. Wichert. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] "Advanced ZODB for Python Programmers" article: volatile example
Le lundi 24 mai 2010 12:02:40, Wichert Akkerman a écrit : > which prevents the exception overhead. Right, thought this is a tradeoff question: writing as exception handling is faster for "volatile is present" case, but slower for "volatile is missing" case. -- Vincent Pelletier ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ClientDisconnected with ZEORaid
Hi,
I've been testing out ZEORaid and it seems to work fine in itself, but
ZODB client connections seem to be getting into a ClientDisconnected
state and never getting out. Zope has a main ZEORaid to connect to and
several fallbacks. Most of the time when it's connection is dropped it
switch to checking it's backups. However sometimes I'll get a log
trace like this.
2010-05-24T17:35:28 WARNING ZEO.zrpc (30496) CW: error connecting to
('x.x.x.188', 8323): ECONNREFUSED
--
2010-05-24T17:35:28 INFO ZEO.ClientStorage (30496) Testing connection
2010-05-24T17:35:56 ERROR ZODB.Connection Couldn't load state for 0x36df
...
raise ClientDisconnected()
ClientDisconnected
It looks like a transaction has started while the ZEOClient is still
verifying the connection and this causes the ClientDisconnected error.
This would be fine except that it never seems to retry again.
Looking at the code my reading is that this behaviour is by design.
--- http://svn.zope.org/ZODB/trunk/src/ZEO/zrpc/client.py ---
class ConnectThread(threading.Thread):
"""Thread that tries to connect to server given one or more addresses.
The thread is passed a ConnectionManager and the manager's client as
arguments. It calls testConnection() on the client when a socket
connects; that should return 1 or 0 indicating whether this is a
preferred or a fallback connection. It may also raise an exception, in
which case the connection is abandoned.
---
Is there any good reason not to retry connecting?
---
Dylan Jay
Plone Solutions Manager. www.pretaweb.com
P +612 80819071 M +61421477460
skype - dylan_jaytwitter - djay75
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Notes on using wrapper storages for record transformation
Le dimanche 23 mai 2010 16:11:44, Jim Fulton a écrit : > I'm not sure what you mean by object level locks. I've been planning to > add something similar to object-level locks during the commit process, so > multiple transactions can be doing commit-related activity at once. That's the feature I intended to express. > Independent of that, It should be possible to check for conflicts and > deal with them on the client during the first phase of 2-phase commit. Yes, this is how it's done in NEO (and IIRC in ZEO), by making "store" return a None until answers are received from server. > Note that ZEO doesn't get the storage lock until the end of the first > phase. Ah, I missed this. Getting a bit more out of initial topic: I am currently searching for papers describing how conflict resolution is supposed to happen, and more exactly if it is safe to "chain-resolve" for a single transaction. Example, using a similar representation of action as in Atul Adya thesis: w1(x1) w2(x2) c2 c1 -> conflict for x1, "based" on x0 and to "rebase" onto x2, so transaction 1 starts resolving. In the meanwhile: w3(c3) c3 w1(x1') c1 -> conflict for transaction 1 again. This step can be repeated as many times as desired, then eventually: w1(x1") c1 -> success I could not find a doc spelling out this case yet. >From what you said on lock not being held during the first phase, I infer that it is already standard for this to happen. > As with NEO, zc.zodbdgc was motivated by multiple databases where a single > database doesn't have enough information to perform garbage collection. Thanks for pointing this package out, I didn't know it. -- Vincent Pelletier ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Conflict resolution (was Re: Notes on using wrapper storages for record transformation)
On Mon, May 24, 2010 at 10:57 AM, Vincent Pelletier wrote: > Le dimanche 23 mai 2010 16:11:44, Jim Fulton a écrit : > Getting a bit more out of initial topic: I am currently searching for papers > describing how conflict resolution is supposed to happen, and more exactly if > it is safe to "chain-resolve" for a single transaction. IMO, there's nothing particularly "safe" about conflict resolution. Conflict resolution weakens transaction isolation in ways that can (but generally don't :) produce incorrect transactions. In the future, in addition to doing conflict resolution on the client, I'd like to make it more configurable so that, among other things, it can be disabled. > Example, using a similar representation of action as in Atul Adya thesis: Reference? > > w1(x1) w2(x2) c2 c1 > -> conflict for x1, "based" on x0 and to "rebase" onto x2, so transaction 1 > starts resolving. In the meanwhile: > w3(c3) c3 w1(x1') c1 > -> conflict for transaction 1 again. This step can be repeated as many times > as desired, then eventually: > w1(x1") c1 > -> success I don't follow the above. > I could not find a doc spelling out this case yet. Let me see if I understand the case: If we do conflict resolution before the commit lock is held, then there if a chance that the object under conflict gets updated again while we're doing conflict resolution, so we may have to to conflict resolution again. Is that the case you're talking about? This case can't occur in the current ZODB implementation, so there's nothing written about it. > >From what you said on lock not being held during the first phase, I infer > >that > it is already standard for this to happen. That's not what I said. I said it gets the lock at the end of the first phase. The lock is held while conflict resolution is performed. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Conflict resolution (was Re: Notes on using wrapper storages for record transformation)
Le lundi 24 mai 2010 17:23:54, Jim Fulton a écrit : > Reference? "Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions" http://pmg.csail.mit.edu/pubs/adya99__weak_consis-abstract.html > If we do conflict resolution before the commit lock is held, > then there if a chance that the object under conflict gets updated > again while we're doing conflict resolution, so we may have to to > conflict resolution again. Is that the case you're talking about? Yes. > That's not what I said. I said it gets the lock at the end of the > first phase. The lock is held while conflict resolution is > performed. Oops, I didn't realise that ZEO was postponing Storage.store calls until tpc_vote call. I now realise that this is required to not interleave stores from different transactions in underlying storage. -- Vincent Pelletier ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Conflict resolution (was Re: Notes on using wrapper storages for record transformation)
On Mon, May 24, 2010 at 3:03 PM, Vincent Pelletier wrote: > Le lundi 24 mai 2010 17:23:54, Jim Fulton a écrit : >> Reference? > > "Weak Consistency: A Generalized Theory and Optimistic Implementations for > Distributed Transactions" > http://pmg.csail.mit.edu/pubs/adya99__weak_consis-abstract.html Thanks. Looks tasty. :) I look forward to reading it. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
