Re: [ZODB-Dev] Twisted, ZODB and zc.twist
Thanks for another clear answer! I'll simply ignore zc.twist (for now at least) and set up a classic thread pool. I'll still probably use zc.twist's source as an example of how to write transaction retrial on ConflicErrors and ClientDisconnecteds as I haven't found anything similar elsewhere. Erik 2011/7/29 Gary Poster gary.pos...@gmail.com: On Jul 28, 2011, at 6:38 PM, Erik Allik wrote: Gary, since you're the author of zc.twist (as I understand), I take this as an exhaustive answer, and thank you :) However, I'm still curious to know zc.twist would provide any benefit when used together with a thread pool. The thread examples in its README (which I find hard to understand) indicate that there must be some meaning to using zc.twist with threads. The threads in the README are there to let me explore a running reactor within the doctest. They are not anything to emulate. The README overstates zc.twist's usefulness. As you point out, it blocks. If you assume that the ZODB is functioning normally, it won't block much but with standard definitions of blocking, it fails. For zc.async, that's fine--but a while True loop and a time.sleep() to give some breathing room probably would have been fine, with hindsight, and much simpler. The use of Twisted is a very questionable convenience, and the way zc.async uses Twisted is not to be copied 99% or more of the time. Gary Thanks again and best regards, Erik 2011/7/29 Gary Poster gary.pos...@gmail.com: Hello Erik. As the author, I'll suggest that you move on to a ZODB connection pool used by the usual Twisted-provided thread pool. zc.twist is maybe interesting and maybe clever, but of very limited usefulness. I suspect it is only used in zc.async, and that's probably as it should be. It's a case of premature generalization. If you are going to use Twisted in anything like a normal way, and the ZODB in anything like a normal way, use a connection pool and threads. Best, Gary On Jul 28, 2011, at 4:12 PM, Erik Allik wrote: Hello everyone, I hope this is the right mailing list to post this question to. I'm trying to build an Twisted application that uses ZODB and I've also found zc.twist. I've read its README as well as the source code to try to understand how exactly and why I should use zc.twist, and also asked the same questions on irc.freenode.net #zope but without a conclusive answer. What I'm not able to understand is whether using zc.twist will allow me to do all ZODB access in the main Twisted thread, or if still need to set up a thread pool and deferToThread any code that accesses the DB. The README of zc.twist has: Everything can be done within the main thread, so it can be full-bore Twisted usage, without threads. However, looking at the source code of zc.twist, I cannot find any reference to async communication (callbacks/polling) with ZODB, so it's difficult to conclude that all DB access is non-blocking and can be done in a single thread. Also, there are some examples with threads in the README. I would highly appreciate if somebody with a clearer understanding of this could explain to me whether I in fact need to set up a thread pool, and in any case, what exactly is zc.twist for regardless of whether it needs to be used with a threadpool or not. Thanks in advance! Regards, Erik Allik P.S. I'd be very thankful for any references to open source projects/code using ZODB with Twisted, or even zc.twist (besides zc.async). ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Twisted, ZODB and zc.twist
On Jul 29, 2011, at 4:38 AM, Erik Allik wrote: Thanks for another clear answer! I'll simply ignore zc.twist (for now at least) and set up a classic thread pool. I'll still probably use zc.twist's source as an example of how to write transaction retrial on ConflicErrors and ClientDisconnecteds as I haven't found anything similar elsewhere. Cool. FWIW, http://pypi.python.org/pypi/zc.queue does something similar without threads. I prefer it (despite the fact that I said that this testing approach has known weaknesses) if your use cases allow it. Gary Erik 2011/7/29 Gary Poster gary.pos...@gmail.com: On Jul 28, 2011, at 6:38 PM, Erik Allik wrote: Gary, since you're the author of zc.twist (as I understand), I take this as an exhaustive answer, and thank you :) However, I'm still curious to know zc.twist would provide any benefit when used together with a thread pool. The thread examples in its README (which I find hard to understand) indicate that there must be some meaning to using zc.twist with threads. The threads in the README are there to let me explore a running reactor within the doctest. They are not anything to emulate. The README overstates zc.twist's usefulness. As you point out, it blocks. If you assume that the ZODB is functioning normally, it won't block much but with standard definitions of blocking, it fails. For zc.async, that's fine--but a while True loop and a time.sleep() to give some breathing room probably would have been fine, with hindsight, and much simpler. The use of Twisted is a very questionable convenience, and the way zc.async uses Twisted is not to be copied 99% or more of the time. Gary Thanks again and best regards, Erik 2011/7/29 Gary Poster gary.pos...@gmail.com: Hello Erik. As the author, I'll suggest that you move on to a ZODB connection pool used by the usual Twisted-provided thread pool. zc.twist is maybe interesting and maybe clever, but of very limited usefulness. I suspect it is only used in zc.async, and that's probably as it should be. It's a case of premature generalization. If you are going to use Twisted in anything like a normal way, and the ZODB in anything like a normal way, use a connection pool and threads. Best, Gary On Jul 28, 2011, at 4:12 PM, Erik Allik wrote: Hello everyone, I hope this is the right mailing list to post this question to. I'm trying to build an Twisted application that uses ZODB and I've also found zc.twist. I've read its README as well as the source code to try to understand how exactly and why I should use zc.twist, and also asked the same questions on irc.freenode.net #zope but without a conclusive answer. What I'm not able to understand is whether using zc.twist will allow me to do all ZODB access in the main Twisted thread, or if still need to set up a thread pool and deferToThread any code that accesses the DB. The README of zc.twist has: Everything can be done within the main thread, so it can be full-bore Twisted usage, without threads. However, looking at the source code of zc.twist, I cannot find any reference to async communication (callbacks/polling) with ZODB, so it's difficult to conclude that all DB access is non-blocking and can be done in a single thread. Also, there are some examples with threads in the README. I would highly appreciate if somebody with a clearer understanding of this could explain to me whether I in fact need to set up a thread pool, and in any case, what exactly is zc.twist for regardless of whether it needs to be used with a threadpool or not. Thanks in advance! Regards, Erik Allik P.S. I'd be very thankful for any references to open source projects/code using ZODB with Twisted, or even zc.twist (besides zc.async). ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] I would like to know if api exists nowadays to get list of all objects which where modified in a transaction ?
Dear Stephane, some time ago, someone ask how to get list of all objects which where modified in a transaction : http://article.gmane.org/gmane.comp.web.zope.zodb/5734 Jim Fulton said « there isn't a public API for this ». I would like to know if this api exists nowadays ? I do not know if this is what you are looking for: https://gist.github.com/1113786 It's a small thing that i hacked some months ago, and that provides a detailed log on what is going on behind the scenes (client-side). I agree that something like this would be useful, maybe as an add-on to ZODB just for debugging? I hope it helps. Cheers, Pedro -- José Pedro Ferreira Software Developer, Indico Project http://indico-software.org +---+ + '``'--- `+ CERN - European Organization for Nuclear Research + |CERN| / + 1211 Geneve 23, Switzerland + ..__. \. + IT-UDS-AVC + \\___.\ + Office: 513-1-005 + /+ Tel. +41227677159 +---+ ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] I would like to know if api exists nowadays to get list of all objects which where modified in a transaction ?
On Fri, Jul 29, 2011 at 03:21:23PM +0200, Pedro Ferreira wrote: some time ago, someone ask how to get list of all objects which where modified in a transaction : http://article.gmane.org/gmane.comp.web.zope.zodb/5734 I do not know if this is what you are looking for: https://gist.github.com/1113786 It's a small thing that i hacked some months ago, and that provides a detailed log on what is going on behind the scenes (client-side). I've a smaller hack in zodbbrowser (sorry for the long URL, launchpad sucks): http://bazaar.launchpad.net/~zodbbrowser-dev/zodbbrowser/trunk/view/head:/src/zodbbrowser/browser.py#L84 Basically, if you don't mind peeking in private attributes of transaction and connection objects, you can easily find out what objects were added/modified. I wouldn't hesitate use this for debugging, but I would not build business logic on top of this. Marius Gedminas -- It also makes sense in the case of 1872, where most of Horace Greenly's electors came to the sensible conclusion that the death of their candidate made him unfit for office. -- matthewwdaly explains the workings of the US Electoral College signature.asc Description: Digital signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
