Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Tue, Feb 15, 2011 at 5:54 PM, Paul Moore p.f.mo...@gmail.com wrote: The PEP should address what will happen with the dependency on zope.interface. Getting interfaces into the stdlib has *also* been discussed often in the past, and has never happened. It might even be contentious enough to warrant a second sub-PEP covering just that area. The equivalent functionality was subsumed by the inclusion of ABC support (which is what a standalone core event loop should probably use instead of zope interfaces). Definitely a PEP worth pursuing. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
2011/2/13 Antoine Pitrou solip...@pitrou.net: It would then be subject to python-dev development policy rather than twisted dev policy (which is even stricter!). Would the twisted devs *really* want that? We could use the same processes we have for externally maintained libraries, but they have without fail caused us problems. Oh, I agree with you. -1 on any new externally maintained library. The other issue is that just because we provide an alternative doesn't mean that everyone automatically stops using asyncore and migrates. Of course. asyncore's problem is not that its a maintenance burden, it's that it's really subpar compared to everything else out there. That said, Giampaolo has committed to taking it forward, so perhaps the 3.3 version of asyncore will be much (?) better. I must say that asyncore can surely be improved but not so that it can reach the flexibility offered by Twisted. Or at least, not without changing some aspects of the current API and break backward compatibility. I'll try to summarize what I think is wrong with asyncore so that maybe someone can chime in and propose ideas. Guido was right when he stated that providing a future-proof and maximally flexible design of such an API is not easy, and this is exactly the problem with asyncore. It provides a subclass-based API which doesn't work well in all those cases where I want to mix different functionallities as in: - I want a base TCP dispatcher - ...with buffered output capabilities a-la asynchat - ...which is able to limit the speed for incoming data - ...and that can also switch to SSL Although I don't use it, it seems that Twisted managed to do this by splitting the concepts of transport and protocol / application and by using zope.interface. At the current state, asyncore is not able to do this flexibly. It should at least separate transport and protocol, but again, I can't imagine how exactly and it would surely have a cost in terms of backward compatibility. Another problem is that asyncore is pretty low-level, and this is why the outcome is a code which looks monkey patched. Where Twisted provides a dataReceived() method, asyncore provides handle_read(): the user is supposed to override handle_read() and manually call recv() which might either fail (e.g. retry later or disconnected) or succeed. The same applies for all other aspects of a TCP connection: handle_accept() - accept(), handle_connect() - connect() and handle_write - send(). They all might fail and all need to be handled with care individually (see for example: http://bugs.python.org/issue6706 ). This aspect might be mitigated by providing a serie of higher lever classes (e.g. TCPServer, UDPServer, TCPConnection, UDPConnection, SSLTCPConnection) providing an API similar to: http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html ...but the need of a separation between transport and protocol layers is still needed. Last but not least, the asyncore reactor (asyncore.loop()) is not tied with the dispatcher. From the dispatcher we have no reference to the reactor, hence we cannot register/unregister file descriptors, forcing the main loop to iterate over all dispatcher instances and making impossible to benefit of epoll() and kqueue(), which are crucial for scalable servers handling thousands simultaneous requests: http://bugs.python.org/issue6692 As for what we can *currently* do without going into too much trouble I can mention: http://bugs.python.org/issue10084 http://bugs.python.org/issue1641 As for Twisted core, I think it would be a nice addition for the stdlib, but for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. The main reason why I decided to use asyncore is that, despite it's huge gaps and lack of base functionnalities, I can read its source code, understand what's going on and extend it via monkey patching. It might seem a poor approach but it worked for me and couldn't do the same when I tried to use Twisted. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Giampaolo Rodolà g.rod...@gmail.com wrote: Although I don't use it, it seems that Twisted managed to do this by splitting the concepts of transport and protocol / application and by using zope.interface. You might want to look at the ILU core, too, just for ideas. Somewhat to my surprise, the link http://www2.parc.com/istl/projects/ILU/ still works. The protocol/transport distinction is at ftp://ftp.parc.xerox.com/pub/ilu/2.0b1/manual-html/manual_14.html#SEC475. The key requirements for an async loop, IMO, are the normal file descriptor state change notifications, support for timer events, and support for time-bounded work tasks (that get run when nothing is happening). The Tornado IOLoop does all three of these; also worth taking a look at: http://www.tornadoweb.org/. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Giampaolo Rodolà wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 14 Feb, 10:15 pm, greg.ew...@canterbury.ac.nz wrote: Giampaolo Rodol� wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. Can you try to be more constructive? Generalizations like this don't help the process at all. They don't explain why Twisted's APIs shouldn't be adopted in the stdlib and they don't explain what APIs _should_ be adopted. It's basically just stop energy. I'm not picking on Giampaolo because despite the small portion of his message you quoted, his full email actually contained quite a bit of useful technical information. It wasn't just an unsupported snipe. As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 2/14/2011 5:15 PM, Greg Ewing wrote: Giampaolo Rodolà wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. I find this hard to believe, given the brainpower behind Twisted and the willingness of the Twisted devs to help with this. Starting from scratch seems like a very bad idea. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sat, Feb 12, 2011 at 8:20 PM, exar...@twistedmatrix.com wrote: What part do you think is a hard problem? Convincing people to switch to a new API? I think the hard parts is coming up with an API that's simple enough to learn quickly but powerful enough for to cover most use-cases and cleanly extendable to cover use-cases we haven't thought of. If we go with something based on or inspired by Twisted, that solves some problems, but creates others. Will users be able to later migrate to using Twisted proper? Will the standard library module and Twisted go out of sync? What happens if a user tries to use both the standard library module and Twisted? Please understand that I'm not saying these are insurmountable problems. I'm just suggesting things that might be good to address in a PEP. *Defining* the new API doesn't seem very hard to me. If you have the skills and experience so that designing a async API is not as hard for you, please run with it. :-) Personally, I would love to see asyncore deprecated in favor of something better. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Tue, Feb 15, 2011 at 10:45 AM, exar...@twistedmatrix.com wrote: As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. So far in this discussion (I'm not really contributing very much) I agree with several things: a) We should have a PEP outlining the proposed new async lib. b) It should be general purpose enough to use without Twisted (for example) I like the idea of having an async core in the std. lib that takes care of cross-platform polling of I/O descriptors, notifications and timers. cheers James -- -- James Mills -- -- Problems are solved by method ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Tue, Feb 15, 2011 at 11:09 AM, Daniel Stutzbach stutzb...@google.com wrote: If we go with something based on or inspired by Twisted, that solves some problems, but creates others. Will users be able to later migrate to using Twisted proper? Will the standard library module and Twisted go out of sync? What happens if a user tries to use both the standard library module and Twisted? Or any other async / application framework. --JamesMills ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 15 February 2011 00:45, exar...@twistedmatrix.com wrote: As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. FWIW, I am +1 on seeing a PEP for a twisted-based async framework. Probably targeted at 3.3, that should be plenty of time. I'd like it to be upward compatible with Twisted proper. If I'm expanding the scope of my code anywhere, it will be to full twisted, and I'd rather not have to rewrite what's already there. I've no reason to criticise any of the other async frameworks out there, but it seems clear to me that Twisted is well established as the best of breed in this area. The PEP should address what will happen with the dependency on zope.interface. Getting interfaces into the stdlib has *also* been discussed often in the past, and has never happened. It might even be contentious enough to warrant a second sub-PEP covering just that area. While I'm sure there's still plenty of technical issues we can cover in this thread, I think that a PEP would focus discussion a lot more clearly. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sun, Feb 13, 2011 at 2:20 PM, exar...@twistedmatrix.com wrote: The desire is there, but it's a hard problem. There was a similar discussion before PyCon 2009, but not much came of it: http://mail.python.org/pipermail/python-dev/2009-March/086678.html I started working on a PEP last year, but I didn't get very far partly because I doubted the desire. What part do you think is a hard problem? Convincing people to switch to a new API? *Defining* the new API doesn't seem very hard to me. If there is an essential subset of the API that the Twisted devs think would be a suitable replacement for asyncore, while providing a more straightforward migration path into Twisted itself, then it certainly makes sense to include it. The other possible sticking point I can see is that I don't know how Twisted's licensing works - is there anyone with the legal authority to appropriately license the code to the PSF for inclusion in the standard library? Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sun, 13 Feb 2011 19:18:52 +1000 Nick Coghlan ncogh...@gmail.com wrote: If there is an essential subset of the API that the Twisted devs think would be a suitable replacement for asyncore, while providing a more straightforward migration path into Twisted itself, then it certainly makes sense to include it. That subset would be the reactor (actually, the various reactor implementations) and its close dependencies. However, that might already amount to a sizeable chunk of code :-) (for good reason, of course: even Twisted Core does much, much more than asyncore). The other possible sticking point I can see is that I don't know how Twisted's licensing works - is there anyone with the legal authority to appropriately license the code to the PSF for inclusion in the standard library? Twisted's license is MIT-like so I don't think there would be any so-called licensing problem. :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 13/02/2011 14:23, Antoine Pitrou wrote: On Sun, 13 Feb 2011 19:18:52 +1000 Nick Coghlanncogh...@gmail.com wrote: If there is an essential subset of the API that the Twisted devs think would be a suitable replacement for asyncore, while providing a more straightforward migration path into Twisted itself, then it certainly makes sense to include it. That subset would be the reactor (actually, the various reactor implementations) and its close dependencies. However, that might already amount to a sizeable chunk of code :-) (for good reason, of course: even Twisted Core does much, much more than asyncore). It would then be subject to python-dev development policy rather than twisted dev policy (which is even stricter!). Would the twisted devs *really* want that? We could use the same processes we have for externally maintained libraries, but they have without fail caused us problems. This is usually due to maintainers leaving or going dark, which *probably* wouldn't be the case with twisted, nonetheless we've been burned enough times to be cautious about adding new externally maintained packages to the standard library. Not to mention that the twisted tests have quite a few non standard library dependencies, so integrating it would be non-trivial. That's after it has been ported to Python 3. The other issue is that just because we provide an alternative doesn't mean that everyone automatically stops using asyncore and migrates. That means the maintenance burden of asyncore doesn't necessarily go away, we just add a new maintenance burden (albeit one with lots of advantages - certainly in principle it would be *great* to have twisted-core in the standard library). The other possible sticking point I can see is that I don't know how Twisted's licensing works - is there anyone with the legal authority to appropriately license the code to the PSF for inclusion in the standard library? Twisted's license is MIT-like so I don't think there would be any so-called licensing problem. :-) That's not sufficient (IIUC). The code *authors* (copyright owners) have to agree, and probably have to sign contributor agreements. :-) Twisted have gone through an IP management process already I believe, so it is certainly possible. Michael Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
It would then be subject to python-dev development policy rather than twisted dev policy (which is even stricter!). Would the twisted devs *really* want that? We could use the same processes we have for externally maintained libraries, but they have without fail caused us problems. Oh, I agree with you. -1 on any new externally maintained library. The other issue is that just because we provide an alternative doesn't mean that everyone automatically stops using asyncore and migrates. Of course. asyncore's problem is not that its a maintenance burden, it's that it's really subpar compared to everything else out there. That said, Giampaolo has committed to taking it forward, so perhaps the 3.3 version of asyncore will be much (?) better. The other possible sticking point I can see is that I don't know how Twisted's licensing works - is there anyone with the legal authority to appropriately license the code to the PSF for inclusion in the standard library? Twisted's license is MIT-like so I don't think there would be any so-called licensing problem. :-) That's not sufficient (IIUC). The code *authors* (copyright owners) have to agree, and probably have to sign contributor agreements. :-) Well, of course. Or at least that's the theory. In practice, the algebra of open source licenses is quite well-known and non-copyleft code usually can be combined freely without any worries. (and do you think the zlib authors signed a contributor agreement for inclusion in Python distributions? :-)) Twisted have gone through an IP management process already I believe, so it is certainly possible. IP management process? What is that horrible jargon supposed to mean? :) I don't think the Twisted people are into legalese, and I've never signed an agreement when contributing (admittedly little) code to Twisted. They did relicense Twisted once (from LGPL to MIT-like), but that probably means they asked every past contributor. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: I was thinking of something lighter-weight than that. Twisted Core I just had a look at the docs for Twisted Core, and it lists 10 sub-modules. The only one that really looks core to me is twisted.internet. Drilling into that reveals another 39 public sub-sub-modules and 10 private ones. Sorry, but you'll have to chop it back quite a bit more than that before it's focused enough to be a stlib module, I think. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 08:06 pm, greg.ew...@canterbury.ac.nz wrote: exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: I was thinking of something lighter-weight than that. Twisted Core I just had a look at the docs for Twisted Core, and it lists 10 sub-modules. The only one that really looks core to me is twisted.internet. Drilling into that reveals another 39 public sub-sub-modules and 10 private ones. Sorry, but you'll have to chop it back quite a bit more than that before it's focused enough to be a stlib module, I think. Excluding stuff is not hard, seriously. It's not hard to see that wxPython integration doesn't belong in the stdlib. There are more useful aspects of the task to discuss. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Mon, Feb 14, 2011 at 8:11 AM, exar...@twistedmatrix.com wrote: Excluding stuff is not hard, seriously. It's not hard to see that wxPython integration doesn't belong in the stdlib. There are more useful aspects of the task to discuss. I think part of the problem is that those of us that aren't Twisted users aren't familiar enough with it to know which of the elements in twisted.internet would be important to include in a stdlib reactor package (or whatever it ended up being called). So we see the size of twisted.internet and start to get nervous. Our fears may be unfounded in practice, but we don't know that up front. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Mon, Feb 14, 2011 at 8:11 AM, exar...@twistedmatrix.com wrote: On 08:06 pm, greg.ew...@canterbury.ac.nz wrote: exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: I was thinking of something lighter-weight than that. Twisted Core I just had a look at the docs for Twisted Core, and it lists 10 sub-modules. The only one that really looks core to me is twisted.internet. Drilling into that reveals another 39 public sub-sub-modules and 10 private ones. Sorry, but you'll have to chop it back quite a bit more than that before it's focused enough to be a stlib module, I think. Excluding stuff is not hard, seriously. It's not hard to see that wxPython integration doesn't belong in the stdlib. There are more useful aspects of the task to discuss. I don't mean to but in here and I may have no business doing so... But what about circuits.core ? cheers James -- -- James Mills -- -- Problems are solved by method ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 13/02/2011 22:24, James Mills wrote: On Mon, Feb 14, 2011 at 8:11 AM,exar...@twistedmatrix.com wrote: On 08:06 pm, greg.ew...@canterbury.ac.nz wrote: exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewinggreg.ew...@canterbury.ac.nz wrote: I was thinking of something lighter-weight than that. Twisted Core I just had a look at the docs for Twisted Core, and it lists 10 sub-modules. The only one that really looks core to me is twisted.internet. Drilling into that reveals another 39 public sub-sub-modules and 10 private ones. Sorry, but you'll have to chop it back quite a bit more than that before it's focused enough to be a stlib module, I think. Excluding stuff is not hard, seriously. It's not hard to see that wxPython integration doesn't belong in the stdlib. There are more useful aspects of the task to discuss. I don't mean to but in here and I may have no business doing so... But what about circuits.core ? Well, what about it? The virtue of twisted is that even if we haven't all used it, we've all heard of it. That speaks volumes about its penetration into the python world. Note that the requirements for inclusion in the standard library (and at this point the conversation should really move to python-ideas) are *ideally*: * well established and widely used * well written and tested (including working on the major platforms that python runs on) * solves a common problem * the owners are submitting the code for conclusion and we have someone (preferably more than one) commited to maintaining the code in the core for the forseeable future * can be integrated with python-as-it-is-at-the-moment without bringing in new dependencies that *shouldn't* go into python core Twisted certainly meets the first three of those requirements, the last two are uncertain and still being discussed. We *don't* go around fishing for projects to include which is why we haven't suggested alternatives. There has been ongoing musing about including parts of twisted for many years however, and the core contributor to this discussion (Jean-Paul Calderone) is one of the lead developers of twisted. I think if we *were* going to include an alternative async event loop into the Python standard library there would have to be very good reasons for it *not* to be twisted, just because of the prominence of twisted within the python ecosystem. All the best, Michael Foord cheers James -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Mon, Feb 14, 2011 at 8:36 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: Well, what about it? The virtue of twisted is that even if we haven't all used it, we've all heard of it. That speaks volumes about its penetration into the python world. Just a mere suggestion. The fact that this discussion exists means that Twisted may end up being in the std. lib in the end because no-one can come up with a better? solution that meets all requirements. In any case, there are other alternatives. I realize we're not discussing them but it's nice to know what is and can be included in the std. lib, etc. I'll just follow and keep quiet now :) cheers James -- -- James Mills -- -- Problems are solved by method ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 2/13/2011 5:23 PM, Nick Coghlan wrote: On Mon, Feb 14, 2011 at 8:11 AM,exar...@twistedmatrix.com wrote: Excluding stuff is not hard, seriously. It's not hard to see that wxPython integration doesn't belong in the stdlib. There are more useful aspects of the task to discuss. I think part of the problem is that those of us that aren't Twisted users aren't familiar enough with it to know which of the elements in twisted.internet would be important to include in a stdlib reactor package (or whatever it ended up being called). To me, this is what a PEP would be for -- to present a concrete proposal listing inclusions that could be evaluated. The someone familiar with asyncore could compare feature lists to decide if the new module would have everything needed without too much other stuff. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: Nick Coghlan wrote: Flawed API + popularity = years of fun* So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Nick Coghlan wrote: Flawed API + popularity = years of fun* So maybe it's time to design a new module with a better API and deprecate the old one? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Antoine Pitrou wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. I was thinking of something lighter-weight than that. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. I was thinking of something lighter-weight than that. Twisted Core Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 12 February 2011 23:10, exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. I was thinking of something lighter-weight than that. Twisted Core Is anyone willing to package up Twisted Core for stdlib inclusion, then? Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 12:13 am, p.f.mo...@gmail.com wrote: On 12 February 2011 23:10, exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. I was thinking of something lighter-weight than that. Twisted Core Is anyone willing to package up Twisted Core for stdlib inclusion, then? Paul. Do people want to seriously consider deprecating asyncore and adding a replacement for it to the stdlib? (Hey, PyCon is coming up. How convenient. :) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sat, Feb 12, 2011 at 4:22 PM, exar...@twistedmatrix.com wrote: Do people want to seriously consider deprecating asyncore and adding a replacement for it to the stdlib? (Hey, PyCon is coming up. How convenient. :) The desire is there, but it's a hard problem. There was a similar discussion before PyCon 2009, but not much came of it: http://mail.python.org/pipermail/python-dev/2009-March/086678.html -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 12:34 am, stutzb...@google.com wrote: On Sat, Feb 12, 2011 at 4:22 PM, exar...@twistedmatrix.com wrote: Do people want to seriously consider deprecating asyncore and adding a replacement for it to the stdlib? (Hey, PyCon is coming up. How convenient. :) The desire is there, but it's a hard problem. There was a similar discussion before PyCon 2009, but not much came of it: http://mail.python.org/pipermail/python-dev/2009-March/086678.html I started working on a PEP last year, but I didn't get very far partly because I doubted the desire. What part do you think is a hard problem? Convincing people to switch to a new API? *Defining* the new API doesn't seem very hard to me. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
I started working on a PEP last year, but I didn't get very far partly because I doubted the desire. What part do you think is a hard problem? Convincing people to switch to a new API? *Defining* the new API doesn't seem very hard to me. I must say that the only time I needed the functionality asyncore provides in some Python code, 5 minutes of browsing and reading pointed me to Twisted anyway. I think that having the ultimately recommended library for such programming in stdlib is a good idea, or at least some core of it. We have a tradition of new modules replacing old ones with the same functionality gradually (command line argument parsing, for example) and if someone is motivated enough to prepare a comprehensive PEP and then working on pulling it through, I think it may work. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 11:04 PM, giampaolo.rodola python-check...@python.org wrote: Author: giampaolo.rodola Date: Fri Feb 11 14:04:18 2011 New Revision: 88395 Log: asyncore: introduce a new 'closed' attribute to make sure that dispatcher gets closed only once. In different occasions close() might be called more than once, causing problems with already disconnected sockets/dispatchers. Giampaolo, This checkin and the previous one are not appropriate for the release candidate phase of the 3.2 release. At the very least, they need to identify the second core dev that reviewed them, as well as a reference to the tracker issue where the RM approved them for inclusion. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
I'm sorry, I'm going to revert those checkins. They are very minor changes which I'm sure don't break anything, but I understand your complain. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/2/11 Nick Coghlan ncogh...@gmail.com: On Fri, Feb 11, 2011 at 11:04 PM, giampaolo.rodola python-check...@python.org wrote: Author: giampaolo.rodola Date: Fri Feb 11 14:04:18 2011 New Revision: 88395 Log: asyncore: introduce a new 'closed' attribute to make sure that dispatcher gets closed only once. In different occasions close() might be called more than once, causing problems with already disconnected sockets/dispatchers. Giampaolo, This checkin and the previous one are not appropriate for the release candidate phase of the 3.2 release. At the very least, they need to identify the second core dev that reviewed them, as well as a reference to the tracker issue where the RM approved them for inclusion. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Le vendredi 11 février 2011 à 14:52 +0100, Giampaolo Rodolà a écrit : New Revision: 88395 Log: asyncore: introduce a new 'closed' attribute to make sure that dispatcher gets closed only once. In different occasions close() might be called more than once, causing problems with already disconnected sockets/dispatchers. (...) I'm sorry, I'm going to revert those checkins. They are very minor changes which I'm sure don't break anything, but I understand your complain. dispatcher.closing is a public attribute: some programs my rely on it. I checked mine: it uses connected, but not closing :-) I think that it will be fine for Python 3.3, but not for 3.2 (too late). And you should document your change, because it is the public API. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 6:28 AM, Victor Stinner victor.stin...@haypocalc.com wrote: Le vendredi 11 février 2011 à 14:52 +0100, Giampaolo Rodolà a écrit : New Revision: 88395 Log: asyncore: introduce a new 'closed' attribute to make sure that dispatcher gets closed only once. In different occasions close() might be called more than once, causing problems with already disconnected sockets/dispatchers. (...) I'm sorry, I'm going to revert those checkins. They are very minor changes which I'm sure don't break anything, but I understand your complain. dispatcher.closing is a public attribute: some programs my rely on it. I checked mine: it uses connected, but not closing :-) I think that it will be fine for Python 3.3, but not for 3.2 (too late). And you should document your change, because it is the public API. And finally remember that asyncore is the most monkey-patched module in the world. :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 8:06 AM, Guido van Rossum gu...@python.org wrote: And finally remember that asyncore is the most monkey-patched module in the world. :-) I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, 11 Feb 2011 10:11:54 -0800 Daniel Stutzbach stutzb...@google.com wrote: On Fri, Feb 11, 2011 at 8:06 AM, Guido van Rossum gu...@python.org wrote: And finally remember that asyncore is the most monkey-patched module in the world. :-) I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. Would that be a Mapping or a Sequence? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
2011/2/11 Antoine Pitrou solip...@pitrou.net: On Fri, 11 Feb 2011 10:11:54 -0800 Daniel Stutzbach stutzb...@google.com wrote: On Fri, Feb 11, 2011 at 8:06 AM, Guido van Rossum gu...@python.org wrote: And finally remember that asyncore is the most monkey-patched module in the world. :-) I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. Would that be a Mapping or a Sequence? Actually, probably Laughable. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Yeah, the original API design (which is very inflexible) and the lack of maintenance for many years is at the base of asyncore problems. I still think it worths some love as a stdlib module, though. For 3.3 I have in mind to revamp asyncore/asynchat a bit by introducing SSL support and finally add a scheduler (issue 1641). --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/2/11 Daniel Stutzbach stutzb...@google.com: On Fri, Feb 11, 2011 at 8:06 AM, Guido van Rossum gu...@python.org wrote: And finally remember that asyncore is the most monkey-patched module in the world. :-) I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 10:36 AM, Antoine Pitrou solip...@pitrou.netwrote: Daniel Stutzbach stutzb...@google.com wrote: I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. Would that be a Mapping or a Sequence? Before or after monkey-patching? :-) -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sat, Feb 12, 2011 at 5:56 AM, Giampaolo Rodolà g.rod...@gmail.com wrote: Yeah, the original API design (which is very inflexible) and the lack of maintenance for many years is at the base of asyncore problems. I still think it worths some love as a stdlib module, though. Oh, definitely. It's popularity is half the problem :) Flawed API + lack of popularity = just fix it Flawed API + popularity = years of fun* *For a given definition of fun, that may or may not align with any real person's idea of fun ;) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Antoine Pitrou writes: Would that be a Mapping or a Sequence? Sure it would be nowhere near as predictable as a Mapping or Sequence, so Isuppose it would be a Container ... although the probability of OverflowException is near 1. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
