Re: [Python-Dev] [python-committers] [RELEASED] Python 3.2 rc 3
On Mon, 14 Feb 2011 07:40:00 +0100 Georg Brandl ge...@python.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On behalf of the Python development team, I'm happy to announce the third release candidate of Python 3.2. Python 3.2 is a continuation of the efforts to improve and stabilize the Python 3.x line. Since the final release of Python 2.7, the 2.x line will only receive bugfixes, and new features are developed for 3.x only. [snip] It looks good:-) V. small suggestion: how about putting the New, Improved, and Deprecated Modules in What's New in alphabetical order? -- Mark Summerfield, Qtrac Ltd, www.qtrac.eu C++, Python, Qt, PyQt - training and consultancy Programming in Python 3 - ISBN 0321680561 http://www.qtrac.eu/py3book.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
2011/2/13 Antoine Pitrou solip...@pitrou.net: It would then be subject to python-dev development policy rather than twisted dev policy (which is even stricter!). Would the twisted devs *really* want that? We could use the same processes we have for externally maintained libraries, but they have without fail caused us problems. Oh, I agree with you. -1 on any new externally maintained library. The other issue is that just because we provide an alternative doesn't mean that everyone automatically stops using asyncore and migrates. Of course. asyncore's problem is not that its a maintenance burden, it's that it's really subpar compared to everything else out there. That said, Giampaolo has committed to taking it forward, so perhaps the 3.3 version of asyncore will be much (?) better. I must say that asyncore can surely be improved but not so that it can reach the flexibility offered by Twisted. Or at least, not without changing some aspects of the current API and break backward compatibility. I'll try to summarize what I think is wrong with asyncore so that maybe someone can chime in and propose ideas. Guido was right when he stated that providing a future-proof and maximally flexible design of such an API is not easy, and this is exactly the problem with asyncore. It provides a subclass-based API which doesn't work well in all those cases where I want to mix different functionallities as in: - I want a base TCP dispatcher - ...with buffered output capabilities a-la asynchat - ...which is able to limit the speed for incoming data - ...and that can also switch to SSL Although I don't use it, it seems that Twisted managed to do this by splitting the concepts of transport and protocol / application and by using zope.interface. At the current state, asyncore is not able to do this flexibly. It should at least separate transport and protocol, but again, I can't imagine how exactly and it would surely have a cost in terms of backward compatibility. Another problem is that asyncore is pretty low-level, and this is why the outcome is a code which looks monkey patched. Where Twisted provides a dataReceived() method, asyncore provides handle_read(): the user is supposed to override handle_read() and manually call recv() which might either fail (e.g. retry later or disconnected) or succeed. The same applies for all other aspects of a TCP connection: handle_accept() - accept(), handle_connect() - connect() and handle_write - send(). They all might fail and all need to be handled with care individually (see for example: http://bugs.python.org/issue6706 ). This aspect might be mitigated by providing a serie of higher lever classes (e.g. TCPServer, UDPServer, TCPConnection, UDPConnection, SSLTCPConnection) providing an API similar to: http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html ...but the need of a separation between transport and protocol layers is still needed. Last but not least, the asyncore reactor (asyncore.loop()) is not tied with the dispatcher. From the dispatcher we have no reference to the reactor, hence we cannot register/unregister file descriptors, forcing the main loop to iterate over all dispatcher instances and making impossible to benefit of epoll() and kqueue(), which are crucial for scalable servers handling thousands simultaneous requests: http://bugs.python.org/issue6692 As for what we can *currently* do without going into too much trouble I can mention: http://bugs.python.org/issue10084 http://bugs.python.org/issue1641 As for Twisted core, I think it would be a nice addition for the stdlib, but for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. The main reason why I decided to use asyncore is that, despite it's huge gaps and lack of base functionnalities, I can read its source code, understand what's going on and extend it via monkey patching. It might seem a poor approach but it worked for me and couldn't do the same when I tried to use Twisted. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rights on the tracker
On Mon, 14 Feb 2011 00:16:05 +0100, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 13 Feb 2011 14:47:01 + Alexis Métaireau ale...@notmyidea.org wrote: * Is it possible to automatically be in the noisy list for distutils2' bug reports ? Someone else with the right knowledge and power will have to do that. Done. I hope. This is the first one we (currently) have where more than one person is being auto-nosied, so I hope I got the syntax right. Someone should test it. This is separate from the meta-tracker issue Ãric mentioned, it applies only to issues where the distutils2 component is newly added. (Antoine: FYI, the place this is edited is http://bugs.python.org/component, and I don't know anything more than what is explained on that scree :). -- R. David Murray www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Giampaolo Rodolà g.rod...@gmail.com wrote: Although I don't use it, it seems that Twisted managed to do this by splitting the concepts of transport and protocol / application and by using zope.interface. You might want to look at the ILU core, too, just for ideas. Somewhat to my surprise, the link http://www2.parc.com/istl/projects/ILU/ still works. The protocol/transport distinction is at ftp://ftp.parc.xerox.com/pub/ilu/2.0b1/manual-html/manual_14.html#SEC475. The key requirements for an async loop, IMO, are the normal file descriptor state change notifications, support for timer events, and support for time-bounded work tasks (that get run when nothing is happening). The Tornado IOLoop does all three of these; also worth taking a look at: http://www.tornadoweb.org/. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Giampaolo Rodolà wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rights on the tracker
Done. I hope. This is the first one we (currently) have where more than one person is being auto-nosied, so I hope I got the syntax right. It should be fine: roundup_tracker= select * from component_add_as_nosy where nodeid = 25; linkid | nodeid + 7641 | 25 12434 | 25 (2 Zeilen) It might still be useful to test it, since actually evaluating the property might also go wrong, but that should be fine as well: for component in components: users = db.component.get(component, 'add_as_nosy') nosy |= set(users) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 14 Feb, 10:15 pm, greg.ew...@canterbury.ac.nz wrote: Giampaolo Rodol� wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. Can you try to be more constructive? Generalizations like this don't help the process at all. They don't explain why Twisted's APIs shouldn't be adopted in the stdlib and they don't explain what APIs _should_ be adopted. It's basically just stop energy. I'm not picking on Giampaolo because despite the small portion of his message you quoted, his full email actually contained quite a bit of useful technical information. It wasn't just an unsupported snipe. As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 2/14/2011 5:15 PM, Greg Ewing wrote: Giampaolo Rodolà wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. I find this hard to believe, given the brainpower behind Twisted and the willingness of the Twisted devs to help with this. Starting from scratch seems like a very bad idea. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Sat, Feb 12, 2011 at 8:20 PM, exar...@twistedmatrix.com wrote: What part do you think is a hard problem? Convincing people to switch to a new API? I think the hard parts is coming up with an API that's simple enough to learn quickly but powerful enough for to cover most use-cases and cleanly extendable to cover use-cases we haven't thought of. If we go with something based on or inspired by Twisted, that solves some problems, but creates others. Will users be able to later migrate to using Twisted proper? Will the standard library module and Twisted go out of sync? What happens if a user tries to use both the standard library module and Twisted? Please understand that I'm not saying these are insurmountable problems. I'm just suggesting things that might be good to address in a PEP. *Defining* the new API doesn't seem very hard to me. If you have the skills and experience so that designing a async API is not as hard for you, please run with it. :-) Personally, I would love to see asyncore deprecated in favor of something better. -- Daniel Stutzbach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Tue, Feb 15, 2011 at 10:45 AM, exar...@twistedmatrix.com wrote: As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. So far in this discussion (I'm not really contributing very much) I agree with several things: a) We should have a PEP outlining the proposed new async lib. b) It should be general purpose enough to use without Twisted (for example) I like the idea of having an async core in the std. lib that takes care of cross-platform polling of I/O descriptors, notifications and timers. cheers James -- -- James Mills -- -- Problems are solved by method ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On Tue, Feb 15, 2011 at 11:09 AM, Daniel Stutzbach stutzb...@google.com wrote: If we go with something based on or inspired by Twisted, that solves some problems, but creates others. Will users be able to later migrate to using Twisted proper? Will the standard library module and Twisted go out of sync? What happens if a user tries to use both the standard library module and Twisted? Or any other async / application framework. --JamesMills ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 15 February 2011 00:45, exar...@twistedmatrix.com wrote: As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. FWIW, I am +1 on seeing a PEP for a twisted-based async framework. Probably targeted at 3.3, that should be plenty of time. I'd like it to be upward compatible with Twisted proper. If I'm expanding the scope of my code anywhere, it will be to full twisted, and I'd rather not have to rewrite what's already there. I've no reason to criticise any of the other async frameworks out there, but it seems clear to me that Twisted is well established as the best of breed in this area. The PEP should address what will happen with the dependency on zope.interface. Getting interfaces into the stdlib has *also* been discussed often in the past, and has never happened. It might even be contentious enough to warrant a second sub-PEP covering just that area. While I'm sure there's still plenty of technical issues we can cover in this thread, I think that a PEP would focus discussion a lot more clearly. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
