2011/2/13 Antoine Pitrou <solip...@pitrou.net>: > >> It would then be subject to python-dev development policy rather than >> twisted dev policy (which is even stricter!). Would the twisted devs >> *really* want that? We could use the same processes we have for >> "externally maintained" libraries, but they have without fail caused us >> problems. > > Oh, I agree with you. -1 on any new externally maintained library. > >> The other issue is that just because we provide an alternative doesn't >> mean that everyone automatically stops using asyncore and migrates. > > Of course. asyncore's problem is not that its a maintenance burden, it's > that it's really subpar compared to everything else out there. > That said, Giampaolo has committed to taking it forward, so perhaps the > 3.3 version of asyncore will be much (?) better.
I must say that asyncore can surely be improved but not so that it can reach the flexibility offered by Twisted. Or at least, not without changing some aspects of the current API and break backward compatibility. I'll try to summarize what I think is wrong with asyncore so that maybe someone can chime in and propose ideas. Guido was right when he stated that providing a future-proof and maximally flexible design of such an API is not easy, and this is exactly the problem with asyncore. It provides a subclass-based API which doesn't work well in all those cases where I want to mix different functionallities as in: - I want a base TCP dispatcher - ...with buffered output capabilities a-la asynchat - ...which is able to limit the speed for incoming data - ...and that can also switch to SSL Although I don't use it, it seems that Twisted managed to do this by splitting the concepts of "transport" and "protocol" / "application" and by using zope.interface. At the current state, asyncore is not able to do this flexibly. It should at least separate transport and protocol, but again, I can't imagine how exactly and it would surely have a cost in terms of backward compatibility. Another problem is that asyncore is pretty low-level, and this is why the outcome is a code which looks monkey patched. Where Twisted provides a dataReceived() method, asyncore provides handle_read(): the user is supposed to override handle_read() and manually call recv() which might either fail (e.g. "retry later" or "disconnected") or succeed. The same applies for all other aspects of a TCP connection: handle_accept() -> accept(), handle_connect() -> connect() and handle_write -> send(). They all might fail and all need to be handled with care individually (see for example: http://bugs.python.org/issue6706 ). This aspect might be mitigated by providing a serie of higher lever classes (e.g. TCPServer, UDPServer, TCPConnection, UDPConnection, SSLTCPConnection) providing an API similar to: http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html ...but the need of a separation between transport and protocol layers is still needed. Last but not least, the asyncore "reactor" (asyncore.loop()) is not tied with the dispatcher. >From the dispatcher we have no reference to the reactor, hence we cannot register/unregister file descriptors, forcing the main loop to iterate over all dispatcher instances and making impossible to benefit of epoll() and kqueue(), which are crucial for scalable servers handling thousands simultaneous requests: http://bugs.python.org/issue6692 As for what we can *currently* do without going into too much trouble I can mention: http://bugs.python.org/issue10084 http://bugs.python.org/issue1641 As for Twisted core, I think it would be a nice addition for the stdlib, but for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and "taste" of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be "adapted" a bit. The main reason why I decided to use asyncore is that, despite it's huge gaps and lack of base functionnalities, I can read its source code, understand what's going on and extend it via monkey patching. It might seem a poor approach but it worked for me and couldn't do the same when I tried to use Twisted. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
