On May 12, 2008, at 5:07 PM, James Y Knight wrote:
Surely you need DNS lookup to make a socket connection? Do you mean
to provide that in an external library via a threadpool?
No, I don't mean to, because I don't care enough to bother. But if
you or someone else did, you'd be free to.
You do need a framework. Using socket functions correctly (and
portably) in non-blocking mode is not trivial.
I need a library, not a framework. And I may not even need to write
it myself. (For example, for making HTTP requests, I can use pycurl.)
1) Using apache is certainly a valid option performance-wise. Apache
is pretty fast (obviously not the fastest server ever, but pretty
good...). So if it has the features/packaging you need, by all
means, use it. The advantage IMO of python servers is that they're
lighter-weight deployment-wise and more easily configurable by code.
Fair enough. But I'm specifically interested in doing non-blocking I/
O on an asynchronous server.
2) If your app uses a database, you probably might as well just run
it in a thread, because you're most likely going to use a blocking
database API anyhow.
Yes, the compatibility of database and other API's with an
asynchronous execution model is important. Some (like MySQL) don't
support non-blocking connections, so you'd have to work around that
with threads or some other technique. Others (like PostgreSQL) do
provide an async API, which could be used with my proposed
extensions. (Manlio Perillo has an example of how this works with his
nginx mod_wsgi module at http://hg.mperillo.ath.cx/nginx/mod_wsgi/file/tip/examples/nginx-postgres-async.py.)
This is another issue you have to worry about to keep your app non-
blocking, but I don't think it's an insurmountable one. And again,
any library you develop to support these operations, written in terms
of the proposed non-blocking I/O extensions, will be usable on any
server that supports the extensions.
3) If your app does not make use of outgoing sockets, then
3a) If it also doesn't use wsgi.input, you could inform the WSGI
server that it can just run the app not in a thread as it won't be
blocking.
3b) If it does use wsgi.input, but doesn't need to read it
incrementally, you could inform the server that it should pre-read
the input and then run the app directly, not in a thread, as it
won't be blocking.
If none of the above apply, that is: you do not use a database, you
do use incremental reading of wsgi.input, or an outgoing socket
connection, /then/ an async WSGI extension might be useful. I claim
that will cover a small subset of WSGI apps.
As I mentioned above, the database issue is a real one, but it can be
dealt with. I would like to be able to allow incremental reading of
wsgi.input, but I don't see how to do this without breaking
middleware. (If you have suggestions, please let me know.) As for
outgoing socket connections, I'm willing to accept the cost of a DNS
lookup; if someone else isn't, then they're free to write some kind of
local lookup server that their app talks to over a socket, and other
applications running on other servers can enjoy the fruits of their
labor.
I regret calling my proposal "Extensions for Asynchronous Servers",
since clearly that encompasses a much broader range of functionality
for you than it does for me. All I'm interested in is the ability to
poll file descriptors (and the things that allows me to do), and in
the next revision of my proposal I'll strive to make that clear. If
you have an application that requires functionality beyond that, then
my proposal won't be sufficient for your needs.
Chris
_______________________________________________
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
