Re: [Web-SIG] Proposal for asynchronous WSGI variant
Christopher Stawarz ha scritto: (I'm new to the list, so please forgive me for making my first post a specification proposal :) Browsing through the list archives, I see there's been some inconclusive discussions on adding better support for asynchronous web servers to the WSGI spec. Since such support would be very useful for some upcoming projects of mine, I decided to take a shot at specing out and implementing it. I'd be grateful for any feedback you have. If this seems like something worth pursuing, I would also welcome collaborators to help develop the spec further. I'm glad to know that there are some other people interested in asynchronous application, do you have seen my extensions to WSGI in my module for Nginx? The extension is documented here: http://hg.mperillo.ath.cx/nginx/mod_wsgi/file/tip/README see the Extensions chapter. For some examples: http://hg.mperillo.ath.cx/nginx/mod_wsgi/file/tip/examples/nginx-postgres-async.py http://hg.mperillo.ath.cx/nginx/mod_wsgi/file/tip/examples/nginx-poll-sleep.py Note that in Nginx the request body is pre-read before the application is called (in fact wsgi.input is either a cStringIO or File object). Unfortunately there is a *big* usability problem: the extension is based on a well specified feature of WSGI: the gateway can suspend the execution of the WSGI application when it yields. However if the asynchronous code is present in a child function, we have something like this: def application(environ, start_response): def nested(): while True: poll(xxx) yield '' yield result for r in nested(): if not r: yield '' yield r That is, all the functions in the chain have to yield, and is not very good. The solution is to use coroutines, and I'm planning to integrate greenlets (from the pylib project) into the WSGI module for Nginx. [...] Regards Manlio Perillo ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] [proposal] wsgiref.util.abs_url
Phillip J. Eby ha scritto:
At 06:27 PM 5/5/2008 +0200, Manlio Perillo wrote:
Phillip J. Eby ha scritto:
I think that it doesn't accept a relative URL, it accepts an absolute
path.
What do you mean?
environ = {}
setup_testing_defaults(environ)
url = '/a/b/'
That's a relative URL that's also an absolute path. Try a relative URL
like './a/b', or just plain 'a/b'.
self.failUnlessEqual(
util.abs_url(environ, url), 'http://127.0.0.1/a/b/')
I also think that using urlparse.urljoin() with either request_uri()
or application_uri() would be a clearer (and tested) way to obtain an
absolute URL, and more generally useful.
But application_uri also includes SCRIPT_NAME.
Yes, and you might want to use it as the base against which a relative
URL will be resolved -- i.e. an application-relative URL, vs. a
request-relative URL. In fact, application_uri() would probably be
*more* useful, since if you want a request-relative URL, there's no need
to turn it into an absolute URL, since you could just use it in its
relative form.
Yes, but this is not always the case.
Note, however, that in either case, using a relative URL that's an
absolute path (e.g. '/a/b'), will still produce the same result as your
function would. It's just that urljoin also works properly for all
kinds of relative urls, not just the absolute-path subset.
You are right, thanks.
Regards Manlio Perillo
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Proposal for asynchronous WSGI variant
On May 5, 2008, at 10:08 PM, Graham Dumpleton wrote: If write() isn't to be returned by start_response(), then do away with start_response() if possible as per discussions for WSGI 2.0. I think start_response() is necessary, because the application may need to yield for I/O readiness (e.g. to read the request body, as in my example app) before it decides what response status and headers to send. Also take note of: http://www.wsgi.org/wsgi/Amendments_1.0 and think about how Python 3.0 would affect things. OK, will do. I'd also rather it not be called AWSGI as not sufficient distinct from WSGI. If you want to pursue this asynchronous style, then be more explicitly and call it ASYNC-WSGI and use 'asyncwsgi' tag in environ. Good point. It'd be easy to type wsgi when you meant awsgi, or vice versa. But I think I'd prefer wsgi_async to asyncwsgi. Thanks, Chris ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Proposal for asynchronous WSGI variant
On May 6, 2008, at 6:17 AM, Manlio Perillo wrote: I'm glad to know that there are some other people interested in asynchronous application, do you have seen my extensions to WSGI in my module for Nginx? Yes, I have, and I had your module in mind as a potential provider of the AWSGI interface. Note that in Nginx the request body is pre-read before the application is called (in fact wsgi.input is either a cStringIO or File object). Although I didn't state it explicitly in my spec, my intention is for the server to be able to implement awsgi.input in any way it likes, as long as it provides a recv() method. It's totally acceptable for the request body to be pre-read. Unfortunately there is a *big* usability problem: the extension is based on a well specified feature of WSGI: the gateway can suspend the execution of the WSGI application when it yields. However if the asynchronous code is present in a child function, we have something like this: ... That is, all the functions in the chain have to yield, and is not very good. Yes, you're right. However, if you're willing/able to use Python 2.5, you can use the new features of generators to implement a call stack that lets you call child functions and receive return values and exceptions from them. I've implemented this in awsgiref.callstack. Have a look at http://pseudogreen.org/bzr/awsgiref/examples/echo_request_with_callstack.py for an example of how it works. The solution is to use coroutines, and I'm planning to integrate greenlets (from the pylib project) into the WSGI module for Nginx. Interesting, but it's not clear to me how/if this would work. Can you explain more or point me to some code? Thanks, Chris ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Proposal for asynchronous WSGI variant
This is a very interesting initiative.
However there are few problems:
- there is no support for chunked input - that would require having support
for readline in the first place, also, it should be the gateway's business
decoding the chunked input.
- the original wsgi spec somewhat has some support for streaming and
asynchronicity [*1]
- i don't see how removing the write callable will help (i don't see a issue
having the server providing a stringio.write as the write callable for
synchronous apps)
- passing nonstring values though middleware will make using/porting
existing wsgi middleware hairy (suppose you have a middleware that applies
some filter to the appiter - you'll have your code full of isinstance
nastiness)
Also, have you looked at the existing gateway implementations with
asynchronous support?
There are a bunch of them:
http://trac.wiretooth.com/public/wiki/asycwsgi
http://chiral.j4cbo.com/trac
http://wiki.secondlife.com/wiki/Eventlet
my own shot at the problem: http://code.google.com/p/cogen/
and manlio's mod_wsgi for nginx
(I may be missing some)
However there is absolutely no unity in handling the wsgi.input (or
equivalent)
[*1]In my implementation i do a bunch of tricks to make use of regular wsgi
middleware with async apps possible - i have a bunch of working examples
using pylons:
- the extensions in the environ (like your environ['awsgi.readable'])
return a empty string that penetrates most[*2] middleware and set the actual
message (like your (token, fd, timeout) tuple on some internal object)
From this point of view, an async middleware stack is just a set of
middleware that supports streaming.
Please see:
http://cogen.googlecode.com/svn/trunk/docs/cogen.web.async.html
http://cogen.googlecode.com/svn/trunk/docs/cogen.web.wsgi.html
[*2] middleware that consume the app iter ruin that pattern, but regardless,
they are not compliant to the wsgi spec (see
http://www.python.org/dev/peps/pep-0333/#middleware-handling-of-block-boundaries
)
- notable examples are most of the exception handling middleware (they can't
work otherwise anyway)
On Tue, May 6, 2008 at 4:30 AM, Christopher Stawarz [EMAIL PROTECTED]
wrote:
(I'm new to the list, so please forgive me for making my first post a
specification proposal :)
Browsing through the list archives, I see there's been some
inconclusive discussions on adding better support for asynchronous web
servers to the WSGI spec. Since such support would be very useful for
some upcoming projects of mine, I decided to take a shot at specing
out and implementing it. I'd be grateful for any feedback you have.
If this seems like something worth pursuing, I would also welcome
collaborators to help develop the spec further.
The name for this proposed specification is the Asynchronous Web
Server Gateway Interface (AWSGI). As the name suggests, the spec is
closely related to WSGI and is most easily described in terms of how
it differs from WSGI. AWSGI eliminates the following parts of WSGI:
- the environment variables wsgi.version and wsgi.input
- the write() callable returned by start_response()
AWSGI adds the following environment variables:
- awsgi.version
- awsgi.input
- awsgi.readable
- awsgi.writable
- awsgi.timeout
In addition, AWSGI allows the application iterable to yield two types
of data:
- byte strings, handled as in WSGI
- the result of calling awsgi.readable or awsgi.writable, which
indicates that the application should be paused and restarted when
a specified file descriptor is ready for reading or writing
Because of AWSGI's similarity to WSGI, a simple wrapper can be used to
run AWSGI applications on WSGI servers without alteration.
The following example application demonstrates typical usage of AWSGI.
This application simply reads the request body and sends it back to
the client. Each time it wants to receive data from the client, it
first tests awsgi.input for readability and then calls its recv()
method. If awsgi.input is not readable after one second, the
application sends a 408 Request Timeout response to the client and
terminates:
def echo_request_body(environ, start_response):
input = environ['awsgi.input']
readable = environ['awsgi.readable']
nbytes = int(environ.get('CONTENT_LENGTH') or 0)
output = ''
while nbytes:
yield readable(input, 1.0) # Time out after 1 second
if environ['awsgi.timeout']:
msg = 'The request timed out.'
start_response('408 Request Timeout',
[('Content-Type', 'text/plain'),
('Content-Length', str(len(msg)))])
yield msg
return
data = input.recv(nbytes)
if not data:
break
output += data
nbytes -= len(data)
start_response('200 OK', [('Content-Type', 'text/plain'),
('Content-Length',
