This is a revised version of my AWSGI proposal from last week. While
many of the details remain the same, the big change is that I'm now
proposing a set of extensions to standard WSGI, rather than a separate
specification for asynchronous servers.
The updated proposal is included below. I've also posted it at
http://wsgi.org/wsgi/Specifications/async
The bzr repository for my reference implementation (which is only
partially updated to match the new spec) is now at
http://pseudogreen.org/bzr/wsgiorg_async_ref/
I'd appreciate your comments.
Thanks,
Chris
Abstract
--------
This specification defines a set of extensions that allow WSGI
applications to run effectively on asynchronous (aka event driven)
servers.
Rationale
---------
The architecture of an asynchronous server requires all I/O
operations, including both interprocess and network communication, to
be non-blocking. For a WSGI-compliant server, this requirement
extends to all applications run on the server. However, the WSGI
specification does not provide sufficient facilities for an
application to ensure that its I/O is non-blocking. Specifically,
there are two issues:
* The methods provided by the input stream (``environ['wsgi.input']``)
follow the semantics of the corresponding methods of the ``file``
class. In particular, each of these methods can invoke the
underlying I/O function (in this case, ``recv`` on the socket
connected to the client) more than once, without giving the
application the opportunity to check whether each invocation will
block.
* WSGI does not provide the application with a mechanism to test
arbitrary file descriptors (such as those belonging to sockets or
pipes opened by the application) for I/O readiness.
This specification defines a standard interface by which asynchronous
servers can provide the required facilities to applications.
Specification
-------------
Servers that want to allow applications to perform non-blocking I/O
must add four new variables to the WSGI environment:
``x-wsgiorg.async.input``, ``x-wsgiorg.async.readable``,
``x-wsgiorg.async.writable``, and ``x-wsgiorg.async.timeout``. The
following sections describe these extensions.
Non-blocking Input Stream
~~~~~~~~~~~~~~~~~~~~~~~~~
The ``x-wsgiorg.async.input`` variable provides a non-blocking
replacement for ``wsgi.input``. It is an object with one method,
``read(size)``, that behaves like the ``recv`` method of
``socket.socket``. This means that a call to ``read`` will invoke the
underlying socket ``recv`` **no more than once** and return **at
most** ``size`` bytes of data (possibly less). In addition, ``read``
may return an empty string (zero bytes) **only** if the client closes
the connection or the application attempts to read more data than is
specified by the ``CONTENT_LENGTH`` variable.
Before each call to ``read``, the application **must** test the input
stream for readiness with ``x-wsgiorg.async.readable`` (see below).
The result of calling ``read`` on a non-ready input stream is
undefined.
As with ``wsgi.input``, the server is free to implement
``x-wsgiorg.async.input`` using any technique it chooses (performing
reads on demand, pre-reading the request body, etc.). The only
requirements are for ``read`` to obey the expected semantics and the
input object to be accepted as the first argument to
``x-wsgiorg.async.readable``.
Testing File Descriptors for I/O Readiness
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The variables ``x-wsgiorg.async.readable`` and
``x-wsgiorg.async.writable`` are callable objects that accept two
positional arguments, one required and one optional. In the following
description, these arguments are given the names ``fd`` and
``timeout``, but they are not required to have these names, and the
application **must** invoke the callables using positional arguments.
The first argument, ``fd``, is either an integer representing a file
descriptor or an object with a ``fileno`` method that returns such an
integer. (In addition, ``fd`` may be ``x-wsgiorg.async.input``, even
if it lacks a ``fileno`` method.) The second, optional argument,
``timeout``, is either ``None`` or a floating-point value in seconds.
If omitted, it defaults to ``None``.
When called, ``readable`` and ``writable`` return the empty string
(``''``), which **must** be yielded by the application iterable to the
server (passing through any middleware). The server then suspends
execution of the application until one of the following conditions is
met:
* The specified file descriptor is ready for reading or writing.
* ``timeout`` seconds have elapsed without the file descriptor
becoming ready for I/O.
* The server detects an error or "exceptional" condition (such as
out-of-band data) on the file descriptor.
Put another way, if the application calls ``readable`` and yields the
empty string, it will be suspended until
``select.select([fd],[],[fd],timeout)`` would return. If the
application calls ``writable`` and yields the empty string, it will be
suspended until ``select.select([],[fd],[fd],timeout)`` would return.
If ``timeout`` seconds elapse without the file descriptor becoming
ready for I/O, the variable ``x-wsgiorg.async.timeout`` will be true
when the application resumes. Otherwise, it will be false. The value
of ``x-wsgiorg.async.timeout`` when the application is first started
or after it yields each response-body string is undefined.
The server may use any technique it desires to detect when an
application's file descriptors are ready for I/O. (Most likely, it
will add them to the same event loop that it uses for accepting new
client connections, receiving requests, and sending responses.)
Examples
--------
The following application reads the request body and sends it back to
the client unmodified. Each time it wants to receive data from the
client, it first tests ``environ['x-wsgiorg.async.input']`` for
readability and then calls its ``read`` method. If the input stream
is not readable after one second, the application sends a ``408
Request Timeout`` response to the client and terminates::
def echo_request_body(environ, start_response):
input = environ['x-wsgiorg.async.input']
readable = environ['x-wsgiorg.async.readable']
nbytes = int(environ.get('CONTENT_LENGTH') or 0)
output = ''
while nbytes:
yield readable(input, 1.0) # Time out after 1 second
if environ['x-wsgiorg.async.timeout']:
msg = 'The request timed out.'
start_response('408 Request Timeout',
[('Content-Type', 'text/plain'),
('Content-Length', str(len(msg)))])
yield msg
return
data = input.read(nbytes)
if not data:
break
output += data
nbytes -= len(data)
content_type = (environ.get('CONTENT_TYPE') or 'application/
octet-stream')
start_response('200 OK', [('Content-Type', content_type),
('Content-Length', str(len(output)))])
yield output
The following middleware component allows an application that uses the
``x-wsgiorg.async`` extensions to run on a server that does not
support them, without any modification to the application's code::
def dummy_async(application):
def wrapper(environ, start_response):
input = environ['wsgi.input']
environ['x-wsgiorg.async.input'] = input
select_args = [None]
def readable(fd, timeout=None):
select_args[0] = ([fd], [], [fd], timeout)
return ''
def writable(fd, timeout=None):
select_args[0] = ([], [fd], [fd], timeout)
return ''
environ['x-wsgiorg.async.readable'] = readable
environ['x-wsgiorg.async.writable'] = writable
for result in application(environ, start_response):
if result or (not select_args[0]):
yield result
else:
if select_args[0][2][0] is input:
environ['x-wsgiorg.async.timeout'] = False
else:
ready = select.select(*select_args[0])
environ['x-wsgiorg.async.timeout'] = (ready ==
([],[],[]))
select_args[0] = None
return wrapper
Problems
--------
* The empty string yielded by an application after calling
``readable`` or ``writable`` must pass through any intervening
middleware and be detected by the server. Although WSGI explicitly
requires middleware to relay such strings to the server (see
`Middleware Handling of Block Boundaries
<http://python.org/dev/peps/pep-0333/#middleware-handling-of-block-boundaries
>`_),
some components may not, making them incompatible with this
specification.
* Although the extensions described here make it *possible* for
applications to run effectively on asynchronous servers, they do not
(and cannot) *ensure* that they do so. As is the case with any
cooperative multitasking environment, the burden of ensuring that
all application code is non-blocking rests with application authors.
Other Possibilities
-------------------
* To prevent an application that does blocking I/O from blocking the
entire server, an asynchronous server could run each instance of the
application in a separate thread. However, since asynchronous
servers achieve high levels of concurrency by expressly *avoiding*
multithreading, this technique will almost always be unacceptable.
* The `greenlet <http://codespeak.net/py/dist/greenlet.html>`_ package
enables the use of cooperatively-scheduled micro-threads in Python
programs, and a WSGI server could potentially use it to pause and
resume applications around blocking I/O operations. However, such
micro-threading is not part of the Python language or standard
library, and some server authors may be unwilling or unable to make
use of it.
Open Issues
-----------
* Some third-party libraries (such as `PycURL
<http://pycurl.sourceforge.net/>`_) provide non-blocking interfaces
that may need to monitor multiple file descriptors for I/O readiness
simultaneously. Since this specification allows an application to
wait on only one file descriptor at a time, it may be difficult or
impossible for applications to use such libraries.
Although this specification could be extended to include an
interface for waiting on multiple file descriptors, it is unclear
whether it would be easy (or even possible) for all servers to
implement it. Also, the appropriate behavior for a multi-descriptor
wait is not obvious. (Should the application be resumed when a
single descriptor is ready? All of them? Some minimum number?)
_______________________________________________
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
