While the Web-SIG is trying to hash out PEP 444, I thought it would
be a good idea to have a backup plan that would allow the Python 3
stdlib to move forward, without needing a major new spec to settle
out implementation questions.
After all, even if PEP 333 is ultimately replaced by PEP 444, it's
probably a good idea to have *some* sort of WSGI 1-ish thing
available on Python 3, with bytes/unicode and other matters settled.
In the past, I was waiting for some consensuses (consensi?) on
Web-SIG about different approaches to Python 3, looking for some sort
of definite, "yes, we all like this" response. However, I can see
now that this just means it's my fault we don't have a spec yet. :-(
So, unless any last-minute showstopper rebuttals show up this week,
I've decided to go ahead officially bless nearly all of what Graham
Dumpleton (who's not only the mod_wsgi author, but has put huge
amounts of work into shepherding WSGI-on-Python3 proposals, WSGI
amendments, etc.) has proposed, with a few minor exceptions.
In other words: almost none of the following is my own original work;
it's like 90% Graham's. Any praise for this belongs to him; the only
thing that belongs to me is the blame for not doing this
sooner! (Sorry Graham. You asked me to do this ages ago, and you were right.)
Anyway, I'm posting this for comment to both Python-Dev and the
Web-SIG. If you are commenting on the technical details of the
amendments, please reply to the Web-SIG only. If you are commenting
on the development agenda for wsgiref or other Python 3 library
issues, please reply to Python-Dev only. That way, neither list will
see off-topic discussions. Thanks!
The Plan
========
I plan to update the proposal below per comments and feedback during
this week, then update PEP 333 itself over the weekend or early next
week, followed by a code review of Python 3's wsgiref, and
implementation of needed changes (such as recoding os.environ to
latin1-captured bytes in the CGI handler).
To complete the changes, it is possible that I may need assistance
from one or more developers who have more Python 3 experience. If
after reading the proposed changes to the spec, you would like to
volunteer to help with updating wsgiref to match, please let me know!
The Proposal
============
Overview
--------
1. The primary purpose of this update is to provide a uniform porting
pattern for moving Python 2 WSGI code to Python 3, meaning a pattern
of changes that can be mechanically applied to as little code as
practical, while still keeping the WSGI spec easy to programmatically
validate (e.g. via ``wsgiref.validate``).
The Python 3 specific changes are to use:
* ``bytes`` for I/O streams in both directions
* ``str`` for environ keys and values
* ``bytes`` for arguments to start_response() and write()
* text stream for wsgi.errors
In other words, "strings in, bytes out" for headers, bytes for bodies.
In general, only changes that don't break Python 2 WSGI
implementations are allowed. The changes should also not break
mod_wsgi on Python 3, but may make some Python 3 wsgi applications
non-compliant, despite continuing to function on mod_wsgi.
This is because mod_wsgi allows applications to output string headers
and bodies, but I am ruling that option out because it forces every
piece of middleware to have to be tested with arbitrary combinations
of strings and bytes in order to test compliance. If you want your
application to output strings rather than bytes, you can always use a
decorator to do that. (And a sample one could be provided in wsgiref.)
2. The secondary purpose of the update is to address some
long-standing open issues documented here:
http://www.wsgi.org/wsgi/Amendments_1.0
As with the Python 3 changes, only changes that don't retroactively
invalidate existing implementations are allowed.
3. There is no tertiary purpose. ;-) (By which I mean, all other
kinds of changes are out-of-scope for this update.)
4. The section below labeled "A Note On String Types" is proposed for
verbatim addition to the "Specification Overview" section in the PEP;
the other sections below describe changes to be made inline at the
appropriate part of the spec, and changes that were proposed but are
rejected for inclusion in this amendment.
A Note On String Types
----------------------
In general, HTTP deals with bytes, which means that this
specification is mostly about handling bytes.
However, the content of those bytes often has some kind of textual
interpretation, and in Python, strings are the most convenient way to
handle text.
But in many Python versions and implementations, strings are Unicode,
rather than bytes. This requires a careful balance between a usable
API and correct translations between bytes and text in the context of
HTTP... especially to support porting code between Python
implementations with different ``str`` types.
WSGI therefore defines two kinds of "string":
* "Native" strings (which are always implemented using the type named ``str``)
* "Bytestrings" (which are implemented using the ``bytes`` type in
Python 3, and ``str`` elsewhere)
So, even though HTTP is in some sense "really just bytes", there are
many API conveniences to be had by using whatever Python's default
``str`` type is.
Do not be confused however: even if Python's ``str`` is actually
Unicode under the hood, the *content* of a native string is still
restricted to bytes! See the section on `Unicode Issues`_ later in
this document.
In short: where you see the word "string" in this document, it refers
to a "native" string, i.e., an object of type ``str``, whether it is
internally implemented as bytes or unicode. Where you see references
to "bytestring", this should be read as "an object of type ``bytes``
under Python 3, or type ``str`` under Python 2".
Clarifications (To be made in-line)
-----------------------------------
The following amendments are clarifications to parts of the existing
spec that proved over the years to be ambiguous or insufficiently
specified, as well as some attempts to correct practical errors.
(Note: many of these issues cannot be completely fixed in WSGI 1
without breaking existing implementations, and so the text below has
notations such as "(MUST in WSGI 2)" to indicate where any
replacement spec for WSGI 1 should strengthen them.)
* If an application returns a body iterator, a server (or middleware)
MAY stop iterating over it and discard the remainder of the output,
as long as it calls any close() method provided by the
iterator. Applications returning a generator or other custom
iterator SHOULD NOT assume that the entire iterator will be
consumed. (This change makes it explicit that caching middleware or
HEAD-processing servers can throw away the response body.)
* start_response() SHOULD (MUST in WSGI 2) check for errors in the
status or headers at the time it's called, so that an error can be
raised as close to the problem as possible
* If start_response() raises an error when called normally (i.e.
without exc_info), it SHOULD be an error to call it a second time
without passing exc_info
* The SERVER_PORT variable is of type str, just like any other CGI
environ variable. (According to the WSGI wiki, "some
implementations" expect it to be an integer, even though there is
nothing in the WSGI spec that allows a CGI variable to be anything but a str.)
* A server SHOULD (MUST in WSGI 2) support the size hint argument to
readline() on its wsgi.input stream.
* A server SHOULD (MUST in WSGI 2) return an empty bytestring from
read() on wsgi.input to indicate an end-of-file condition. (In WSGI
2, language should be clarified to allow the input stream length and
CONTENT_LENGTH to be out of sync, for reasons explained in Graham's blog post.)
* A server SHOULD (MUST in WSGI 2) allow read() to be called without
an argument, and return the entire remaining contents of the stream
* If an application provides a Content-Length header, the server
SHOULD NOT (MUST NOT in WSGI 2) send more data to the client than was
specified in that header, whether via write(), yielded body
bytestrings, or via a wsgi.file_wrapper. (This rule applies to
middleware as well.)
* wsgi.errors is a text stream accepting "native strings"
Rejected Amendments
-------------------
* Manlio Perillo's suggestion to allow header specification to be
delayed until the response iterator is producing non-empty
output. This would've been a possible win for async WSGI, but could
require substantial changes to existing servers.
_______________________________________________
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
