Re: [Web-SIG] Any practical reason type(environ) must be dict (not subclass)?

[Cory Benfield]

> On 25 Mar 2016, at 15:04, Jason Madden <jason.mad...@nextthought.com> wrote:
> 
> 
>> On Mar 25, 2016, at 05:01, Cory Benfield <c...@lukasa.co.uk> wrote:
>> 
>> Given that gevent is keeping hold of its own reference to the environ, why 
>> does gevent not simply wrap the environ dict in a class that implements this 
>> functionality directly? In that manner, gevent can expose its own error 
>> handling behaviour as desired, and continue to follow PEP-.
> 
> I did consider that, but didn't want to do that unless there were actual 
> practical problems passing the same object that gevent references. Making a 
> copy just to pass to the application adds additional time and memory 
> requirements that are always nice to avoid in a server.

For what it’s worth, I’m not advocating a copy. I’m advocating a class like 
this:


class SecureDictWrapper(collections.MutableMapping):
def __init__(self, environ):
self._environ = environ

That class would then implement the MutableMapping API and delegate its calls 
through to the dictionary itself. There would still only be one dictionary: the 
only new allocation is for the wrapper class. The overhead is small. =)

Cory



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Any practical reason type(environ) must be dict (not subclass)?

[Cory Benfield]

> On 24 Mar 2016, at 16:29, Jason Madden  wrote:
> Well, here's a practical use :) And the two points above do not apply to this 
> practical use, I think. (1) doesn't apply because `__repr__` is not going to 
> change and isn't fancy. (2) doesn't apply because gevent keeps a reference to 
> the environ its creates and passes to the app, so if middleware passes a new 
> dict(environ) on to the app, gevent's own error handling is still secure; 
> consider passing a SecureEnviron to the app a best-effort at 
> secure-by-default---if the user configures their application such that this 
> feature is disabled for part of the stack, that's on the application. No 
> feature of gevent will break, and it's better than not having the option at 
> all IMHO.

Given that gevent is keeping hold of its own reference to the environ, why does 
gevent not simply wrap the environ dict in a class that implements this 
functionality directly? In that manner, gevent can expose its own error 
handling behaviour as desired, and continue to follow PEP-.

In fact, I believe this is exactly what PJ was getting at. The ability to 
subclass the dictionary (in this case, to subclass it with one that hides some 
keys on printing) is only useful to the entity that does the subclassing, 
because there is no guarantee that the subclass will not be lost somewhere else 
in the WSGI stack. However, if subclassing is only useful to you there is 
another alternative to the problem, which is to compose the environ dict into 
an object that applies the custom behaviour.

Because of that, I’m disinclined to want to widen the spec here. PJ’s original 
analysis is right: allowing subclasses does not provide more utility than 
disallowing them, but it does allow more bugs to creep in due to inconsistent 
expectations. Better to have an object with a known set of behaviours and have 
applications/servers wrap it in custom function.

Cory



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Cory Benfield]

> On 10 Mar 2016, at 23:56, Andrew Godwin  wrote:
> 
> I would indeed want to require servers to always fold headers together into a 
> comma-separated list, as that's what the RFC says, and it then means 
> applications only have to deal with one kind of multi-header!

Well….kinda?

The RFC says that multiple headers are *semantically equivalent* to the joined 
form, but does not in any sense require that it be done. (The normative 
language in RFC 7230 is MAY.)

I had this discussion recently with Brian Smith: while there is only one 
correct way to fold/unfold headers, anywhere on the spectrum between completely 
folded and completely unfolded is a perfectly valid representation of the HTTP 
header block. This means that there’s no *rules* about how a server is supposed 
to do it, at least from the IETF. ASGI is of course totally allowed to add its 
own rules, and requiring that they be folded is not terrible.

FWIW, in my experience, I’ve found that “list of tuples” is really the most 
likely to be correct way to represent a header block, because it provides some 
assurances to the user that the header block has not been aggressively 
transformed from how it was sent on the wire. While the *rules* are that the 
folded representation is supposed to be semantically equivalent to the unfolded 
representation, there is nonetheless some information implicit in those headers 
being separate.

My intuition when writing this kind of thing is to pass applications (like 
Django) the most meaningful representation I can, and then allow the 
application to make its own decisions about what meaning they’re willing to 
lose. That’s why I’d advocate for “list of two-tuples of bytestrings” as the 
representation. However, I don’t think there’s anything *wrong* with forcing 
the headers to be joined by the server where possible: it’s just not how I’d do 
it. ;)

> Set-cookie is the annoying thing here, though. That's why it's dict inbound 
> and list of tuples outbound right now, and I just don't know if I want to 
> make the inbound one a list of tuples too, given I do definitely want to 
> force servers to concat headers together (unless I find any examples of that 
> screwing things up)

You could make the inbound one a list of tuples but still require that the 
servers concat headers. The rule then would be that it needs to be possible for 
an application to say `dict(headers)` without any loss of meaning.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Cory Benfield]

> On 10 Mar 2016, at 18:36, Andrew Godwin  wrote:
> 
> 
> Second, if it were me I’d remove the `status_text` field on the `Response` 
> object. Custom status text is a terrible misfeature (especially as HTTP/2 
> doesn’t support it), and in 99% of cases you’re just wasting data by 
> repeatedly sending the default phrase that the server already knows.
> 
> Well, it IS optional; you only need to send it if you're changing it from the 
> default or providing an unusual new value (e.g. 418). We could change the 
> spec to say servers don't have to abide by it, too. I have done a project in 
> the past with custom reason phrases, that's all :)

You monster! ;)

For what it’s worth, I object to the use of reason phrases because, as with all 
things in HTTP, they were far-too-broadly specified. The rules for parsing the 
reason phrase are super broad (the reason phrase allows \t, space, and then all 
bytes from 0x21 to 0xFF *excluding* 0x7F (ASCII DEL). This means that it’s 
sometimes possible to encode a reason phrase containing non-ASCII/non-Latin-1 
codepoints in UTF-8 (I’ve seen this happen), and then everything gets really 
terrible really fast.

IMO, I think almost nothing would be lost by just quietly removing it from the 
specification. The only loss is in setting “unusual” values, and FWIW I think 
that’s *also* unwise: if it can’t be found here[0] then the unusual status code 
is nothing but vanity, because it’s no more precise than the X00 version that 
already exists (no user agent can take action on it).

Again, just my 2¢.

Cory


[0]: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Cory Benfield]

> On 10 Mar 2016, at 00:34, Andrew Godwin  wrote:
> 
> To that end, I did some work to make the underlying mechanism Django Channels 
> uses into more of a standard, which I have codenamed ASGI; while initially I 
> intended for it to be a Django documented API, as I've gone further with the 
> project I've come to believe it could be useful to the Python community at 
> large.
> 

Andrew,

Thanks for this work! I’ve provided some proposed changes as pull requests 
against the channels repository. I’ll ignore those for the rest of the email: 
we can discuss them on GitHub.

I also have a few more general notes. I didn’t make PRs for these, mostly 
because they’re too “vague” as feedback goes to be concretely handled by me.

First, your HTTP section has request headers serialized to a dict and response 
headers serialized to a list of tuples. I’m not sure how I feel about that 
asymmetry: it might be cleaner just to use lists-of-tuples in both places and 
allow application frameworks to handle translation to dictionary if they 
require it.

Second, if it were me I’d remove the `status_text` field on the `Response` 
object. Custom status text is a terrible misfeature (especially as HTTP/2 
doesn’t support it), and in 99% of cases you’re just wasting data by repeatedly 
sending the default phrase that the server already knows.

Third, you’re currently sending header fields with unicode names and byte 
string values. That’s understandable, but I wonder if it’s worthwhile trying to 
limit the behaviour of compliant servers in encoding/decoding those header 
fields. For example, you could assert that the unicode header names will always 
use the Latin-1 codec when encoding/decoding. This is mostly me being paranoid 
about poorly written apps/servers issuing bad bytes onto the network. I should 
note that RFC 7230 strictly limits header names to US-ASCII, but Latin-1 would 
be the defensive choice against already-badly-written apps.

Your section on server push is great, whoever wrote that is clearly a genius. ;)

You define web socket data frames with an incrementing counter from zero, but 
also note that the maximum integer size is Python’s sys.maxint (you actually 
aren’t that clear about it, which might be a good idea). While this is 
*probably* not a problem, you may want to note that really long running or 
active web socket connections are at risk of exhausting the ‘order’ counter, 
and define a behaviour if that happens.

Otherwise, this is an interesting specification. I’m certainly open to helping 
push it through the PEP process if you’d like assistance with that.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Changes for WSGI 1.1

[Cory Benfield]

Yeah, the directionality needs to be specified more clearly here. Thanks!

Cory

> On 17 Feb 2016, at 17:55, Michael Merickel <mmeri...@gmail.com> wrote:
> 
> I'm not sure the "backward-incompatible" text in your proposal makes sense. I 
> think you actually mean to say that applications written to use new WSGI 1.1 
> features will not work with WSGI 1.0 servers. That's obvious though by the 
> version number bump. I'd be very concerned, however, if my application using 
> only WSGI 1.0 features didn't work on a WSGI 1.1 server (which is what I 
> thought you were saying the first time I saw backward-incompatible in the 
> text).
> 
> On Wed, Feb 17, 2016 at 6:55 AM, Dirkjan Ochtman <dirk...@ochtman.nl 
> <mailto:dirk...@ochtman.nl>> wrote:
> On Wed, Feb 17, 2016 at 12:51 PM, Cory Benfield <c...@lukasa.co.uk 
> <mailto:c...@lukasa.co.uk>> wrote:
> > Please let me know what you think!
> 
> I reviewed all the pull requests and they look good to me, save one
> tiny nit that I left a comment for.
> 
> Cheers,
> 
> Dirkjan
> ___
> Web-SIG mailing list
> Web-SIG@python.org <mailto:Web-SIG@python.org>
> Web SIG: http://www.python.org/sigs/web-sig 
> <http://www.python.org/sigs/web-sig>
> Unsubscribe: 
> https://mail.python.org/mailman/options/web-sig/mmericke%2Bwebsig%40gmail.com 
> <https://mail.python.org/mailman/options/web-sig/mmericke%2Bwebsig%40gmail.com>
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

[Web-SIG] Changes for WSGI 1.1

[Cory Benfield]

All,

After having had a fairly useful discussion with you all over the past few 
weeks, I’ve begun working on the next draft of the WSGI PEP, WSGI 1.1. This 
work is being tracked in a GitHub repository[0]. Currently, I’ve made a few 
changes directly on the master branch that I believe are editorial[1]: updating 
references and making linguistic changes.

So far, almost all non-editorial changes that have been proposed in this list 
I’ve made on branches and opened GitHub Pull Requests for. You can find these 
proposals here[2]. At this time, I’d like to call for people on this list who 
care to please offer their review on these proposed changes. You can view the 
changes by clicking into the specific pull request and evaluating the diff. The 
pull request should also include a short rationale for the change, or a 
description of it at the very least.

I’d also like to welcome others to propose changes that I have not yet drafted, 
or to propose changes to my drafts. Please use the standard GitHub Pull Request 
flow if you are happy with that model: if not, I’ll also accept emailed patches.

Please let me know what you think!

Cory

[0]: https://github.com/python-web-sig/wsgi-1.1
[1]: https://github.com/python-web-sig/wsgi-1.1/commits/master
[2]: https://github.com/python-web-sig/wsgi-1.1/pulls


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Collating follow-up on the future of WSGI

[Cory Benfield]

> On 21 Jan 2016, at 06:39, Benoit Chesneau  wrote:
> 
> because i am not speaking about making a specification, but a way to expose 
> in the API (environ) custom extensions that a server want to experiment. 
> there are actually no easy way except checking "wsgi." indeed but that 
> doesn't make it as clear as a separate namespace where to put all server 
> extensions could be. Like capability field is in imap world.
> 
> Also I am not trying to force anything, I want to discuss about a possible 
> update of the wsgi spec  which I thought was this thread about. What I just 
> want to discuss is the *current* usage of some extensions that have been 
> rapidly dismissed as unworkable. Like I said I will come with a more formal 
> specification about them but I wanted to discuss first about and collect 
> counter arguments which are good too.

To clarify my own original message: when I described socket escape as 
‘unworkable’, I expressly meant within the core WSGI specification as a 
mandatory feature.

I remain interested in seeing a proposal for a formalised specification for it 
as a WSGI extension, and if you’d like help in writing that proposal I’m happy 
to lend a hand.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Collating follow-up on the future of WSGI

[Cory Benfield]

> On 20 Jan 2016, at 06:04, Graham Dumpleton  wrote:
> 
> For response content, if a WSGI application currently doesn’t set a 
> Content-Length on response, A HTTP/1.1 web server is at liberty to chunk the 
> response.
> 
> So I am not sure what is missing.

My specific concern is the distinction between “at liberty to” and “required 
to”. Certain behaviours that make sense with chunked transfer encoding do not 
make sense without it: for example, streaming API endpoints that return events 
as they arrive. Sending this kind of response with a HTTP/1.0-style 
content-length absent response (framed by connection termination) is utterly 
confusing, especially as some APIs consider the chunk framing to be semantic.

This can and does bite people, because while all major production WSGI servers 
use chunked transfer encoding in this situation, not all WSGI implementations 
do: in fact, wsgiref does not. This means that if an application has a 
production design requirement to use chunked transfer encoding in its responses 
it cannot rely on the server actually providing it.

I see two solutions to this problem: we could mandate that HTTP/1.1 responses 
that have no content length must be chunked, rather than falling back to 
HTTP/1.0 style connection-termination-framed responses, or we could have 
servers stuff something in the environ dictionary that can be checked by 
applications. Or, I suppose, we can conclude that this problem is not large 
enough, and that it’s “caveat developer”.

That, however, was my concern regard chunked responses.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

[Web-SIG] Collating follow-up on the future of WSGI

[Cory Benfield]

All,

Thanks so much for your feedback to my original request for comments on the 
future of WSGI. You provided a ton of really useful feedback: when printed out 
on my printer it ended up at about 50 pages of information that was really 
engaging reading. I also want to thank you all for keeping the tone of the 
discussion so positive. On an topic like this one it can get tricky and 
emotionally charged very easily, and you did a great job of avoiding that 
problem.

I spent a few hours this morning going over your feedback and trying to extract 
some common threads. Altogether I believe that most participants were mostly in 
agreement over the direction we should take, with a few outliers in each case. 
I’d like to summarise what I believe were the big take-aways from that 
discussion to confirm that I’ve understood everyone.

I’d also like the members of this SIG to take this opportunity to discuss these 
proposals more concretely. Rather than expressing our sentiments about WSGI and 
its future more generally, I want people to critique and offer opinions on 
*these specific proposals*. The goal here is to get an understanding of whether 
these are worth doing, how we need to prioritise the work, and whether there’s 
anything we’re missing.

The below is formatted in restructured text for clarity, because it’s quite a 
lot of information.


Concrete Proposals
==

WSGI


Overall, there was strong and fairly unanimous sentiment that WSGI itself 
should more or less be left alone. Minor adjustments would be valuable, and we 
should pursue them, but the contributors do not believe that making a 
substantial revision to WSGI would be adviseable.

I therefore propose we revise the WSGI specification to WSGI 1.1 and then 
consider it "final". The following points were raised for revising WSGI.

Asynchronous WSGI
~

This was generally regarded as too substantial a change to shoehorn into WSGI.

Benoit proposed that we could achieve this change by adding a correlator to 
WSGI. This would allow servers to associate a given call to ``write()`` with a 
specific request/response (most useful in HTTP/2 where this could correspond to 
a stream ID). This would then allow WSGI to transition to a purely 
callback-based model that could in principle cohabitate with an async protocol 
in Python.

This proposal is worth highlighting not becuase I think we should pursue it 
with our revision of WSGI, but because it's worth considering for any future 
specification we come up with. Note also that Graham pointed out that this 
would require some careful rethinking of reads from ``wsgi.input``.

Server Push
~~~

We can support HTTP/2 server push using Link headers. This could optionally be 
supplemented by defining a WSGI extension that provides a callable for doing 
server push, which would be a mild improvement over the Link header approach. A 
simple proposal for how to do this can easily be implemented without revising 
the WSGI specification, and may not even need to be enshrined in a PEP. 
However, if we revise the WSGI PEP we may want to provide a small section that 
indicates how to add these headers.

Socket Escape Hatch
~~~

Aside from Benoit, server operators were unanimously dismissive of the idea of 
a socket 'escape hatch'. In general it seems like servers would not be capable 
of achieving this. I think, therefore, this idea is unworkable.

Chunked Transfer Encoding
~

It would be nice to formalise chunked transfer encoding in WSGI. Currently 
there is no way to signal to applications that chunked transfer encoding is in 
use by the client, or for the application to request it from the server.

This seemed to be a low priority work item, but if we can make this enhancement 
easily then it's worth considering.

Bytes and Unicode
~

Several contributors expressed dissatisfaction for PEP-'s approach to 
headers (namely, Latin-1-encoded Unicode strings), and expressed a preference 
for using bytestrings. If we attempt this change, we have a 
backward-compatibility concern, so we may have to live with this decision.

Regardless, this should be taken as a warning sign for any other specification 
we attempt: more on that later.

REQUEST_URI environ variable


Multiple contributors expressed an interest in bringing this environment 
variable into WSGI directly, making it a required part of the environ 
dictionary. An alternative name for this was RAW_URI.

Header Joining and Name Normalization
~

Armin pointed out that PEP- does not mention what should happen to 
normalize header names and to join header fields that appear multiple times in 
a header block. CGI does not appear to proscribe a behaviour here either.

A revision of PEP- should cover how header names get normalised (where the 
answer is basically "like CGI"), and how servers should 

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 6 Jan 2016, at 20:06, Graham Dumpleton <graham.dumple...@gmail.com> wrote:
> 
> 
>> On 6 Jan 2016, at 10:19 PM, Cory Benfield <c...@lukasa.co.uk> wrote:
>> 
>> 
>>> On 6 Jan 2016, at 09:48, Graham Dumpleton <graham.dumple...@gmail.com> 
>>> wrote:
>>> 
>>> If this does solve the push issue, what is there in HTTP/2 then that one 
>>> couldn’t do via the existing WSGI interface?
>> 
>> Well, plenty, but none that we’d *want* to expose via WSGI with the possible 
>> exception of long-running bi-directional communications channels like 
>> Websockets, which you’ve already expressed a desire to expose in a different 
>> API. =)
> 
> Can you elaborate more on the ‘plenty’ part.
> 
> This was the issue in the past. People appeared to want access to everything. 
> Thus why the belief you may as well allow them a separate API purpose built 
> for it. Maybe people are becoming more realistic in expectations now as to 
> what they really need for a typical web application as HTTP/2 is better 
> understood.

Sure. =)

HTTP/2 has the following extra things that a user may want control over:

- Preventing headers being added to the compression context. Good servers will 
automatically do this for things like Set-Cookie, but an application may have 
special knowledge about a header being security-relevant. There is no way to 
signal this in WSGI.
- Forcing certain headers to be added to the compression context. The flip side 
of the first part is that an application may know that a header is likely to be 
repeatedly re-used, despite being something that would ordinarily vary, and it 
may want to pass this knowledge to the server.
- Server push before the complete set of response headers are ready (i.e. 
before the original response body is rendered).
- Long-running bi-directional communications channels, like Websockets.
- Signaling information about client priority to the application so it can 
allocate resources effectively.
- Signaling information about flow control to the application so it can 
allocate resources effectively. (Both of these two could *maybe* be done using 
generators with WSGI, but it would be imperfect.)
- Controlling flow control windows for large responses (e.g. deliberately 
shrinking or widening them).
- Controlling flow control widows for connections as a whole.
- Graceful connection shutdown (emitting GOAWAY but continuing to process the 
outstanding requests/responses).
- Efficient stream cancellation (e.g. sending RST_STREAM for unwanted/invalid 
requests without tearing down the connection, likely related to the previous 
point)

HTTP/2 is a very featureful protocol which complex applications could really 
take advantage of. Worth remembering.


>> Pushing via Link headers is not ideal because it delays the push until after 
>> the headers are ready to go, and there’s a tricky ordering concern here (RFC 
>> 7540 points out that any PUSH_PROMISE frames should be sent before the 
>> response headers and body are sent, which means that we temporarily block 
>> the response that is ready to go from WSGI. This is a minor concern, but 
>> worth noting.)
>> 
>> However, I’m happy to say that Pushing via Link headers is the way to go if 
>> we’d rather not specify a WSGI-specific API for it.
> 
> 
> It appears that Link could at least be a fallback.
> 
> The idea of a separate callable to push resources in WSGI environ could still 
> be dealt with as an extension specification and so not need changes to the 
> WSGI specification itself. Worst case is all that callable does is add Link 
> headers to the response. This would likely have to be the case in Apache with 
> mod_h2 as wouldn’t expect handlers to have direct access to an API to do it 
> any other way, plus in mod_wsgi daemon mode is in the wrong process to access 
> any API.

Yup, this is definitely an option.



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 6 Jan 2016, at 09:19, Aymeric Augustin 
>  wrote:
> 
> Hello Benoît,
> 
> Thanks for clarifying that you also had the reverse problem in mind, headers 
> sent by applications. This side is less problematic in the sense that 
> application authors can adapt to stronger requirements.
> 
> In general this is a bit of a mess due to differences between what the RFC 
> 2616 says and what browsers do in practice. That’s why I believe the 
> pragmatic solution is to exchange bytes. (This isn’t a major issue in the 
> grand scheme of things anyway.)
> 
> Best regards,
> 

Folks, just a reminder: RFC 2616 is dead. RFC 7230 says that *newly defined* 
header fields should limit their field values to US-ASCII, but older header 
fields are a crapshoot (though it notes that “in practice, most” header field 
values use US-ASCII).

Regardless, it seems to me that the correct method of communicating field 
values would have been byte strings.

Cory



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 6 Jan 2016, at 09:48, Graham Dumpleton  wrote:
> 
> If this does solve the push issue, what is there in HTTP/2 then that one 
> couldn’t do via the existing WSGI interface?

Well, plenty, but none that we’d *want* to expose via WSGI with the possible 
exception of long-running bi-directional communications channels like 
Websockets, which you’ve already expressed a desire to expose in a different 
API. =)

Pushing via Link headers is not ideal because it delays the push until after 
the headers are ready to go, and there’s a tricky ordering concern here (RFC 
7540 points out that any PUSH_PROMISE frames should be sent before the response 
headers and body are sent, which means that we temporarily block the response 
that is ready to go from WSGI. This is a minor concern, but worth noting.)

However, I’m happy to say that Pushing via Link headers is the way to go if 
we’d rather not specify a WSGI-specific API for it.

Cory



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 5 Jan 2016, at 00:12, Graham Dumpleton <graham.dumple...@gmail.com> wrote:
> 
> 
>> On 4 Jan 2016, at 11:27 PM, Cory Benfield <c...@lukasa.co.uk 
>> <mailto:c...@lukasa.co.uk>> wrote:
>> 
>> All,
>> 
>> **TL;DR: What do you believe WSGI 2.0 should and should not do? Should we do 
>> it at all?**
>> 
>> It’s a new year, and that means it’s time for another attempt to get WSGI 
>> 2.0 off the ground. Many of you may remember that we attempted to do this 
>> last year with Rob Collins leading the charge, but unfortunately personal 
>> commitments made it impossible for Rob to keep pushing that attempt forward.
> 
> Although you call this round 2, it isn’t really. Robert’s effort was not the 
> first time someone has pushed a WSGI 2.0 variant. So this is more like being 
> about round 5 or 6.
> 
> In part because of those repeated attempts by people to propose something and 
> label it as WSGI 2.0, I am very cool on reusing the WSGI 2.0 moniker. You 
> will find little or no mention of ‘WSGI 2.0’ as a label in:
> 
> https://github.com/python-web-sig/wsgi-ng 
> <https://github.com/python-web-sig/wsgi-ng>
> 
> That is probably somewhat due to my grumbling about the use of ‘WSGI 2.0’ 
> back then.
> 
> Time has moved on and so the bad feelings and memories associated with the 
> ‘WSGI 2.0’ label due to early failed efforts have faded, but I would still 
> suggest avoiding the label ‘WSGI 2.0’ if at all possible.

Thanks for that feedback. Consider WSGI 2.0 a catch-all name for the purposes 
of this specific discussion (the “what do we want WSGI to be going forward” 
one). As you’ve suggested here, it’s entirely possible that the result of this 
discussion will be several PEPs/APIs, or none at all, and it’s entirely 
possible that none of them would be called WSGI 2.0.

> My general feeling is that if any proposed changes to the existing WSGI (PEP 
> ) specification cannot be technically implemented on all existing WSGI 
> server/adapter implementations that any new specification should not still be 
> called WSGI.
> 
> In other words, even if many of these implementations may not be used much 
> any more, it must be able to work, without needing to mark things as 
> optional, on CGI, FASTCGI, SCGI, mod_wsgi, gunicorn, uWSGI, Waitress, etc etc.
> 
> This is purely to avoid the confusion whereby implementations cannot or 
> choose not to implement any new specification. The last thing any WSGI server 
> author wants is having to deal with a constant stream of questions and bug 
> reports about not supporting an updated specification where technically it 
> was never going to be possible. We have some obligation not to inflict this 
> on what are, in nearly all cases, volunteers in the Open Source world who 
> work on these things in their spare time and who are not doing it as part of 
> their paid employment.

Can I clarify this requirement a bit? Are you wanting to say that any future 
version of WSGI must be entirely compatible with PEP : that is, may not 
introduce optional features or change existing behaviour, only clarify? Please 
don’t mistake this for me challenging the idea: I’m wanting to get a good 
understanding of what you’re suggesting with this, not agreeing or disagreeing 
at this stage.

> For example, mod_wsgi already supports HTTP/2 by virtue of the fact that the 
> mod_h2 module in Apache exists. The existing internal APIs of Apache and how 
> mod_wsgi uses those means that HTTP/2 bridges into the WSGI world with no 
> code changes to mod_wsgi.

Agreed. If all we want is to keep the request/response cycle intact, then WSGI 
supports H2 already. One possibility that has already been suggested here would 
be to define a HTTP/2 extension to WSGI, advertised in the environ dict, that 
allows the application to signal pushes to the server. This would be a fairly 
simple extension to write and implement.

> They are therefore two different APIs and so why WebSocket should be dealt 
> with in a separate specification and not carry the WSGI label at all. A 
> specific WSGI server could still support the new WebSocket API, but purely 
> because it decides to support both in the same process. Not because the 
> WebSocket API makes use of the WSGI specification.

That’s reasonable: I’d be happy to have websocket support either be a WSGI 
extension or, as you suggest here, a wholly new API. One difficulty with 
creating a new API from whole cloth is encouraging server authors to support 
it, but it’s certainly possible to do. I’d like to hear back from the uWSGI, 
gunicorn, and Twisted folks in addition to yourself about whether they’d be 
interested in implementing such a non-WSGI API.

>> - Graceful incremental adoption path - no upgrade-all-co

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

Forwarding this message from the django-developers list.

Hi Cory,

I’m not subscribed to web-sig but I read the discussion there. Feel free to 
forward my answer to the group if you think it’s useful.

I have roughly the same convictions as Graham Dumpleton. If you want to support 
HTTP/2 and WebSockets, don’t start with design decisions anchored in CGI. 
Figure out what a simple and flexible API for these new protocols would be, 
specify it, implement it, and make sure it degrades gracefully to HTTP/1. You 
may be able to channel most of the communication through a single generator, 
but it’s unclear to me that this will be the most convenient design.

If you want to improve WSGI, here’s a list of mistakes or shortcomings in PEP 
 that you can take a stab at. There’s a general theme: for a specification 
that looks at the future, I believe that making modern PaaS-based deployments 
secure by default matters more than not implementing anything beyond what’s 
available in legacy CGI-based deployments.

1. WSGI is prone to header injection vulnerabilities issues by design due to 
the conversion of HTTP headers to CGI-style environment variables: if the 
server doesn’t specifically prevent it, X-Foo and X_Foo both become HTTP_X_Foo. 
I don’t believe it’s a good choice to destructively encode headers, expect 
applications to undo the damage somehow, and introduce security vulnerabilities 
in the process. If mimicking CGI is still considered a must-have — 1% of 
current Python web programmers may have heard about it, most of them from PEP 
 — then that burden should be pushed onto the server, not the application.

2. More generally, I fail to see how mixing HTTP headers, server-related 
inputs, and environment variables in a dict adds values. It prevents iterating 
on each collection separately. It only makes sense if not offering more 
features than CGI is a design goal; in that case, this discussion doesn’t serve 
a purpose anyway. It would be nicer and possibly more secure if the application 
received separately:

a. Configuration information, which servers could read from environment 
variables by default for backwards compatibility, but could also get through 
more secure channels and restrict to what the application needs in order to 
better isolate it from the entire OS.
b. Server APIs mandated by the spec, per request.
c. HTTP headers, per request.

3. Stop pretending that HTTP is a unicode protocol, or at least stop ignoring 
reality when doing so. WSGI enforces ISO-8859-1-decoded str objects in the 
environ, which is just wrong. It’s all the more a surprising choice since this 
change was driven by Python 3, that UTF-8 is the correct choice, and that 
Python 3 defaults to UTF-8. Django has to re-encode and re-decode before doing 
anything with HTTP headers: 
https://github.com/django/django/blob/d5b90c8e120687863c1d41cf92a4cdb11413ad7f/django/core/handlers/wsgi.py#L231-L253

4. Normalize the way to tell the application about the original protocol, IP 
address and port. When dev and ops responsibilities are separate, this is 
clearly an ops responsibility, but due to the lack of standardization devs end 
up dealing with this problem in custom middleware, when they do it at all. 
Everyone keeps getting it wrong, which introduces security vulnerabilities. 
Also it always breaks silently on infrastructure changes.

5. Improve request / response length handling and connection closure. Armin and 
Graham have talked about in the past and know the topic better than I do. 
There’s also a rejected PEP by Armin which made sense to me.

As you can see from these comments, I don’t quite share the design choices that 
led to WSGI as it currently stands. I think it will be easier to build a new 
standard than evolve the current one.

I hope this helps!

Aymeric


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

[Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

All,

**TL;DR: What do you believe WSGI 2.0 should and should not do? Should we do it 
at all?**

It’s a new year, and that means it’s time for another attempt to get WSGI 2.0 
off the ground. Many of you may remember that we attempted to do this last year 
with Rob Collins leading the charge, but unfortunately personal commitments 
made it impossible for Rob to keep pushing that attempt forward.

Since then, the need for a revision of WSGI has become even more apparent. 
Casual discussion on the web has indicated that application developers are 
uncomfortable with the limitations of WSGI. These limitations are providing an 
incentive for both application developers and server developers to take an 
end-run around WSGI in an attempt to get a framework that is more suitable for 
the modern web. A great example of the result of WSGI’s deficiencies is Andrew 
Godwin’s channels work[0] for Django, which represents a paradigm shift in 
application development that takes it far away from what WSGI is today.

For this reason, I think we need to try again to get WSGI 2.0 off the ground. 
But I don’t believe we can do this without getting broad consensus from the 
developer community that a revision to WSGI is needed, and without 
understanding what developers need from a new revision of WSGI. This should 
take into account the prior discussions we’d had on this thread: however, I’m 
also going to actively solicit feedback from some of the more notable WSGI 
implementers, to ensure that whatever comes out of this SIG is something that 
they would actually use.

This WG already had a list of requirements, which are as follows:

- Support servers speaking HTTP/1.x, HTTP/2 and Websockets (potentially all on 
a single port).
- Support graceful degradation for applications that can use HTTP/2 but still 
support HTTP/1.x requests.
- Graceful incremental adoption path - no upgrade-all-components requirement 
baked into the design.
- Support Python 2.7 and 3.x (where x is not yet discussed)
- Support the existing ecosystem of containers (such as mod_wsgi) with the new 
API. We want a clean, fast and approachable API, and we want to ensure that its 
no less friendly to work with than WSGI, for all that it will expose much more 
functionality.
- Apps need to be able to tell what protocol is in use, and what optional 
features are available. For instance, HTTP/2 PUSH PROMISE is an optional 
feature that can be disabled by clients. Websockets needs to expose a socket 
like object, and so on.
- Support websockets
- Support HTTP/2
- Support HTTP/1.x (which may be just 'point at PEP-’.)
- Continue to support lightweight shims being built on top such as 
https://github.com/Pylons/webob/blob/master/webob/request.py

I believe that all of these requirements are up for grabs, and subject to 
change and consensus discussion. In this thread, then, I’d like to hear from 
people about these requirements and others. What do you believe WSGI 2.0 should 
do? Just as importantly, what do you believe it should not do? What prior art 
should we take into account? Should we bother revising WSGI at all, or should 
we let the wider application ecosystem pursue its own solutions à la Django's 
channels? Should we simply adopt Andrew Godwin’s ASGI draft[1] on which 
channels is based and call *that* WSGI 2.0?

Right now I want this to be very open. I’d like people to come up with a broad 
statement listing what they believe should and should not be present in WSGI. 
This first stage of the work is very general: I just want to get a feeling for 
what the community believes is important. Once we’re done with that, if the 
consensus is that this work is worth pursuing, I’ll come up with an initial 
draft that we can start making concrete changes to.

In the short term, I’m going to keep this consultation open for **at least two 
weeks**: that is, I will not start working on an initial draft PEP until at 
least the **18th of January**. If you believe there are application or server 
developers that should be involved in this discussion, please reach out to them 
and point them to this list. I personally have CC’d some people that I believe 
need to be involved in the discussion, but please reach out to others as well.

I’d really love to come to the end of 2016 with a solid direction for the 
future of web programming in Python. I’m looking forward to working with you 
all on achieving that.

Thanks,

Cory


[0]: https://channels.readthedocs.org/en/latest/
[1]: https://channels.readthedocs.org/en/latest/asgi.html


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 4 Jan 2016, at 14:48, Damjan Georgievski  wrote:
> 
>> **TL;DR: What do you believe WSGI 2.0 should and should not do? Should we do 
>> it at all?**
> …
>> - Support websockets
>> - Support HTTP/2
> 
> What does HTTP/2 support mean? What features of HTTP/2 need to be
> exposed in the wsgi api?

(CC-ing the list)

The current WSGI API does not provide any consensus method for doing server 
push. Such a thing could absolutely be done as an extension to WSGI in its 
current form, and we should consider that.

More generally, HTTP/2 is a bit more generous with what can be done with a 
stream than is the case in HTTP/1.1. For example, a stream could in principle 
be kept open indefinitely and used as a bi-directional communications channel: 
WSGI in its current form does not make that easy to do.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 4 Jan 2016, at 12:27, Cory Benfield <c...@lukasa.co.uk> wrote:
> 
> All,
> 
> **TL;DR: What do you believe WSGI 2.0 should and should not do? Should we do 
> it at all?**

Having set up the conversation, I also want to take part in it. So let me 
outline what I think we need from WSGI 2.

In my opinion, right now WSGI suffers from the following problems:

1. No out-of-the-box support for the concurrent programming techniques that are 
becoming extremely popular in the Python community and that should be 
encouraged.
2. No support for WebSockets, HTTP/2 server push, or any kind of generalised 
“extension” to the HTTP/1.1 request-response model.
3. A lack of an “escape hatch” that allows an application to take 
responsibility for a connection from the server, thereby ensuring that servers 
WSGI servers are the bottleneck for innovative and unusual programming 
techniques.
4. A confused bytes/unicode model that makes it mandatory to mix bytes and 
unicode in confusing ways at different parts of the stack.
5. A backward-looking design that favours applications doing in-memory 
buffering of responses over flushing response data to disk as it becomes 
available.

For this reason, my ideal WSGI 2.0 would be a relatively minor revision of PEP 
, focusing on addressing these core concerns. PEP  actually contains a 
great deal of what we already need, but either does not make it mandatory or 
fails to provide examples of how the tools can be used to solve the problem.

Problem (4) is easily resolved by coming to a consensus position on what fields 
should be byte strings, what ones should be unicode strings, and then very 
clearly setting out the rules for handling each side of that interaction.

I’d like to remove the `write` callable and instead **require** that the return 
value from the application callable be an iterable that produces the body. This 
should help encourage applications to use generators (or coroutines, as 
discussed later) to iteratively generate their output. This should help with 
problem (5) because it will allow us to propagate backpressure through a WSGI 
server to its application, based on how swiftly the server is able to flush 
data to the network.

Problem (2) is not actually entirely true: uWSGI has demonstrated that it’s 
entirely possible to extend WSGI to handle websockets, and such a model could 
presumably also be used for HTTP/2 server push. I’d like to take the approach 
of essentially creating ‘blessed’ extensions for websockets and HTTP/2 server 
push that tie in with the requirement for concurrency in problem (1). These 
will serve double duty, firstly ensuring that there’s a widely applicable 
general approach to supporting these features, while also providing a useful 
example of how WSGI 2 can be extended to support similar HTTP protocol 
extensions in future.

Fixing problem (3) seems useful to me, but I definitely need feedback from 
server implementers to see how they feel about becoming essentially dumb TCP 
proxies for applications in some scenarios.

Fixing problem (1) is the hardest, and for that reason I’ve reached out to a 
few Python concurrency experts. The best idea I have right now is based on 
trying to standardise on the ‘awaitable’ type and the ‘async for’ logic from 
PEP 492 to base WSGI 2 on asynchronous iterators. Essentially, the WSGI 2 
callable would return a PEP 493 asynchronous iterable.

I have some concerns about this approach, however. Firstly, I’m nervous about 
how well this works when called from C (e.g. for servers like uWSGI). Secondly, 
I want to make sure that it continues to function well with other event loop 
implementations (e.g. Twisted). For that reason, I’m open to alternative 
suggestions here. However, I do believe we need to support Python applications 
running concurrently in an event loop (e.g. asyncio-based or Twisted-based).

I have one further thing that I believe is important: I believe we need to 
support Python 2.7. It’s my sincere belief that if WSGI 2 is Python 3 only, we 
will struggle to get anyone to implement it: both servers and application 
frameworks will wait for people to take up Python 3 before implementing it, and 
users will not take up Python 3 until servers and application frameworks 
support WSGI 2.

What do people think of these goals?

Cory



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 4 Jan 2016, at 14:56, Damjan Georgievski  wrote:
> 
 **TL;DR: What do you believe WSGI 2.0 should and should not do? Should we 
 do it at all?**
>>> …
 - Support websockets
 - Support HTTP/2
>>> 
>>> What does HTTP/2 support mean? What features of HTTP/2 need to be
>>> exposed in the wsgi api?
>> 
>> (CC-ing the list)
>> 
>> The current WSGI API does not provide any consensus method for doing server 
>> push. Such a thing could absolutely be done as an extension to WSGI in its 
>> current form, and we should consider that.
>> 
>> More generally, HTTP/2 is a bit more generous with what can be done with a 
>> stream than is the case in HTTP/1.1. For example, a stream could in 
>> principle be kept open indefinitely and used as a bi-directional 
>> communications channel: WSGI in its current form does not make that easy to 
>> do.
>> 
> 
> So will a general solution for both HTTP/2 and Websockets be exposing
> the underlaying socket as an 'wsgi.fd' environment variable?

I don’t believe that will work.

Both HTTP/2 and Websockets have framing logic, and HTTP/2 also has a 
moderately-complex connection-level state machine that makes passing off the FD 
to an application a fraught endeavour. I suspect in both cases they will be 
best handled by having a extension protocol that allows the protocol-specific 
logic to function sensibly. In both cases, standard Python generators used as 
coroutines would be totally suitable, with concurrency being the only fly in 
the ointment to getting this right.

Cory


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Cory Benfield]

> On 4 Jan 2016, at 15:08, Armin Ronacher  wrote:
> 
> I honestly do not think that you can have it both ways: a WSGI specification 
> and a raw socket.  Maybe we reached the point where WSGI should just be 
> deprecated and frameworks themselves will fill the gap. We would only specify 
> a data exchange layer so that frameworks can interoperate in some way or 
> another.

This is one of the bits of feedback I expected we’d get, and it’s one we really 
do need to consider. It’s possible that the time for WSGI is coming to a close.

However, I’d like to try not to give up as a first step. =)

Your core question seems to be: “why do we need a spec that specifies 
concurrency?” I think this is reasonable. One way out might be to take the 
route of ASGI[0], which essentially uses a message broker to act as the 
interface between server and application. This lets applications handle their 
own concurrency without needing to co-ordinate with the server. From there the 
spec doesn’t need to handle concurrency, as the application and server are in 
separate processes.

However, if the application and server run together (as with WSGI today), I 
don’t think we can get out of needing to talk about concurrency, because *not* 
talking about it essentially forces the application to assume that each 
request/response cycle runs in a new process and that it cannot share memory or 
resources. That’s what we have at the moment, and using something like asyncio 
or Twisted in such an environment is very tricky and essentially requires that 
you run on top a server that also uses them (see Hendrix or Klein).

Let’s step back for a moment and consider the most simple case: one request, 
one response, both sent across WSGI as a complete entity. Could we not specify 
that the application callable has to return an object that is analogous to a 
future/promise/Deferred: essentially, an object that can be waited on until the 
response is ready. In that situation, a WSGI server could repeatedly call the 
application callable and then wait until any of the future/promise/Deferreds is 
ready, then send that data back in the response. This would allow the 
application to use its own concurrency model. Setting aside whether it’s a good 
model (it’s not), do you agree that it would work?

If it would, I think it’s reasonable to consider whether we can come up with a 
different, sufficiently-general approach to this problem, potentially based on 
that kind of approach.

Cory


[0]: https://channels.readthedocs.org/en/latest/asgi.html


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Pre-PEP: The WSGI Middleware Escape for Native Server APIs

[Cory Benfield]

On 30 September 2014 08:41, Roberto De Ioris robe...@unbit.it wrote:
 While i totally like your proposal, i fear it will not solve one of the
 biggest problems without another layer:

 currently (and i speak as the uWSGI author, so i am the first guilty here)
 when you want to use non-WSGI features you generally call into server api
 (like the one exposed in the 'uwsgi' virtual module). This means each
 server has its api, and this result as middlewares and apps to be adapted
 to each one (if possible)

 My proposal is to push mensa but to standardize a series of api
 (websockets and push at least) on top of it so that frameworks and
 middlewares can use them without worrying about the lower stack.

This was exactly the concern I was about to articulate. Having a
standard way to 'escape' WSGI is great, but what it does is force us
down a road where any application that wants to use HTTP/2 or
WebSockets picks one server at the start of its life and is
effectively tied to that server.

Any application small enough to be easily ported is also small enough
that it isn't a reasonable test of the API. Any application large
enough to really provide insight into the APIs is also large enough
that it will rapidly become tightly coupled to its server
implementation.

Additionally, it's a cost for server authors (unless they think they
really do have the ability to provide the 'best' API around which all
of us will rally). Server authors are going to have enough work just
making their servers speak HTTP/2 out the front, asking them to also
invest work in designing an API that *might* get used by a small
fraction of applications is really a big ask.

Finally, the odds of us getting buy-in from frameworks is surely not
very high. What interest will, for example, Armin Ronacher have in
having support for uWSGI's specific HTTP/2 API in werkzeug/Flask? What
about gunicorn's? Or mod_wsgi's?

I appreciate the argument for wanting to let server+middleware authors
develop the APIs themselves and then standardise around it, I really
do. But without a concrete plan of who is going to make the first
investment, I think it just leaves us sitting around doing nothing. A
better approach would be to say, as Roberto suggested, hey, here's
this generic WSGI escape mechanism, and here are some generic HTTP/2
and Websocket APIs you can escape to. We could even version those
APIs, allowing for communal development of them between server
authors. That provides the initial escape hook and an initial
direction, reducing the risk for individual server authors.
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI for HTTP/2.0 ?

[Cory Benfield]

On 20 September 2014 08:23, Robert Collins robe...@robertcollins.net wrote:
 I will happily discuss stuff with you off-list, but I'm not
 particularly interested in having the primary effort be cabal style -
 HTTP/2 has managed to go through a much harder rev with very strong
 personalities and much the same sort of death possible as you're
 concerned about here with great transparency.

That's true...but it has been extremely painful for all concerned.
There is minimal appetite left in the WG to continue with the work,
and a number of people quite want to put a pin in things to just give
themselves a break. I suspect this is what Graham is worried about,
and as I recall he is speaking from bitter experience.

 I don' t think thats incompatible with your needs though - for
 instance, if you want to stay offlist and debate privately to avoid
 1000-cut-pain : thats fine, but I reserve the right to summarise and
 discuss things here (and equally to ignore kibbitzing here that isn't
 being productive).

I'd like to propose instead something of a third way, inspired by the
HTTPBis. A small cabal, off-list, come up with an initial proposal
that is essentially complete (a la SPDY). That proposal should ideally
have running code behind it so that it can be played with (also like
SPDY).

That proposal is then brought to the list for further refinement. This
allows Graham to input early, and ensures open review of the proposal
while those who don't want to participate in the fully-open forum can
absent themselves from the discussion.

Of course, I'd like to help regardless of the actual procedure we use.

Cory
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI for HTTP/2.0 ?

[Cory Benfield]

On 20 September 2014 15:17, Benoit Chesneau bchesn...@gmail.com wrote:
 1) HTTP 1.1 vs HTTP 2:

 -  HTTP 1.1 and HTTP2 have quite the same high level syntax (methods, uri,
 headers, ...) but the way the data is transported differs. (data are sent by
 frames in HTTP 2).

Yes, this is correct. *In principle*, much of the way WSGI transmits
information can remain exactly the same, at least from the perspective
of working with HTTP/2.

 - in HTTP 2, data can be encrypted and compressed.

Not by default. HTTP/2 DATA frame compression got removed from the
draft spec in draft-13. It's currently available in a draft extension,
but it'll only ever be an extension to HTTP/2. HTTP/2 encryption is
just the same as for HTTP/1.1: TLS.

 At the application, things doesn't change that much. Everything can appear
 like before. The only change is the PUSH feature.

Server Push is important, but I think you've missed some really key
points in HTTP/2 that are potentially valuable to expose at the
application level.

Firstly, HPACK provides special provision for marking some headers as
'never index'[0]. This is for security reasons, and is intended to
signal that no-one should keep that header value in their header
tables. We may well want to expose this functionality.

Secondly, HTTP/2 DATA frames can be padded. Assuming padding remains
in the spec (not guaranteed), this is another security feature that we
may want to expose to the application. (Exposing this to application
is kinda stupid, but we can't leave it to the server because it won't
know what to pad and what not to pad.)

Thirdly, we need to remember that HTTP/2 streams are flow controlled.
This requires the design to very carefully consider how a response
blocked by flow control behaves.

Fourthly, the multiplexing is *prioritised*. This priority information
may need to be accessible to the application in order to make
decisions based on it.

Fifthly, while HTTP/2 is *able* to handle the standard HTTP/1.1
request-response cycle, it needn't be *limited* to it. In particular,
long-polling works a whole lot better in HTTP/2 because of fact that
stream lifetime is potentially unlimited. Similarly, because streams
are bidirectional it may become popular to use HTTP/2 streams as
ad-hoc websocket connections. These are all suggestions that we
shouldn't necessarily cleave too closely to the current WSGI paradigm.

I'm sure I've missed some other things as well. What I wanted to
highlight is that HTTP/2 is a subtle, complex protocol that is much
more powerful than the one it replaces. We should very carefully
consider how we approach a new WSGI specification, because we're going
to be stuck with it for the next few years.

I do think the idea of collating feedback is a good one, however.

[0]: 
https://tools.ietf.org/html/draft-ietf-httpbis-header-compression-09#section-7.2.3
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI for HTTP/2.0 ?

[Cory Benfield]

On 13 September 2014 19:40, Robert Collins robe...@robertcollins.net wrote:
 Is anyone interested in collaborating on an update to WSGI to support
 HTTP/2's new features?

I'd be happy to help. I know extremely little about WSGI (though I'm
sure I can read up on it), but I'm pretty heavily involved with HTTP/2
(as you know!) so I think I can provide some value.

Cory
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com