Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

Yes, I thought that was the case. I think adding lowercase normalisation to
header names to the spec would be sensible (daphne already does this, but
I'd like to make it reliable upon)

Andrew

On Fri, Mar 11, 2016 at 10:03 AM, Collin Anderson <cmawebs...@gmail.com>
wrote:

> http2 makes all header names lowercase
>
> On Fri, Mar 11, 2016 at 12:59 PM, Andrew Godwin <and...@aeracode.org>
> wrote:
>
>> One thing I did want to ask - is it worth still squashing everything down
>> to the same case? Daphne already clears out headers with _ in them to avoid
>> that CVE about it, and header case is never semantic, or so I thought?
>>
>> Andrew
>>
>> On Fri, Mar 11, 2016 at 9:56 AM, Andrew Godwin <and...@aeracode.org>
>> wrote:
>>
>>>
>>>
>>> On Fri, Mar 11, 2016 at 2:28 AM, Cory Benfield <c...@lukasa.co.uk>
>>> wrote:
>>>
>>>>
>>>> On 10 Mar 2016, at 23:56, Andrew Godwin <and...@aeracode.org> wrote:
>>>>
>>>> I would indeed want to require servers to always fold headers together
>>>> into a comma-separated list, as that's what the RFC says, and it then means
>>>> applications only have to deal with one kind of multi-header!
>>>>
>>>>
>>>> Well….kinda?
>>>>
>>>> The RFC says that multiple headers are *semantically equivalent* to the
>>>> joined form, but does not in any sense require that it be done. (The
>>>> normative language in RFC 7230 is MAY.)
>>>>
>>>> I had this discussion recently with Brian Smith: while there is only
>>>> one correct way to fold/unfold headers, anywhere on the spectrum between
>>>> completely folded and completely unfolded is a perfectly valid
>>>> representation of the HTTP header block. This means that there’s no *rules*
>>>> about how a server is supposed to do it, at least from the IETF. ASGI is of
>>>> course totally allowed to add its own rules, and requiring that they be
>>>> folded is not terrible.
>>>>
>>>> FWIW, in my experience, I’ve found that “list of tuples” is really the
>>>> most likely to be correct way to represent a header block, because it
>>>> provides some assurances to the user that the header block has not been
>>>> aggressively transformed from how it was sent on the wire. While the
>>>> *rules* are that the folded representation is supposed to be semantically
>>>> equivalent to the unfolded representation, there is nonetheless some
>>>> information implicit in those headers being separate.
>>>>
>>>> My intuition when writing this kind of thing is to pass applications
>>>> (like Django) the most meaningful representation I can, and then allow the
>>>> application to make its own decisions about what meaning they’re willing to
>>>> lose. That’s why I’d advocate for “list of two-tuples of bytestrings” as
>>>> the representation. However, I don’t think there’s anything *wrong* with
>>>> forcing the headers to be joined by the server where possible: it’s just
>>>> not how I’d do it. ;)
>>>>
>>>> Set-cookie is the annoying thing here, though. That's why it's dict
>>>> inbound and list of tuples outbound right now, and I just don't know if I
>>>> want to make the inbound one a list of tuples too, given I do definitely
>>>> want to force servers to concat headers together (unless I find any
>>>> examples of that screwing things up)
>>>>
>>>>
>>>> You could make the inbound one a list of tuples but still require that
>>>> the servers concat headers. The rule then would be that it needs to be
>>>> possible for an application to say `dict(headers)` without any loss of
>>>> meaning.
>>>>
>>>
>>> Yes, I think this is a good argument - my worry has always been that the
>>> "no multiples" is more of a soft rule that some clients might break or some
>>> apps might rely on the ordering/multiplicity of things, so preserving it is
>>> _probably_ helpful (and as you say, it lets the header names go back to
>>> bytestrings).
>>>
>>> I'll modify the spec and then update Daphne and Channels to match; I can
>>> leave Channels parsing both types for a bit, at least.
>>>
>>> Collin's point about http2's handling of headers is on point, too - if
>>> the new spec is deliberately thinned down to that point but no further,
>>> it's probably wise to follow them since they know much more about it than I
>>> do.
>>>
>>> Andrew
>>>
>>
>>
>> ___
>> Web-SIG mailing list
>> Web-SIG@python.org
>> Web SIG: http://www.python.org/sigs/web-sig
>> Unsubscribe:
>> https://mail.python.org/mailman/options/web-sig/cmawebsite%40gmail.com
>>
>>
>
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

One thing I did want to ask - is it worth still squashing everything down
to the same case? Daphne already clears out headers with _ in them to avoid
that CVE about it, and header case is never semantic, or so I thought?

Andrew

On Fri, Mar 11, 2016 at 9:56 AM, Andrew Godwin <and...@aeracode.org> wrote:

>
>
> On Fri, Mar 11, 2016 at 2:28 AM, Cory Benfield <c...@lukasa.co.uk> wrote:
>
>>
>> On 10 Mar 2016, at 23:56, Andrew Godwin <and...@aeracode.org> wrote:
>>
>> I would indeed want to require servers to always fold headers together
>> into a comma-separated list, as that's what the RFC says, and it then means
>> applications only have to deal with one kind of multi-header!
>>
>>
>> Well….kinda?
>>
>> The RFC says that multiple headers are *semantically equivalent* to the
>> joined form, but does not in any sense require that it be done. (The
>> normative language in RFC 7230 is MAY.)
>>
>> I had this discussion recently with Brian Smith: while there is only one
>> correct way to fold/unfold headers, anywhere on the spectrum between
>> completely folded and completely unfolded is a perfectly valid
>> representation of the HTTP header block. This means that there’s no *rules*
>> about how a server is supposed to do it, at least from the IETF. ASGI is of
>> course totally allowed to add its own rules, and requiring that they be
>> folded is not terrible.
>>
>> FWIW, in my experience, I’ve found that “list of tuples” is really the
>> most likely to be correct way to represent a header block, because it
>> provides some assurances to the user that the header block has not been
>> aggressively transformed from how it was sent on the wire. While the
>> *rules* are that the folded representation is supposed to be semantically
>> equivalent to the unfolded representation, there is nonetheless some
>> information implicit in those headers being separate.
>>
>> My intuition when writing this kind of thing is to pass applications
>> (like Django) the most meaningful representation I can, and then allow the
>> application to make its own decisions about what meaning they’re willing to
>> lose. That’s why I’d advocate for “list of two-tuples of bytestrings” as
>> the representation. However, I don’t think there’s anything *wrong* with
>> forcing the headers to be joined by the server where possible: it’s just
>> not how I’d do it. ;)
>>
>> Set-cookie is the annoying thing here, though. That's why it's dict
>> inbound and list of tuples outbound right now, and I just don't know if I
>> want to make the inbound one a list of tuples too, given I do definitely
>> want to force servers to concat headers together (unless I find any
>> examples of that screwing things up)
>>
>>
>> You could make the inbound one a list of tuples but still require that
>> the servers concat headers. The rule then would be that it needs to be
>> possible for an application to say `dict(headers)` without any loss of
>> meaning.
>>
>
> Yes, I think this is a good argument - my worry has always been that the
> "no multiples" is more of a soft rule that some clients might break or some
> apps might rely on the ordering/multiplicity of things, so preserving it is
> _probably_ helpful (and as you say, it lets the header names go back to
> bytestrings).
>
> I'll modify the spec and then update Daphne and Channels to match; I can
> leave Channels parsing both types for a bit, at least.
>
> Collin's point about http2's handling of headers is on point, too - if the
> new spec is deliberately thinned down to that point but no further, it's
> probably wise to follow them since they know much more about it than I do.
>
> Andrew
>
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

On Fri, Mar 11, 2016 at 2:28 AM, Cory Benfield <c...@lukasa.co.uk> wrote:

>
> On 10 Mar 2016, at 23:56, Andrew Godwin <and...@aeracode.org> wrote:
>
> I would indeed want to require servers to always fold headers together
> into a comma-separated list, as that's what the RFC says, and it then means
> applications only have to deal with one kind of multi-header!
>
>
> Well….kinda?
>
> The RFC says that multiple headers are *semantically equivalent* to the
> joined form, but does not in any sense require that it be done. (The
> normative language in RFC 7230 is MAY.)
>
> I had this discussion recently with Brian Smith: while there is only one
> correct way to fold/unfold headers, anywhere on the spectrum between
> completely folded and completely unfolded is a perfectly valid
> representation of the HTTP header block. This means that there’s no *rules*
> about how a server is supposed to do it, at least from the IETF. ASGI is of
> course totally allowed to add its own rules, and requiring that they be
> folded is not terrible.
>
> FWIW, in my experience, I’ve found that “list of tuples” is really the
> most likely to be correct way to represent a header block, because it
> provides some assurances to the user that the header block has not been
> aggressively transformed from how it was sent on the wire. While the
> *rules* are that the folded representation is supposed to be semantically
> equivalent to the unfolded representation, there is nonetheless some
> information implicit in those headers being separate.
>
> My intuition when writing this kind of thing is to pass applications (like
> Django) the most meaningful representation I can, and then allow the
> application to make its own decisions about what meaning they’re willing to
> lose. That’s why I’d advocate for “list of two-tuples of bytestrings” as
> the representation. However, I don’t think there’s anything *wrong* with
> forcing the headers to be joined by the server where possible: it’s just
> not how I’d do it. ;)
>
> Set-cookie is the annoying thing here, though. That's why it's dict
> inbound and list of tuples outbound right now, and I just don't know if I
> want to make the inbound one a list of tuples too, given I do definitely
> want to force servers to concat headers together (unless I find any
> examples of that screwing things up)
>
>
> You could make the inbound one a list of tuples but still require that the
> servers concat headers. The rule then would be that it needs to be possible
> for an application to say `dict(headers)` without any loss of meaning.
>

Yes, I think this is a good argument - my worry has always been that the
"no multiples" is more of a soft rule that some clients might break or some
apps might rely on the ordering/multiplicity of things, so preserving it is
_probably_ helpful (and as you say, it lets the header names go back to
bytestrings).

I'll modify the spec and then update Daphne and Channels to match; I can
leave Channels parsing both types for a bit, at least.

Collin's point about http2's handling of headers is on point, too - if the
new spec is deliberately thinned down to that point but no further, it's
probably wise to follow them since they know much more about it than I do.

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

On Thu, Mar 10, 2016 at 2:07 PM, Robert Collins <robe...@robertcollins.net>
wrote:

> On 11 March 2016 at 10:34, Andrew Godwin <and...@aeracode.org> wrote:
> >>
> >>
> >> I realise this may sound bikesheddy, but it would be really good to
> >> not call it ASGI. From your docs "
> >> Despite the name of the proposal, ASGI does not specify or design to
> >> any specific in-process async solution, such as asyncio, twisted, or
> >> gevent. Instead, the receive_many function can be switched between
> >> nonblocking or synchronous. This approach allows applications to
> >> choose what’s best for their current runtime environment; further
> >> improvements may provide extensions where cooperative versions of
> >> receive_many are provided."
> >>
> >> I'm worried that folk will assume a parallel between ASGI and asyncio,
> >> but there appears to be none... which is only a problem due to the
> >> room for confusion.
> >
> >
> > Better names are welcome, but I quite like ASGI's similarity to WSGI, and
> > the fact it's pronounceable as a single word. The "Asynchronous" part
> covers
> > the way the whole system operates; async is already an overloaded term,
> and
> > while there might be initial confusion, I think "async" also has strong
> > associations with the sort of problems ASGI solves (like websockets),
> which
> > I think is useful.
>
> Perhaps thats a particularly browser-centric view? There's nothing
> that strongly associates TCP with Python's slant on 'async' for me -
> interfaces on top of message passing can be sync or async - as in fact
> the switch you've got demonstrates :).
>
> Other names?
>
> quick thoughts...
> WSGP (web services gateway protocol)
> MuPGI (multiple protocol gateway interface)


Maybe, but this is specifically oriented as a web-based protocol - I'm not
proposing to replace all network processing here - and in that context,
"async" largely means "I can do things outside a normal request-response
process".

I guess it would take a lot for me to change the name at this point, as
it's already so many places, but I do see your point.


>
>
>
> >>  For consistency, why not a dict unicode -> List[bytes]
> >
> > I personally think this is worse than a list of tuples (which you can at
> > least feed straight into dict()) - the only header that comes through as
> > multiple, ever, is Set-Cookie, after all.
>
> I think you're wrong about that 'only header' statement.
>
> rfc 7230 3.2.2 permits multiple header fields with the same field name
> for all field values defined as comma separated lists, and for
> set-cookie.
>
> So  you can't feed it straight into dict, unless you place a
> requirement on the server to always fold together multiple header
> fields with the same field name and clients to not use that
> either. Oh, and special case Set-cookie.
>

I would indeed want to require servers to always fold headers together into
a comma-separated list, as that's what the RFC says, and it then means
applications only have to deal with one kind of multi-header!

Set-cookie is the annoying thing here, though. That's why it's dict inbound
and list of tuples outbound right now, and I just don't know if I want to
make the inbound one a list of tuples too, given I do definitely want to
force servers to concat headers together (unless I find any examples of
that screwing things up)

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

>
>
>
> I realise this may sound bikesheddy, but it would be really good to
> not call it ASGI. From your docs "
> Despite the name of the proposal, ASGI does not specify or design to
> any specific in-process async solution, such as asyncio, twisted, or
> gevent. Instead, the receive_many function can be switched between
> nonblocking or synchronous. This approach allows applications to
> choose what’s best for their current runtime environment; further
> improvements may provide extensions where cooperative versions of
> receive_many are provided."
>
> I'm worried that folk will assume a parallel between ASGI and asyncio,
> but there appears to be none... which is only a problem due to the
> room for confusion.


Better names are welcome, but I quite like ASGI's similarity to WSGI, and
the fact it's pronounceable as a single word. The "Asynchronous" part
covers the way the whole system operates; async is already an overloaded
term, and while there might be initial confusion, I think "async" also has
strong associations with the sort of problems ASGI solves (like
websockets), which I think is useful.

>  For consistency, why not a dict unicode -> List[bytes]

I personally think this is worse than a list of tuples (which you can at
least feed straight into dict()) - the only header that comes through as
multiple, ever, is Set-Cookie, after all.

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

On Thu, Mar 10, 2016 at 10:57 AM, <chris.d...@gmail.com> wrote:

> On Thu, 10 Mar 2016, Andrew Godwin wrote:
>
> I think you're right, and I've just been stubbornly trying to use a dict as
>> it's slightly "nicer". I honestly considered making both sides dict and
>> cookies the separate thing as they're the only special case, but I suspect
>> that multiple headers are one of those things that might turn out to be
>> useful for some broken client/new feature someday.
>>
>
> It sounds like you consider multiple headers of the same name in
> request and response as some kind of bug or fault. It's not it is
> perfectly legit and something I want to be able to do in my webbby
> frameworks. Vary is the main one.
>
> I know that I can join on ',' in a single header when it is
> represented in a dict but "meh".
>

Well, the protocol server would be the thing that's doing the joining if it
sees multiple headers - you'd always see comma-joined headers from clients
as an ASGI application, which I like as I like consistency.


>
> I totally agree that dicts are much nicer to work with, so I'm not
> sure what the ideal solution is, but I just wanted to raise that
> point about multiple headers. As you were. Carry on. etc.


Yeah, I find the whole comma thing a bit weird, and sort of wonder if it's
actually a workable thing for all HTTP clients. I hope it is.

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

On Thu, Mar 10, 2016 at 1:59 AM, Cory Benfield <c...@lukasa.co.uk> wrote:

>
> > On 10 Mar 2016, at 00:34, Andrew Godwin <and...@aeracode.org> wrote:
> >
> > To that end, I did some work to make the underlying mechanism Django
> Channels uses into more of a standard, which I have codenamed ASGI; while
> initially I intended for it to be a Django documented API, as I've gone
> further with the project I've come to believe it could be useful to the
> Python community at large.
> >
>
> Andrew,
>
> Thanks for this work! I’ve provided some proposed changes as pull requests
> against the channels repository. I’ll ignore those for the rest of the
> email: we can discuss them on GitHub.
>
> I also have a few more general notes. I didn’t make PRs for these, mostly
> because they’re too “vague” as feedback goes to be concretely handled by me.
>
> First, your HTTP section has request headers serialized to a dict and
> response headers serialized to a list of tuples. I’m not sure how I feel
> about that asymmetry: it might be cleaner just to use lists-of-tuples in
> both places and allow application frameworks to handle translation to
> dictionary if they require it.
>

I think you're right, and I've just been stubbornly trying to use a dict as
it's slightly "nicer". I honestly considered making both sides dict and
cookies the separate thing as they're the only special case, but I suspect
that multiple headers are one of those things that might turn out to be
useful for some broken client/new feature someday.


>
> Second, if it were me I’d remove the `status_text` field on the `Response`
> object. Custom status text is a terrible misfeature (especially as HTTP/2
> doesn’t support it), and in 99% of cases you’re just wasting data by
> repeatedly sending the default phrase that the server already knows.
>

Well, it IS optional; you only need to send it if you're changing it from
the default or providing an unusual new value (e.g. 418). We could change
the spec to say servers don't have to abide by it, too. I have done a
project in the past with custom reason phrases, that's all :)


>
> Third, you’re currently sending header fields with unicode names and byte
> string values. That’s understandable, but I wonder if it’s worthwhile
> trying to limit the behaviour of compliant servers in encoding/decoding
> those header fields. For example, you could assert that the unicode header
> names will always use the Latin-1 codec when encoding/decoding. This is
> mostly me being paranoid about poorly written apps/servers issuing bad
> bytes onto the network. I should note that RFC 7230 strictly limits header
> names to US-ASCII, but Latin-1 would be the defensive choice against
> already-badly-written apps.
>

Yes, it's perhaps an unwritten understanding that they're meant to be
encoded/decoded only to latin1, and I believe this is what Daphne does;
they're unicode mostly as that makes keying into the header dictionary much
nicer in py3/unicode_literals land, and because there's a clear encoding
way to handle them.


>
> Your section on server push is great, whoever wrote that is clearly a
> genius. ;)
>
> You define web socket data frames with an incrementing counter from zero,
> but also note that the maximum integer size is Python’s sys.maxint (you
> actually aren’t that clear about it, which might be a good idea). While
> this is *probably* not a problem, you may want to note that really long
> running or active web socket connections are at risk of exhausting the
> ‘order’ counter, and define a behaviour if that happens.
>

Ah, good catch. I'll specify a very high maximum order number for any
protocol and say it rolls over to 0 for the next one, and then I can modify
channels' global_ordering to expect that - I think that's the most sensible
approach here.


>
> Otherwise, this is an interesting specification. I’m certainly open to
> helping push it through the PEP process if you’d like assistance with that.
>
>
If we see some rough agreement on it, yes, I would love some help with that.

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

[Web-SIG] Inviting feedback on my proposed "ASGI" spec

[Andrew Godwin]

Hi all,

As some of you may know, I've been working over the past few months to
bring native WebSocket support to Django, via a project codenamed "Django
Channels" - this is mostly the reason I've been involved in recent WSGI
discussions.

I'm personally of the opinion that WSGI works well for HTTP, with a few
improvements we can roll into a 1.1, but that we also need something else
that can support WebSockets and other future web protocols (e.g. WebRTC
components).

To that end, I did some work to make the underlying mechanism Django
Channels uses into more of a standard, which I have codenamed ASGI; while
initially I intended for it to be a Django documented API, as I've gone
further with the project I've come to believe it could be useful to the
Python community at large.

My intention would be for this spec to sit alongside WSGI, and be a second
option for both servers and frameworks to support (if they wished) that
supports both HTTP and WebSocket connections, as well as a reasonable way
to extend it to future protocols.

All current applications and servers could continue to work via adapter
classes that transform ASGI to WSGI on either end of its HTTP path, which I
think is an important migration consideration.

I'd love some feedback from this group on my proposed specification, and
any major problems you forsee; there are a few issues I know about, mostly
potential performance issues, but in most of those cases I believe the
gains outweigh the loss. The major change is that servers and applications
now run independently, either in separate threads or processes, and
communicate bidirectionally over a "channel layer", rather than the server
calling the application directly.

I'm not yet angling to take this to a PEP, but that would be my eventual
goal; right now, I want to get feedback from people on their major
likes/dislikes, and how it works for various parts of the Python web
ecosystem.

The spec already has an application framework (Django), web/websocket
server (Daphne [1]) and three channel layers [2] implemented, so I've
ironed out some major problems it initially had from working on those, but
I'm not as experienced in the rigours of serving HTTP as most of you are. I
do encourage you, though, to take a look at the rest of the Channels docs
if you want to get an idea of how it works and deploys in practice.

Spec is up here: http://channels.readthedocs.org/en/latest/asgi.html

Helpful quick Q: http://channels.readthedocs.org/en/latest/inshort.html

I do believe that making a clean break from WSGI to a new structure (and
NOT calling it "WSGI 2") is the best thing we can do if we truly want to
support more web protocols properly, and I believe that doing that in a way
that still supports WSGI and provides a nice migration path is important -
I believe ASGI provides both of these things, as well as a relatively
simple core API (one of WSGI's strengths in my opinion) - but I welcome
your opinions as well.

Andrew

[1] https://github.com/andrewgodwin/daphne
[2] https://github.com/andrewgodwin/asgi_redis,
https://github.com/andrewgodwin/asgiref/blob/master/asgiref/inmemory.py,
https://github.com/andrewgodwin/channels/blob/master/channels/database_layer.py
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] Fwd: Collating follow-up on the future of WSGI

[Andrew Godwin]

On Sun, Jan 31, 2016 at 4:42 PM, PJ Eby  wrote:

> On Thu, Jan 21, 2016 at 12:12 AM, Benoit Chesneau 
> wrote:
> > I am not speaking about websockets. You could use it for SSE, or some
> apps
> > could use the Upgrade header to upgrade from http to their own protocol
> > etc... The only discussion i saw about websockets are about the addition
> of
> > an async api or an external api. I am not describing that. I am speaking
> > about providing a low level abstraction like wsgi.input but adding to it
> the
> > support of output. (I was referring to wsgi.multithread...). This low
> level
> > interface would allow anyone to provide its own implementation(server) or
> > usage (application) still acting as a *gateway* .
>
> It sounds like you may be looking for something like this:
>
> https://gist.github.com/pjeby/62e3892cd75257518eb0
>
> It's a proposed standard protocol for breaking out of WSGI from inside
> of a WSGI application, to access other protocols.  It doesn't deal
> with the details of any particular upgraded protocol, it merely
> provides an "upgrade to specifed protocol" API and a way to safely
> pass it through arbitrary middleware.  The `wsgi.upgrades` environment
> key is used to list available protocols.


The idea of a standardised protocol escape is indeed interesting, though
I'm not so keen on the idea of making triply nested functions a requirement
for something like this.

How would you see this interacting with potential asynchronous
frameworks/reactors? The WebOb example only reacts to events also coming
from within a WebSocket abstraction, but what if I wanted to e.g. send a
message on database save? How would my application manage to trigger the
WebSocket, in another thread or process, to do that?

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Andrew Godwin]

On Mon, Jan 4, 2016 at 6:22 PM, Robert Collins 
wrote:
>
> I think that WSGI got many things right - thats why so many things
> support it - but identifying which of its attributes is a factor for
> success, and which isn't is really hard: we're a decade on, more or
> less, and the ecosystem is a lot bigger. w.r.t. ASGI, it sounds like
> mongrel2 just in earlier days. Its a nifty way to build things, but I
> think its missing a fairly important thing: its not conceptually
> simple any more.
>

I appreciate that, though I would argue that the spec makes it seem
more complex than it actually is. You can write a ASGI application in
about 4 lines, in a way that I think is no worse than start_response and
environ in WSGI:

while True:
_, request = layer.receive_many("http.request", block=True)
response = {"content": "Hello World!", "status": 200}
layer.send(request['reply_channel'], response)

There's no reason ASGI couldn't have a WSGI-like layer specified on top for
HTTP requests so it looked more like current WSGI, but with slightly nicer
request and response objects. That is, in fact, mostly what the Django
Channels code does right now.

Mongrel2's design had some other issues, ZeroMQ being top of the list
(it's just not a good queuing system), as well as being way too stateful
and complex.
I could probably go on about the differences at length, but I'll refrain :)


>  - likewise on the application side: we need a Django person engaged
> with this - I reached out for one at the start of the last initiative
> but didn't manage to hook anyone, flask folk being interested would be
> good etc.
>

Hi, I am very interested, obviously.

Django _will probably have_ websocket support by next release, and it
almost certainly will run using ASGI internally; what's up for debate is
how it gets served, either via a WSGI shim, a WSGI-NG shim or native
support in a server.

I'm a big believer in going for implementation and iteration and polish
before jumping to a PEP, which is why I didn't bring ASGI anywhere near
that process yet. Once it's done, tested, implemented and we get it running
well at scale, then I think it would be ready. Django got a big pot of
funding from Mozilla just to get this process done, and several large sites
waiting to beta test things and report back how they work in the real world.

That said, I'm not stuck in some world where Django is the only Python web
framework; I would love to see the input of others. My ideal would be for a
new spec to renew the interest in writing more "raw" web apps in Python,
rather than people turning to other languages which have ostensibly better
WebSocket etc. support.


> On the asyncio / await etc side: I leave this up to the author.
> Thunking to WSGI from awaitable's would be tedious and thread-heavy I
> suspect. Obviously it can be done. AFAIK right now there is no mass
> move to asyncio from the server frameworks around: we're not going to
> solve the problems they have today with WSGI (e.g. the server-specific
> websockets requirements) by  making standardisation be predicated on
> moving something like Django to a new server core. I recall attempts
> to move other big frameworks around years ago now, taking a long long
> time I think the question of 'how can we enable interop of HTTP
> applications in asyncio' is best handled by a dedicated effort
> analogous to WSGI but in that programming model.
>

I agree that tying things to asyncio is a Bad Idea; any proposal should
certainly be more friendly to it, but requiring people to write fully
asynchronous code is a sure way to lose the simplicity of WSGI. We'd have
to do an incredible amount of work to Django to make it fully asynchronous
like that, to the point where it may never happen. What would probably
happen is just serving requests one at a time using thread pools or
something.

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com

Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest

[Andrew Godwin]

Thought I should weigh in on this, as I got mentioned by name in it. Sorry
about maybe not getting the threading right, I wasn't subscribed to the
list still it sprang from the grave this morning!

So, to quote the reply I just sent to Cory in django-developers:


I don't think ASGI would be a suitable replacement for WSGI in the current
form; in particular, I suspect it will have a performance disadvantage,
though I've not quantified yet.

That said, if Django Channels does become the primary method for
communication with web clients, the only thing we would want out of WSGI 2
would be something that could easily plug into ASGI - that is, something
that supports WebSockets, can handle simultaneous connections with many
clients in the same thread via cooperative multitasking or similar, and
allows raw access to sockets. Given those things, an ASGI HTTP/WebSocket
protocol server could simply be a WSGI 2 application, allowing much better
code reuse.


In particular, ASGI is never going to be low-level enough for people who
want to do crazy things; there's no access to input and output socket
streams, for starters. While I've tried to maintain some of the WSGI
familiarity, the whole thing is one step higher a level of abstraction, and
with that brings inflexibility and slight performance decreases that I
suspect people will get mad about if it becomes WSGI 2.

Now, I do think that the majority of web frameworks out there right now
could be ported to ASGI in no time at all; the asgiref package I'm working
on will ship with an ASGI-to-WSGI wrapper that just makes that work, as
long as the WSGI application doesn't try and be too clever with output

I don't want to express an opinion on if this means WSGI 2 is unnecessary
since I'm pretty biased; we need _some_ sort of standard that covers
WebSockets going forward, or we have the uwsgi problem where everyone
invents their own API for it.

My wishlist, though, is basically:

- WebSocket support
- Concurrent client handling support
- WebSockets and HTTP mixed-mode (i.e. on the same port and URLs)
- Fix the bytes/unicode issue that's all over the place in the current spec

These would all work super well in making ASGI itself pluggable into WSGI 2.

HTTP/2 features also need some place (like server push), and there's other
things cropping up like WebRTC that we might need to eventually support.
This is one of the reasons ASGI separates transport from protocol encoding;
I know people will invent new crazy web stuff in the next few years, and I
wanted a pattern that could extend to support most protocol types without
changing the abstraction completely.

Andrew
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: 
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com