Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec
On 11 March 2016 at 10:34, Andrew Godwin <and...@aeracode.org> wrote: >> >> >> I realise this may sound bikesheddy, but it would be really good to >> not call it ASGI. From your docs " >> Despite the name of the proposal, ASGI does not specify or design to >> any specific in-process async solution, such as asyncio, twisted, or >> gevent. Instead, the receive_many function can be switched between >> nonblocking or synchronous. This approach allows applications to >> choose what’s best for their current runtime environment; further >> improvements may provide extensions where cooperative versions of >> receive_many are provided." >> >> I'm worried that folk will assume a parallel between ASGI and asyncio, >> but there appears to be none... which is only a problem due to the >> room for confusion. > > > Better names are welcome, but I quite like ASGI's similarity to WSGI, and > the fact it's pronounceable as a single word. The "Asynchronous" part covers > the way the whole system operates; async is already an overloaded term, and > while there might be initial confusion, I think "async" also has strong > associations with the sort of problems ASGI solves (like websockets), which > I think is useful. Perhaps thats a particularly browser-centric view? There's nothing that strongly associates TCP with Python's slant on 'async' for me - interfaces on top of message passing can be sync or async - as in fact the switch you've got demonstrates :). Other names? quick thoughts... WSGP (web services gateway protocol) MuPGI (multiple protocol gateway interface) >> For consistency, why not a dict unicode -> List[bytes] > > I personally think this is worse than a list of tuples (which you can at > least feed straight into dict()) - the only header that comes through as > multiple, ever, is Set-Cookie, after all. I think you're wrong about that 'only header' statement. rfc 7230 3.2.2 permits multiple header fields with the same field name for all field values defined as comma separated lists, and for set-cookie. So you can't feed it straight into dict, unless you place a requirement on the server to always fold together multiple header fields with the same field name and clients to not use that either. Oh, and special case Set-cookie. -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec
On 10 March 2016 at 13:34, Andrew Godwin <and...@aeracode.org> wrote: > Hi all, > > As some of you may know, I've been working over the past few months to bring > native WebSocket support to Django, via a project codenamed "Django > Channels" - this is mostly the reason I've been involved in recent WSGI > discussions. > > I'm personally of the opinion that WSGI works well for HTTP, with a few > improvements we can roll into a 1.1, but that we also need something else > that can support WebSockets and other future web protocols (e.g. WebRTC > components). > > To that end, I did some work to make the underlying mechanism Django > Channels uses into more of a standard, which I have codenamed ASGI; while > initially I intended for it to be a Django documented API, as I've gone > further with the project I've come to believe it could be useful to the > Python community at large. I realise this may sound bikesheddy, but it would be really good to not call it ASGI. From your docs " Despite the name of the proposal, ASGI does not specify or design to any specific in-process async solution, such as asyncio, twisted, or gevent. Instead, the receive_many function can be switched between nonblocking or synchronous. This approach allows applications to choose what’s best for their current runtime environment; further improvements may provide extensions where cooperative versions of receive_many are provided." I'm worried that folk will assume a parallel between ASGI and asyncio, but there appears to be none... which is only a problem due to the room for confusion. -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Inviting feedback on my proposed "ASGI" spec
On 11 March 2016 at 08:32, Andrew Godwin <and...@aeracode.org> wrote: > > > > > Well, the protocol server would be the thing that's doing the joining if it > sees multiple headers - you'd always see comma-joined headers from clients > as an ASGI application, which I like as I like consistency. For consistency, why not a dict unicode -> List[bytes] ? -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
On 21 January 2016 at 11:27, Benoit Chesneau <bchesn...@gmail.com> wrote: > again. any server can do such implementation if we create a new Resource > abstraction. This abstraction would expose a common api to read and write. > The implementation would be specific to the server. > > Now like we have wsgi.thread I would instead suggest to add a system of > capability or extension like in smtp, imap, ... so the servers that > implement a specific extension can legally published it. Would it work for > you? I don't understand this 'any server can do it' thing. nginx for example, has no actual socket per connection for the uwsgi backends: the data is multiplexed over a dedicated uwsgi framing layer, and the server is responsible for HTTP framing itself. Sure, Python servers that have done listen() and accept() do have a real socket to expose, but thats a small subset of the implementations of WSGI> -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
e guts of HTTP/2 framing to an application. A way of doing Trailers would make sense. > Bytes and Unicode > ~ > > Several contributors expressed dissatisfaction for PEP-'s approach to > headers (namely, Latin-1-encoded Unicode strings), and expressed a preference > for using bytestrings. If we attempt this change, we have a > backward-compatibility concern, so we may have to live with this decision. > > Regardless, this should be taken as a warning sign for any other > specification we attempt: more on that later. > > REQUEST_URI environ variable > > > Multiple contributors expressed an interest in bringing this environment > variable into WSGI directly, making it a required part of the environ > dictionary. An alternative name for this was RAW_URI. If its reasonable available to e.g. Apache modules, I could see doing this. That said, why have two? Why not require that URI be the 'RAW' URI? I don't see the benefit in having two separate variables. > Content Lengths > ~~~ > > We should clarify in the new specification that an application that reads > beyond the logical length of the request as given by CONTENT_LENGTH will have > its reads return immediately with the empty string. Servers are required to > police that logic. This is codifying existing practice, and would also make > CONTENT_LENGTH purely advisory. I don't understand this being a work item. PEP- already has all this in it: - e.g. "A server should return empty bytestrings from any attempt to read from an empty or exhausted input stream. " > Websockets and HTTP/2 > - > > Here, the consensus seemed to be that we should investigate whether we can > come up with WSGI-equivalent APIs for Websockets and HTTP/2. The goal here > would be to allow servers that are interested in offering richer APIs than > WSGI can provide would be able to do so. > > This is particularly valuable for HTTP/2, which can function totally fine > when used with WSGI but which can have much richer functionality made > available to the application via a different API if that application requires > it. > > As a SIG we should begin to consider what these APIs should look like. We > should also bear in mind the more advanced asynchronous interfaces (see > below), and ensure that we can easily proxy these APIs to the more advanced > ones, as we can with WSGI to ASGI. I don't see this belonging in a WSGI 1.1. In a '2.0' or a -NG or some other differentiated thing, sure. > Meta API > > > Graham suggested, along with these more advanced APIs, that we may want a > meta-API or API discovery API that would allow servers to target a single > entry point which then provides access to the callables for WSGI, the > Websockets API, and the HTTP/2 API. We should bear this idea in mind, but I > don't believe we should worry too much about whether we actually build it or > not at this time: it can definitely follow the prototyping work on the HTTP/2 > specific APIs. Yeah, I'm not clear on what Graham wants here. Its not a lack I've felt so far :). > Asynchronous Python > --- > > There was an overall positive reaction to introducing *something* that fills > the WSGI role for applications that use asynchronous Python frameworks. > > Exactly how this would look is unclear at this stage, and this should be > thought of as a long-term goal for this SIG. Amber Brown is clearly working > on an idea, and we also have Andrew's work on ASGI, either of which could be > used as a basis for a future PEP. Guido also had some commentary on this a year or so back, IIRC. -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Collating follow-up on the future of WSGI
On 20 January 2016 at 12:04, Benoit Chesneau <bchesn...@gmail.com> wrote: > > not at all. But I made the assumption that the wsgi server maintained a > thread directly or not where the python application is running . > > In any case there is some sort of wrapping done in the same thread/process > where the python application is running. And then nothing stop to give the > socket away to the application and tell to the server to stop to communicate > with it. What socket? Data could be being passed by shm, for instance. -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
I should also say - thanks for picking this up. I may have been a tad on the grumpy side on my prior mail - new years paging-in-of-everything-after-a-break. -Rob On 5 January 2016 at 01:27, Cory Benfield <c...@lukasa.co.uk> wrote: > All, > > **TL;DR: What do you believe WSGI 2.0 should and should not do? Should we do > it at all?** > > It’s a new year, and that means it’s time for another attempt to get WSGI 2.0 > off the ground. Many of you may remember that we attempted to do this last > year with Rob Collins leading the charge, but unfortunately personal > commitments made it impossible for Rob to keep pushing that attempt forward. > > Since then, the need for a revision of WSGI has become even more apparent. > Casual discussion on the web has indicated that application developers are > uncomfortable with the limitations of WSGI. These limitations are providing > an incentive for both application developers and server developers to take an > end-run around WSGI in an attempt to get a framework that is more suitable > for the modern web. A great example of the result of WSGI’s deficiencies is > Andrew Godwin’s channels work[0] for Django, which represents a paradigm > shift in application development that takes it far away from what WSGI is > today. > > For this reason, I think we need to try again to get WSGI 2.0 off the ground. > But I don’t believe we can do this without getting broad consensus from the > developer community that a revision to WSGI is needed, and without > understanding what developers need from a new revision of WSGI. This should > take into account the prior discussions we’d had on this thread: however, I’m > also going to actively solicit feedback from some of the more notable WSGI > implementers, to ensure that whatever comes out of this SIG is something that > they would actually use. > > This WG already had a list of requirements, which are as follows: > > - Support servers speaking HTTP/1.x, HTTP/2 and Websockets (potentially all > on a single port). > - Support graceful degradation for applications that can use HTTP/2 but still > support HTTP/1.x requests. > - Graceful incremental adoption path - no upgrade-all-components requirement > baked into the design. > - Support Python 2.7 and 3.x (where x is not yet discussed) > - Support the existing ecosystem of containers (such as mod_wsgi) with the > new API. We want a clean, fast and approachable API, and we want to ensure > that its no less friendly to work with than WSGI, for all that it will expose > much more functionality. > - Apps need to be able to tell what protocol is in use, and what optional > features are available. For instance, HTTP/2 PUSH PROMISE is an optional > feature that can be disabled by clients. Websockets needs to expose a socket > like object, and so on. > - Support websockets > - Support HTTP/2 > - Support HTTP/1.x (which may be just 'point at PEP-’.) > - Continue to support lightweight shims being built on top such as > https://github.com/Pylons/webob/blob/master/webob/request.py > > I believe that all of these requirements are up for grabs, and subject to > change and consensus discussion. In this thread, then, I’d like to hear from > people about these requirements and others. What do you believe WSGI 2.0 > should do? Just as importantly, what do you believe it should not do? What > prior art should we take into account? Should we bother revising WSGI at all, > or should we let the wider application ecosystem pursue its own solutions à > la Django's channels? Should we simply adopt Andrew Godwin’s ASGI draft[1] on > which channels is based and call *that* WSGI 2.0? > > Right now I want this to be very open. I’d like people to come up with a > broad statement listing what they believe should and should not be present in > WSGI. This first stage of the work is very general: I just want to get a > feeling for what the community believes is important. Once we’re done with > that, if the consensus is that this work is worth pursuing, I’ll come up with > an initial draft that we can start making concrete changes to. > > In the short term, I’m going to keep this consultation open for **at least > two weeks**: that is, I will not start working on an initial draft PEP until > at least the **18th of January**. If you believe there are application or > server developers that should be involved in this discussion, please reach > out to them and point them to this list. I personally have CC’d some people > that I believe need to be involved in the discussion, but please reach out to > others as well. > > I’d really love to come to the end of 2016 with a solid direction for the > future of web programming in Python. I’m
Re: [Web-SIG] WSGI 2.0 Round 2: requirements and call for interest
core. I recall attempts to move other big frameworks around years ago now, taking a long long time I think the question of 'how can we enable interop of HTTP applications in asyncio' is best handled by a dedicated effort analogous to WSGI but in that programming model. -Rob -- Robert Collins <rbtcoll...@hpe.com> Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI and asyncio (tulip)?
On 15 October 2014 08:41, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 15 Oct 2014 08:40:05 +1300 Robert Collins robe...@robertcollins.net wrote: On 15 October 2014 07:30, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 14 Oct 2014 09:47:35 -0700 Guido van Rossum gu...@python.org wrote: I'm wondering if a small extension to the WSGI protocol might be sufficient to support this: the special environ variable wsgi.async_input could optionally be tied to a standard asyncio stream reader ( https://docs.python.org/3/library/asyncio-stream.html#streamreader), from which bytes can be read using yield from stream.read([nbytes]) or yield from stream.readline(). I think it would be frankly better to hook at the transport/protocol level, and let people wrap that inside an asyncio stream if that's their preference. For things like mod_wsgi and uwsgi, we're not actually implementing the transport or protocol inside of Python at all - its all happening in C and often in an entirely separate process. You may have misunderstood me. I am talking about the Transport and Protocol abstractions defined in PEP 3156. Lets assume I did. Given say nginx + uwsgi + asyncio, you're proposing that there be a uwsgi-asyncio module that listens to the uwsgi socket and demuxes packets from that into something that then exposes a ReadTransport + WriteTransport pair and a Protocol on top of that. That Protocol would have a 1:1 correspondence with a WSGI request, and would *not* be HTTP itself but rather the subset that is exposed via uwsgi? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI and asyncio (tulip)?
On 15 October 2014 10:04, Antoine Pitrou solip...@pitrou.net wrote: ... (that's for the HTTP part; a websockets layer would probably implement a separate transport and accept a separate protocol factory; actually, it could be implemented as a protocol that would parse the websockets protocol and provide its own transport on top of that... there may already be such a thing on the Internet :-)) So thats the bit that I'm having conceptual trouble with - servers may well implement the framing (and I rather think that they have to do so in some deployments), so we need to make sure that we don't do that in this context - we're behind an abstraction. With the HTTP example you gave, it looks fine, and I'm sure that equivalents can be made for websockets etc too. The question in mind is whether that is amenable to the same unification and layering that WSGI brought to the HTTP synchronous case. It looks like you'd make a protocol factory decorator. Will it be sufficiently flexible and interoperable to be something that becomes a lingua franca? I don't know pep-3156 well enough to judge that myself. Seems to me that there are two broad directions here: we can have a WSGI-thing where it looks just a little different to WSGI, or we can have a pep-3156 Protocol interface. We can share a bunch of logic either way - e.g. CONTENT_LENGTH etc, but the mechanics of writing middleware might be quite different. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI and asyncio (tulip)?
On 15 October 2014 11:10, Antoine Pitrou solip...@pitrou.net wrote: Each time a connection is accepted, protocol_factory is called without arguments(**) to create a Protocol, a bidirectional stream Transport is created to represent the network side of the connection, and the two are tied together by calling protocol.connection_made(transport). So where would headers etc be supplied to the protocol for reads (and for outputs)? Since the transport isn't the raw socket, its the bodies only. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On 13 October 2014 16:59, PJ Eby p...@telecommunity.com wrote: On Sun, Oct 12, 2014 at 6:47 PM, Robert Collins robe...@robertcollins.net wrote: FWIW I'm totally fine with you bringing together that PEP - as you say its complementary to what I'm focused on (I believe I even suggested you might want to do that). Did you have any feedback on the proposal itself? I'm particularly counting on you to tell me if I've horribly misunderstood something important about the use cases or the requirements for the protocols themselves. Not yet. Really just got back to stuff today. Rather than digging a hole for myself by commenting until I've absorbed it, let me do that and then I'll comment. :) I think that the upgrade model I've presented will enable you to happily design completely new API paradigms without having to figure out how to tunnel them through a maze of WSGI middleware, with the exception of having reasonable ways to present the incoming request as a WSGI request. But if I've missed something there, please let me know. Sure will. As I said earlier on in our thread, I'm not convinced that presenting new things as WSGI1 requests makes sense. I understand your arguments about adoption, but as I understand it WSGI itself started with nothing implementing it, and yet its now a very common lingua franca. So - I'd like to defer thinking too hard about the migration path, other than ensuring that its possible - and your draft may well be instrumental in some of the conversion paths needed. More once I've absorbed it. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On 13 October 2014 17:12, Benoit Chesneau bchesn...@gmail.com wrote: ... OK, So I should probably know you, but I can't recollect right now what you do or write. Its not clear to me who you were replying to. If Graham - Graham is the mod_wsgi maintainer, which I'm sure you've heard of - he, like you is one of the actors we need engaged and behind this effort. And PJE was the original WSGI maintainer :). Anyway I find it really disturbing the way you're actually acting and try to push your ideas based on private feedback coming from unknown or choosing who should be a reference. That certainly not the right way to have all actors on the table. Because if we go for a new WSGI spec, you certainly want it. And I am speaking as one of these actors. As I said when folk talked about going private in the first thread on this, I'm willing to discuss anything publically or privately - I can't tell folk where they will be comfortable discussing things. But I'm going to do *my* work on this in public, because I think that is essential to get broad consensus. In my opinion, if we want to go further we should first define what are the problem we want to solve, and then get the feedback from all the actors around: I think I've been fairly clear about the problem *I* want to solve. We want to create a clean common API for applications and middleware written in a post HTTP/2 world - where single servers may accept up to all three of HTTP/1.x, HTTP/2 and Websocket connections, and applications and middleware want to be able to take advantage of HTTP/2 and websockets when available, but also degrade gracefully. We also want to ensure that there is a graceful incremental path to adoption of the new API, including Python 2.7 support, and shims to enable existing WSGI apps/middleware/servers to respectively be contained, contain-or-be-contained and contain, things written to this new API. We want a clean, fast and approachable API, and we want to ensure that its no less friendly to work with than WSGI, for all that it will expose much more functionality. - framweorks authors I reached out to a number of such authors directly. I encourage you to do the same. - libraries author Ditto and - server authors Ditto :). If you don't have all actors around and majors are missing, there is probably no point to continue. I do think the idea of having a repository to collect it with people arbitrating the discussions on them on the mailing is a good way to go further. Now I think we are still missing of a clear definition of the problem. This is from what we should start instead of starting by giving our philosophy on how the problem should be solved. Here's my definition of some of the problems: A - there is no common spec equivalent to WSGI that permits writing server side code that takes advantage of HTTP/2. There's *a* http/2 server out there which one can write code for, but that code is either specific to that servers plumbing, or plain WSGI and misses the HTTP/2 goodness. B - WSGI has some oddness and overheads due in large part to the situation it was aiming to fix (which it broadly did) that perhaps we can now come together to fix. C - Support for chunked uploads, comet, bosh and websockets is effectively impossible within WSGI - one ends up writing server specific code, and being tied to a single server - even though multiple servers support (some of) those things. This defeats the point of WSGI IMNSHO: its not that WSGI is broken or anything, its just that we're once again writing all our generic middleware in server-specific fashions. Because the world has moved on and we haven't. I think A and C are crucial if we want to re-instate a lingua franca for the current web, in Python. I'd like to address B, because we can. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] handling different network protocols
One of the issues raised on the github repo was about upgrading to new protocols. Digging into that I think there is a splinter question: how *should* we represent the different network protocols in our python protocol. I've put some thoughts together about this in https://github.com/python-web-sig/wsgi-ng/issues/10 I rather suspect that any answer we have will make some folk unhappy, so I'd like to measure the answers against the baseline concerns: how much code will folk implementing the python protocol(s) have to write- lets minimise boilerplate and checks that the need to remember to put in. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On 14 October 2014 01:18, Benoit Chesneau bchesn...@gmail.com wrote: C - Support for chunked uploads, comet, bosh and websockets is effectively impossible within WSGI - one ends up writing server specific code, and being tied to a single server - even though multiple servers support (some of) those things. This defeats the point of WSGI IMNSHO: its not that WSGI is broken or anything, its just that we're once again writing all our generic middleware in server-specific fashions. Because the world has moved on and we haven't. Chunkedn upload is possible and already handled with Gunicorn. But there is no standard for that. Right. Thus we need one. For C I would separate it from the rest. This a different discussion and imo not everything can be achieved at the same time. Maybe we should start first by fixing them, then go for the next step anyway. So the transition could be incremental in servers and frameworks and actually fix the current spec. What makes C a different discussion? For A (And C), i think we should keep the new specification enough agnostic. Especially since HTTP 2 is not yet completely out. HTTP/2 is in last call stage: it will be entirely finished by the time we get through whatever process we have here. What do you want to see changed in the process I'm following? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] REMOTE_ADDR and proxys
On 30 September 2014 11:47, Alan Kennedy a...@xhaus.com wrote: [Robert] So it sounds like it should be the responsibility of a middleware to renormalize the environment? In order for that to be the case, you have strictly define what normalization means. For a given deployment its well defined. I agree that in general its not. I believe that it is not possible to fully specify normalization, and that any attempt to do so is futile. If you want to attempt it for the specific scenarios that your particular application has to deal with, then by all means code your version of normalization into your application. Or write some middleware to do it. But trying to make normalization a part of a WSGI-style specification is impossible. I don't recall proposing that it should be in a WSGI-style spec. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On 11 October 2014 08:10, PJ Eby p...@telecommunity.com wrote: On Fri, Oct 10, 2014 at 8:56 AM, Graham Dumpleton graham.dumple...@gmail.com wrote: So PJE, please step back and do not go rushing out to create a PEP. That is the worst thing you could do at this point and will only serve to deter people from the community contributing and so stifle proper discussion about this whole topic. Huh? Have you *read* the PEP? The entire point of it is to provide a basis for *experimenting* with new standards, not to stifle discussion of them. It's not even an *API*, for heavens' sake. It's just a description of how to upgrade to new standards from within existing WSGI frameworks, without needing to tunnel responses and without breaking subrequest middleware. FWIW I'm totally fine with you bringing together that PEP - as you say its complementary to what I'm focused on (I believe I even suggested you might want to do that). -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On 11 October 2014 01:56, Graham Dumpleton graham.dumple...@gmail.com wrote: I've pushed up https://github.com/python-web-sig/wsgi-ng/commit/df51d7d6fd4faa4efbe397fda2c323932f967020 which hopefully addresses the process and clarity concerns you expressed. (If not please help me tweak it appropriately). -Rob ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Draft 2: WSGI Response Upgrade Bridging
On 11 October 2014 01:56, Graham Dumpleton graham.dumple...@gmail.com wrote: ... Robert. What I would suggest you do is reboot this whole effort. Go back and perhaps look at how the github repo you setup is structured and make it more obvious how anyone can add their work into it in separate areas of it as need be and not just as issues, if that isn't already clear enough. Document exactly what you want people to do as far as adding anything there. Find people who will work with you on making all this clearer and defining any process. The next step is to make a more definite statement about the timeline for this whole discussion. Thanks for the process critique - I agree. I will put together such improvements in a little bit. I hadn't actually intended to go quiet - my intended next step was to collate the feedback we've had so far (and prompt you for some mod_wsgi orientated feedback). However, about 2 weeks back my Mum died, and that caused a rather big speed bump in the 'what I need to do' chore list, which still isn't over (but at least the crisis wise aspects are) They may want to comment on what should even be addressed in any revisions or extensions to the WSGI specification. In other words, don't limit this to just HTTP/2 and web sockets support. Allow people to raise their pet peeves about the existing WSGI specification so we can perhaps properly address them this time. The whole ASYNC issue with existing WSGI applications also should not be ruled out of scope as far as the comment period. Finally and hopefully, rather than people just complaining about things or giving wish lists, they will present properly fleshed out ideas for how to concretely solve ideas around ASYNC, HTTP/2 and web sockets. TBH I'd be fine with complaints and wish lists - got to start somewhere, and having a clear list of the places WSGI has not met needs would be excellent. One thing you could do, if you like, is to put a PR together for the wsgi-ng repo that adjusts README in the light of your feedback. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Combine wsgi and asyncio - possible?
On 30 September 2014 02:26, Frank Millman fr...@chagford.com wrote: Hi all I am developing a business/accounting application. It is not a web server in the conventional sense, but it uses http, and clients connect to it via a web browser. The server responds to an initial connection by sending a block of javascript which uses on_load() to display a welcome page. After that, all communication is handled by ajax-style messages passed between server and client. At no point is a new page requested or reloaded. Pages are browser constructs :) - I presume you're still speaking HTTP/1.1, with each request and response JSON - so a typical HTTP API implementation? ... Sometimes one user action generates more than one 'event'. I package these up into a list and send them to the server in one message. As the server processes the events, it can result in multiple responses to the client. These are also packaged up and sent in one message. One orthogonal thought here - HTTP/2 and websockets are [differently but relatedly] aimed at solving this in perhaps a cleaner way. HTTP/2 allows you to very sanely (unlike pipelining which had lots of problems) send different messages without waiting for the response, on the same connection. So you don't need to package up the events, just submit them, and you get separate events on the server side, without latency overheads. [There is some encoding overhead of course, but on a TCP stream with its window opened, you shouldn't notice that!]. There are websocket APIs around today that e.g. uwsgi offer. However, it can happen that while the server is working through the events, it needs to send a message to the client to pop up a dialog box, ask a question, and get a response before proceeding. With asyncio I can create a Future to set up and send the message, and use 'yield from asyncio.wait_for(...)' to wait for the response. Straight WSGI can in principle do this, but I suspect that browser APIs won't play nice. Here's how it would work: - make sure your ajax request is chunked, so that we can stream the request up. Don't close the request stream until you've received the full answer to all of your events. Stream the response back from your WSGI app by using yield rather than write() / returning a list. On the client, make sure you can handle multiple JSON documents within the one response without fully buffering it. You'll almost certainly need to be using SSL to avoid hitting a buffering intermediary (things ilke virus scanners etc). Assuming all the intermediaries are well behaved, nothing will buffer either the request or the response, and you'll have bidirectional communication happening. Doing it without browser API support is also possible, but it requires some mind bending - its basically what you're doing in asyncio: you reply from the first context, let the clients answer come back in on a new context, and then handoff from the new context to the existing old one to complete the data. I'm quite sure the asyncio code will be much cleaner for this situation. I am using asyncio.start_server(). For each request, the handler is passed a client_reader and a client_writer. Normally the writer is used to write the response to the original request, but if I need to ask a question, I use the writer to send the 'dialog box' message. When I get the response, I take the new client_writer and pass it back to the original request handler for it to complete the request. As I understand it, wsgi requires me to actually 'return' the response, so I don't have the opportunity to call 'yield from', and I do not get access to the writer object. Any suggestions welcome. HTH, Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI2: write callable?
It occurs to me that we're deep into one of Joey Hess's email-thread anti-patterns, so I'm going to leave this here for now. As I've said, I think the next step forward is to do some experimentation, which I'm sure the existing implementors that have expressed interest in this effort will join me in, and we'll get some indication together about how well [or otherwise] the basic things work. Concurrently, the IETF HTTP wg is now discussing websocket over HTTP/2, which will provide more data points for the API capabilities we'll need. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI2: write callable?
On 29 September 2014 08:32, PJ Eby p...@telecommunity.com wrote: On Sat, Sep 27, 2014 at 5:38 PM, Robert Collins robe...@robertcollins.net wrote: I think we're uncovering important assumptions / facts here. Indeed! For clarity: I'm not interested in a nice API for HTTP/2. I want HTTP/2 and its full featureset to be *possible*, *efficient* and *clear* in a protocol that can replace WSGI - and do so with a fair chance of adoption. Cool. Then my suggestion would be: don't use WSGI as a basis for designing that protocol. Start with something that's a natural fit for the HTTP/2 model, which -- from what I can tell so far -- is nothing like WSGI's simple request/response model. Thats a fair point. I have not been constrained by WSGI today in thinking about this - but since this effort is about updating the standard folk write to, for web server - gateway - app plumbing in Python, WSGI is, for better or worse, the touchstone folk have. WSGI's simple request/response model has been unable to fully handle the modern web since HTTP/1.1 came out (AFAIK none of the gateways have managed to make chunked uploads work right, and trailers are not supported). Thats not a bad thing about WSGI, and putting the union of requirements into a spec can make it unwieldy (see RFC2616 for a classic example :)) - but while we have a lot of frameworks that are a) composed of WSGI adapters and b) WSGI on the top, or WSGI on the bottom, they're not following WSGI all that precisely, because WSGI is too restrictive. And the web has moved on with Websockets in 2011 and HTTP/2 any day now. I think its clear from the broad interest we've got that folk are interested in a new spec. Whats in a name? We could call it something else. RSGI as you humourously suggest, or we can call it WSGI. I think WSGI is the right name, because I don't think we want to aim for a situation where folk writing new servers write both WSGI and $NEWTHING support. They should be able to pick one, write to it well, and have their users choose to downgrade the environment if they have legacy things that are not yet upgraded. So - I'm going to keep drafting this as WSGI2, unless there is consensus here that the name should be different. Right. I do think it might be worthwhile creating a spec for how to create safe middleware-bypassing and rich object server extensions within WSGI, to allow limited use of HTTP/2 features. That might be an interesting thing; I have no real interest in writing it at this point: my intent is to provide a new thing, which may be very similar, or may be strikingly different - thats what this SIG will come up with - which can contain WSGI1 middleware safely via some adapter. I don't have interest in writing a 'do HTTP/2 features from within WSGI1' effort, because I think its a lot of work for little-if-any-gain: we have to have servers that can speak the new wire protocols before we can use the new features, and that means the top of our stack will be $NEWTHING anyway. There are some exceptions, such as the mod_spdy hack to tunnel awareness of to-push resources, but its not clear that that will do the right thing in all circumstances with oblivious middleware, no matter how its spelt in code. [Because, whatever failure mode we choose by default, some middleware will want the other one - at which point its not oblivious, and may as well just be upgraded]. * This almost certainly applies to WSGI as well: WSGI2 - WSGI1 - WSGI2 will have to downgrade to WSGI1. Some things may be tunnelable [and we can try to do that], but the full set of features almost certainly cannot. That depends on what you mean by WSGI2. I think an HTTP/2 gateway API is a different animal than WSGI2 per se. I think there may be room for a request/response WSGI2, distinct from a Python HTTP/2 API, and (mostly) interoperable with WSGI 1. That doesn't mean that the HTTP/2 API might not win over the market and supplant WSGI1/2, I'm just not convinced that it should be positioned as WSGI's successor. (At least, not until I've seen it... ;-) ) Thats fair enough, but in the absence of a better name - and see above: having the need for server and middleware authors to only need to care about one protocol is a key design point - I think calling it WSGI2 is better than calling it something new. If its going to make the discussion hard, I'm ok calling it e.g. NNGI (no name gateway interface) until we're done. From this I drew the proposal to do interop by providing an API [not protocol] that provides WSGI1 on the top and 2 on the bottom, and another that does the reverse: allowing folk to upgrade individual middleware piecemeal, and get the full benefits whenever they have a fully upgraded stack. E.g. leave upgrading debug middleware to the end. Perhaps this is misguided and implementors will reject such assistance? My suggestion would be to make a good HTTP/2 API without any WSGI legacy, and then develop a set
Re: [Web-SIG] WSGI2: write callable?
I think we're uncovering important assumptions / facts here. For clarity: I'm not interested in a nice API for HTTP/2. I want HTTP/2 and its full featureset to be *possible*, *efficient* and *clear* in a protocol that can replace WSGI - and do so with a fair chance of adoption. Ditto websockets. Neither is possible within WSGI today: the base protocol is insufficient, and every implementation of either HTTP/2 or Websockets for app writers only works by depending on extensions that don't meet the basic design principles - for instance exposing the actual server socket as an extension, which mod_wsgi cannot do. So, basic axioms I've been working from: * HTTP/2 cannot be tunnelled through HTTP/1: it can be downgraded, but not tunnelled. An HTTP/2-HTTP1.1-HTTP/2 chain is not capable of the same results as a straight HTTP/2 connection (or chain). * This almost certainly applies to WSGI as well: WSGI2 - WSGI1 - WSGI2 will have to downgrade to WSGI1. Some things may be tunnelable [and we can try to do that], but the full set of features almost certainly cannot. From this I drew the proposal to do interop by providing an API [not protocol] that provides WSGI1 on the top and 2 on the bottom, and another that does the reverse: allowing folk to upgrade individual middleware piecemeal, and get the full benefits whenever they have a fully upgraded stack. E.g. leave upgrading debug middleware to the end. Perhaps this is misguided and implementors will reject such assistance? On 28 September 2014 07:55, PJ Eby p...@telecommunity.com wrote: On Sat, Sep 27, 2014 at 12:20 AM, Robert Collins robe...@robertcollins.net wrote: We should capture these design principles somewhere FAQ-like, since many of the folk participating in this rework weren't part of the original design. A lot of it is in the PEP itself, albeit in ways that seem a lot more obscure now, 10 years later, than they did at the time of writing. It's also spread out among different parts, including the FAQ at the end. :) - I am familiar with PEP, so yeah, does feel a bit obscure :). Thank you for chiming in to reinforce them. Any feature which is added solely to entice an end-consumer of WSGI (vs. a framework or library implementer) is 100% wasted. I understand that argument, but... ... If WSGI 2 adds features that users want, and library/framework developers can reasonably add those features to *their* APIs, then there is a chance that they will do so. But if they have to throw out their whole existing paradigm to do that, or users have to abandon their framework to adopt the WSGI 2 paradigm, then nothing was really gained by the effort. libraries and frameworks exist for the same users. WSGI's ability to say 'and this is up to library/framework developers' is contingent on the protocol being *sufficient* for folk to do that. I suspect a bunch of our discussions are going to end up being around whether specific changes are necessary or things libraries can do. Basically, going after end users puts you in a boil the ocean position. That is, a situation where you must more or less convince everybody to change at the same time in order for the standard to reach critical mass. I had hoped not, due to proposing that we provide an API [not protocol] for adapting between the protocols. That would exist solely to make implementors have an easier time bringing support in incrementally. So - I think you're misinterpreting my thrust as being 'after end users' - I'm not: I'm squarely focused on the implementation problems of server and middleware authors. However, if you are *not* trying to boil the ocean by attracting end users, then anything that you do to benefit them (at the expense of framework, middleware, or server authors) is pure waste, since the incremental strategy (that WSGI was based on in the first place) doesn't depend on end-users using the raw WSGI protocol. As the PEP itself explains: ... If you replace WSGI with WSGI 2 in the above, the rationale remains unchanged. Sure. The above API is cute and clean for the app writer, but for a middleware writer it's a barrel of misery. *Every* piece of middleware that even wants to *read* anything from the response (let alone modify it), now needs to check types of yielded values, accumulate headers, and maybe buffer content. And there are many ways to write that middleware that will be wrong, but *appear* right because the author didn't think of all the ways that an app could violate the middleware author's assumptions. Hang on, why would they buffer content? Buffering response content is currently verboten, and I haven't seen any proposal to change that. I don't understand how phrasing the API as I suggested would lead to buffering being permitted or required. By content I was actually talking about the headers or other metadata. Sorry for the confusion. No worries. Right now buffering of headers is required - the whole 'until the iterator returns
Re: [Web-SIG] WSGI2: write callable?
On 27 September 2014 08:21, Benoit Chesneau bchesn...@gmail.com wrote: On Fri, Sep 26, 2014 at 5:32 AM, Robert Collins robe...@robertcollins.net wrote: ... So I propose we drop the write callable, and include a queue based implementation in the adapter for PEP- code. -Rob What would be the advantage of using a queue compared to simply write to the server? Internally the server can use queue, but why the client should know it? What is the reasoning behind it? The point is to remove the complexity of having both an iterator over content *and* a write method. Thats really complex for server [and middleware] writers. So the interface to send bytes to the container would just be 'yield them'. (Or return a fully populated list). So the point about the Queue is that to support PEP- we either need to retain the write() callable, or we need an adapter that can expose on its upper side the iterator we want, and on the lower side accept *either* an iterator *or* use of write() method - I think you'll find thats quite hard to write without a Queue or similar construct. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI2: write callable?
On 27 September 2014 10:31, PJ Eby p...@telecommunity.com wrote:
On Fri, Sep 26, 2014 at 5:02 PM, Robert Collins
robe...@robertcollins.net wrote:
But perhaps it would be nicer to say:
iterator of headers_dict_or_body_bytes
With the first item yielded having to be headers (or error thrown),and
the last item yielded may be a dict to emit trailers.
So:
def app(environ):
yield {':status': '200'}
yield b'hello world'
yield {'Foo': 'Bar'}
is an entirely valid, if trivial, app.
What do you think?
I think this would make it harder to write middleware, actually, and
for the same reason that I dislike folding status into the headers.
It's a case of flat is better than nested, I think, in both cases.
That is, if the status is always required, it's easier to validate its
presence in a 3-tuple than nested inside another data structure.
I'm intrigued here - validation of the status code is tied into into
the details of the headers. For instance, 301/302 need a Location
header to be valid. So I don't understand how its any easier with
status split out. I'd be delighted to whip up a few constrasting
middleware samples to let us compare and contrast.
Note too that folk can still return bad status codes with a different layout
(status, headers, body, trailers)
return None, {}, [], {}
One thing we could do with the status code in the headers dict is to
default to 200 - the vastly common case (in the same way that throwing
an error generates a 500). Then status wouldn't be required at all for
trivial uses. That would make things easier, no?
As
far as trailers go, I'm not sure what those are used for or how they'd
be used in practice, but my initial thought is that they should be
attached to the response body, analagous to how FileWrapper works.
So a classic example for Trailers is digitally signing streamed
content. Using the same strawman API as above:
def app(environ):
yield {':status': '200}
md5sum = md5.new()
for bytes in block_reader(open('foo', 'rb'), 65536):
md5sum.update(bytes)
yield bytes
digest = md5sum.hexdigest()
signature = sign_bytes(digest.encode('utf8'))
yield {'Content-MD5Sum': digest, 'X-Signature': signature}
Note that this doesn't need to buffer or use a closure.
Writing that with a callback for trailers (which is the only
alternative - its either a callback or a generator - because until the
body is fully handled the content of the trailers cannot be
determined):
def app(environ):
md5sum = md5.new()
def body():
for bytes in block_reader(open('foo', 'rb'), 65536):
md5sum.update(bytes)
yield bytes
def trailers():
digest = md5sum.hexdigest()
signature = sign_bytes(digest.encode('utf8'))
yield {'Content-MD5Sum': digest, 'X-Signature': signature}
return '200', {}, body, trailers
The other alternative is to use a dict as the response object
(analagous to environ as the request object), with named keys for
status, headers, trailers, body, etc. It would then be extensible to
handle things like the Associated content concept.
That might work, though it will force more closures. One of the things
I like about the generator style is the clarity in code that we can
achieve.
In this way, middleware that is simply passing things through
unchanged can do so, while middleware that is creating a new response
can discard the old object.
That seems to apply either way, right?
Here's a body-size logging middleware:
def logger(app):
def middleware(environ):
wrapped = app(environ)
yield next(wrapped)
body_bytes = 0
for maybe_body in wrapped:
if type(maybe_body) is bytes:
body_bytes += len(maybe_body)
yield maybe_body
logging.info(Saw %d bytes for %s % (body_bytes, environ['PATH_INFO']))
return middleware
..
We're bumping the WSGI version, will that serve as a sufficient flag?
I mean, flagged on the app end. For example, wsgi_lite marks apps
that support wsgi_lite with a true-valued `__wsgi_lite__` attribute.
In this way, a container invoking the app knows it can be called with
just an environ (and no start_response).
Ok, So we'd use the absence of such a mark to trigger the WSGI1
adapter automagically? I'm curious if that will work well enough we
are given wsgi_lite or other extensions to wsgi. Perhaps we should
refuse to guess and just supply the adapters and instructions?
So, I'm saying that an app callable would opt in to this new WSGI
version, so that servers and middleware don't need to grow new APIs
for registering apps -- they can auto-detect. Also, having
auto-detection means you can write a decorator (e.g. in wsgiref), to
wrap and convert WSGI 1 apps to WSGI 2, without needing to know if
you're passing something already wrapped. It means that a WSGI 2
server or middleware can just wrap whatever apps it sees, and get back
a WSGI 2 app, whether
Re: [Web-SIG] WSGI server handling absolute URI
I think this makes sense to address - have replied to the ticket. but tl;dr: inconsistency in this space is likely to provoke bugs, so lets make things consistent. -Rob On 12 May 2014 00:45, mouad ben mouad...@gmail.com wrote: Hello, My name is Mouad and this is my first time writing to this mailing list. I hope this is the right mailing list to let interested party to know about a minor bug that i found in some WSGI server that are out there, that doesn't support absolute URI in an raw http request i.e. GET http://domain.com/path HTTP/1.1 I have created an issue for WGIREF http://bugs.python.org/issue21472, and i am waiting for feedback from cPython core developers, and most **importantly** this gist https://gist.github.com/mouadino/7930974 that show who support this feature and who doesn't. HTH, -- Mouad Benchchaoui ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/robertc%40robertcollins.net -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Nodejs cluster
On 19 March 2014 22:02, Tobias Oberstein tobias.oberst...@tavendo.de wrote: We are working on a system (on top of Autobahn) which provides builtin scale-up (multi-core) and scale-out (multi-node) capabilities: https://github.com/crossbario/crossbar This is work in progress and relies on WAMPv2. Here are a couple of links ... Interesting. That looks very similar to Mongrel2. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] WSGI: start_response buffers headers
https://github.com/python-web-sig/wsgi-ng/issues/4 So doing bidirectional streaming is somewhat incompatible with requiring that headers be buffered until actual server side content is available (what if the client is meant to write first... but is itself waiting for the stream to be established). I think making an empty bytestream flush the headers would be sufficient, and preserve much of the niceness. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] WSGI: allowing short reads
https://github.com/python-web-sig/wsgi-ng/issues/5 tl;dr - we don't specify whether read(size) has to return size bytes or just not more than size, today. the IO library is clear that read(n) returns up to n, and also offers read1 that guarantees only one read call. I don't think we need read1 (perhaps I'm wrong) but making read consistent with the io library would be good, I think - particularly for websockets. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
On 22 September 2014 00:08, Robert Collins robe...@robertcollins.net wrote:
On 21 September 2014 16:43, Roberto De Ioris robe...@unbit.it wrote:
I've proposed using github issues instead of documents; we can
synthesis the issues into prose in the draft docs and reference code
itself. I think this will be easier to manage than having a dozen
different comment-documents in the repo.
-Rob
I completely agree and i have already opened two 'issues'. If we change
idea on how to work on it feel free to delete them :)
Cool, thank you!
I've put my thoughts up in them, and pulled out what I think are
clearly sane requirements from them into a nascent requirements.rst
file.
I haven't closed the issues, since the actual spec covering those
requirements doesn't exist. And that leads to what is I think a fairly
key question.
Do we:
- incorporate PEP- by reference [e.g. by saying 'any HTTP/1.{0,1}
request will be processed as per PEP-']
or
- do we want to alter how HTTP/1.{0,1} requests are presented (e.g.
tackling encoding of headers etc)
Timing the question out: I'm going with the latter case: a clean new
spec with consistent handling of feature that are common to all the
supported protocols, and folk that want existing things to keep
running wrap them with an adapter we'll provide.
-Rob
--
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud
___
Web-SIG mailing list
Web-SIG@python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe:
https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
it (horizon, the OpenStack APIs and lmirror), varied experience deploying WSGI and WSGI derived things, and writing code both inside frameworks like Django and Zope3, as well as straight to WSGI. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
On 20 September 2014 19:14, Benoit Chesneau bchesn...@gmail.com wrote: Hi, I would prefer to have this work being done transparently. If we do it rationally it could work imo. Anyway before thinking to change the protocol or criticizing it maybe we could first collect the requirements in HTTP 2 (stream and such) so we can think about possible implementations. And see what it misses in WSGI. I am thinking we could adopt the same path used to decided to go for HTTP 1.x or HTTP 2 on the client part. Ie keeping WSGI and PEP for HTTP 1.1 applications and go for a new interface in HTTP2. But such decision should be done once we have a clear view of what requires HTTP 2 and how it can be handled on the python side. Thoughts? +1 on transparency. Agree that before we consider what we need to change we need to set our goals up - thats basically the charter for a PEP: Here is a straw man in prose form: We want to create a clean common API for applications and middleware written in a post HTTP/2 world - where single servers may accept up to all three of HTTP/1.x, HTTP/2 and Websocket connections, and applications and middleware want to be able to take advantage of HTTP/2 and websockets when available, but also degrade gracefully. We also want to ensure that there is a graceful incremental path to adoption of the new API, including Python 2.7 support, and shims to enable existing WSGI apps/middleware/servers to respectively be contained, contain-or-be-contained and contain, things written to this new API. We want a clean, fast and approachable API, and we want to ensure that its no less friendly to work with than WSGI, for all that it will expose much more functionality. -Rob ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] web-sig mailing list moderating every post?
I'm not sure of the right place to bring this up - I tried to on the web-sig list itself, but the moderator rejected the post. What I tried to post there was Looks like *every* post to web-sig gets manually moderated. That seems like it will make discussion rather hard: can we get that changed (or is there some historical need for it - if so, perhaps we should use python-dev or some other list) ? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] web-sig mailing list moderating every post?
Ugh - this was in my mailbox shortly after the moderator action email from mailman: No, this looks like the spam filter. Don't know what triggered it. Or why it went to you. But the list moderation is turned off (except for non-members posting to the list), and you yourself are not moderated, so... Bill - nothing to see here, move right along, and sorry for the noise. -Rob On 21 September 2014 10:19, Robert Collins robe...@robertcollins.net wrote: I'm not sure of the right place to bring this up - I tried to on the web-sig list itself, but the moderator rejected the post. What I tried to post there was Looks like *every* post to web-sig gets manually moderated. That seems like it will make discussion rather hard: can we get that changed (or is there some historical need for it - if so, perhaps we should use python-dev or some other list) ? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] WSGI for HTTP/2.0 ?
On 21 September 2014 06:15, Randy Syring ra...@thesyrings.us wrote: I'd suggest a wsgi comments github repo. So in the interests of getting things done and the spirit of EAFP I've set up https://github.com/python-web-sig/wsgi-ng. Since I have no deep history in web-sig, I'll happily hand out 'organisation admin' to someone (e.g. Bill) with such history - I'm not trying to land-grab the name, just to use something sensibly named. That said I'd like to keep the direct committers to that specific repository limited to whomever manages to end up collaborating well: I have a better understanding of the burnout issue thanks to the responses in this thread. Workflow: Submit a document to the repo with your comments on the future version of WSGI use any readable format you want (Markdown, RST, plain text, etc.). include name, contact information, background. Make sure to give enough info about your background so the draft team has some context for the proposals and comments you are making. I've proposed using github issues instead of documents; we can synthesis the issues into prose in the draft docs and reference code itself. I think this will be easier to manage than having a dozen different comment-documents in the repo. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] WSGI for HTTP/2.0 ?
So HTTP/2.0 (http://http2.github.io/http2-spec/index.html) is far advanced, and my puny google-fu cannot find any upstream work on making a) updating and or replacing WSGI to support HTTP/2's new capabilities or b) an HTTP/2 capable SimplerServer or similar reference server in the standard library . Huge apologies if I'm wrong and pointers accepted! I did find things like https://evonove.it/blog/2012/django-jetty-spdy-blazing-fast/ which uses Jython and the Jetty web server to do SPDY (the Google experiment that has formed much of the basis of HTTP/2) or https://github.com/tatsuhiro-t/nghttp2/blob/master/python/wsgi.py which doesn't expose any of the new HTTP/2 features. So, I'd like to kick of such work, I think the spec is sufficiently stable now that we can design APIs in Python for it with confidence, even though we may need to tweak things it won't be disruptive. Specific things that I think we need to cater for: - the streaming and multiplexing facilities (http://http2.github.io/http2-spec/index.html#rfc.section.5 and http://http2.github.io/http2-spec/index.html#FrameTypes) - this is a fairly fundamental departure from HTTP/1.x's strict 'request-response' model and exposing it should offer very nice capabilities to site authors. HTTP1.x requests look like a half-closed stream on an HTTP/2 connection, but its entirely possible via the extension mechanism to run bidirectional data on a stream initiated by either end (whiile the client has a single open stream the server can push a new associated stream at any point) - flow control (http://http2.github.io/http2-spec/index.html#fc-principles) - for file uploads for instance, we can now rate limit single clients directly within the protocol - the resource tree (http://http2.github.io/http2-spec/index.html#pri-depend) - if we have concurrent requests being handled for one client it is now possible to explicitly model which ones should be processed and put on the wire first, and this should flow up into the application to a degree - GOAWAY (http://http2.github.io/http2-spec/index.html#ConnectionErrorHandler) - backwards compat - making sure that straight PEP- apps still work well when the server connection is HTTP/2 Is anyone interested in collaborating on an update to WSGI to support HTTP/2's new features? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: https://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
