Re: [Web-SIG] Getting back to WSGI grass roots.
Hi, Graham Dumpleton schrieb: So, rather than throw away completely the idea of bytes everywhere, and rewrite the WSGI specification, we could instead say that the existing conceptual idea of WSGI 1.0 is still valid, and just build on top of it a translation interface to present that as unicode. I could live with that as well. Regards, Armin ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
On Wed, Sep 23, 2009 at 12:43 AM, Graham Dumpleton graham.dumple...@gmail.com wrote: ... Anyway, that is the thought. Should we be looking at WSGI as a set of layers instead of assuming we have to push unicode into the gateway interface layer? +1 Jim -- Jim Fulton ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
Graham wrote: So, rather than throw away completely the idea of bytes everywhere, and rewrite the WSGI specification, we could instead say that the existing conceptual idea of WSGI 1.0 is still valid, and just build on top of it a translation interface to present that as unicode. I don't think we really need to. Almost nothing in WSGI itself actually touches Unicode. HTTP headers may in theory be ISO-8859-1 (and certainly should be handled as such), but in the real world they are exclusively ASCII (anything else breaks browsers). SCRIPT_NAME/PATH_INFO is the only part of the spec that potentially needs more than ASCII, and even then the majority of apps don't put any Unicode characters in those (especially SCRIPT_NAME). I don't think it's worth adding the complication of a two-layer interface just for this one case. If we can hack around SCRIPT_NAME/PATH_INFO separately as per the other thread there's no longer any need for anything but ASCII, so WSGI's strings can be bytes or unicode depending on your taste/Python-version, without it hurting anyone. The important job of mapping * query parameters, * POSTed request bodies, and * response bodies between bytes and unicode remains firmly in the application/framework's area of concern and needs no assistance from WSGI. -- And Clover mailto:a...@doxdesk.com http://www.doxdesk.com/ ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 René Dudfield wrote: On Wed, Sep 23, 2009 at 1:03 PM, Aaron Watters arw1...@yahoo.com wrote: --- On Wed, 9/23/09, Graham Dumpleton graham.dumple...@gmail.com wrote: So, rather than throw away completely the idea of bytes everywhere, and rewrite the WSGI specification, we could instead say that the existing conceptual idea of WSGI 1.0 is still valid, and just build on top of it a translation interface to present that as unicode. Seconded. There should be a lower level that talks bytes and a higher level that talks unicode or whatever. There should also be a way for even higher levels to reach down to the lower level to see the bytes before they got misdecoded by the unicode layer because this will likely be needed in some cases. Is there anything wrong with just adding decoded interpretations to the WSGI environment as separate entries? Also, everything should be as orthogonal as possible. One problem I have with most Web tools and frameworks is they tend to take over and do everything at once when I really only want a little bit of help. WSGI 1 is nice because it just abstracts HTTP and stops there. It was a beautiful piece of work. Kudos. Yeah, wsgi is definitely useful for a wide range of uses cases. Except it's currently not working for a number of use cases... but we can't accommodate them. eg, unicode, tainting, http proxies, http clients, user supplied buffers, async, latest http RFCs, different uses of http compared to 2003, all features of http. This is clear by the variety of web frameworks that don't use wsgi, and the variety of things layered on top of wsgi. There's room for other specifications which consider those use cases not covered by wsgi. http proxies cover the main wsgi use case of being able to use applications on many servers(apache, nginx lighttpd etc) . Things like webob, and cherrypy allow python frameworks to coordinate at a higher level avoiding wsgi as well. It's also clear that async frameworks can be used with wsgi in a non blocking manner given things like the greenlets/stackless and the Eventlet library - which makes use of two underlying async frameworks(twisted, and libevent) given that all of your blocking calls to libraries can be swapped out with versions written for async... like many have already(eg you can use a urllib api like library). We have to keep remembering what the purpose of wsgi is. Opening line of wsgi spec: This document specifies a proposed standard interface between web servers and Python web applications or frameworks, to promote web application portability across a variety of web servers. By limiting its scope we do get something useful out of it(for some people). Application portability is the main wsgi use case. I think that requires a number of things that wsgi doesn't provide - wsgi knows nothing of data stores for example. Application portability is the main thing we should be interested in, and strive for it. Not just on web servers, but on web frameworks too. There's no way I can take any python web application, copy the files onto any python web server and have it work. php can do this, but we still can not do this with python. Well, I kinda disagree. You can easily distribute eggs or whatever Python packages to your favorite HTTP web server and have them working in your current WSGI stack. At least, I don't seem to get how PHP makes this more easily than with Python. Perhaps putting some lights here would be helpful. Please let web frameworks be web frameworks and peps be peps! That is, allow some flexibility for web frameworks designs by writting clear and concise peps, but also forget about fairy features that would increase the document size over 1MB... ;-) Regards, Etienne ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/robillard.etienne%40gmail.com - -- Etienne Robillard robillard.etie...@gmail.com Green Tea Hackers Club http://gthc.org/ Blog: http://gthc.org/blog/ PGP Fingerprint: 178A BF04 23F0 2BF5 535D 4A57 FD53 FD31 98DC 4E57 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkq6IZQACgkQ/VP9MZjcTldOcQCgyEsupfDxrSIpwVBK85iCuOCZ cwQAnRWov+VTQqT2T/4gw84hqnjkL7dG =moxq -END PGP SIGNATURE- ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
--- On Wed, 9/23/09, René Dudfield ren...@gmail.com wrote: Application portability is the main wsgi use case. I think that requires a number of things that wsgi doesn't provide - wsgi knows nothing of data stores for example. Application portability is the main thing we should be interested in, and strive for it. Not just on web servers, but on web frameworks too. Perhaps there should be a notion of managed application resources built as another layer on top of WSGI so you can easily switch between storing things in MySQL or the file system or Google App Engine data store or whatever. Perhaps something along these lines? http://aaron.oirt.rutgers.edu/myapp/docs/W1000_1000.resources There's no way I can take any python web application, copy the files onto any python web server and have it work. php can do this, but we still can not do this with python. I can :). This doesn't require changes to WSGI, however, just appropriate additional layers on top of WSGI which you can call WSGI++ or give another name -- I don't know which is better -- ask a marketing person. -- Aaron Watters http://aaron.oirt.rutgers.edu/myapp/docs/W1100_1400.calc === Little birds are playing Bagpipes on the shore Where the tourists snore Thanks! they say, 'Tis thrilling! Take, oh take this shilling! Let us have no more! -- Lewis Carroll ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
At 02:43 PM 9/23/2009 +1000, Graham Dumpleton wrote: Sorry, after having had a bit of think while eating lunch, I am going to throw up another point of view on this whole issue. So, sit back and be just a little bit concerned. WSGI stands for Web Server GATEWAY Interface. My understanding is that right back at the beginning WSGI was purely intended to only be used at the direct interface with the underlying web server. This is why I understand, in part at least, the term 'gateway is used in the acronym. This is incorrect. WSGI's roots are in an interface that was used inside of PEAK applications, as a way of connecting components, and allowing pieces from different frameworks to be combined in a single application, while also being able to run under CGI or FastCGI or a dedicated server. That interface was basically a CGI-like environ/stdin/stdout/stderr, represented as function call arguments. The original terminology in the December 2003 draft used the term container as a catchall, rather than distinguishing servers from middleware. The problem was that people discovered one could apply the same interface for use as middleware. As we all know, that has been used quite successfully, but has also been equally abused. People didn't discover it - the term first appeared in the second draft of the spec, and was part of the idea before I ever wrote the first posting to the Web-SIG; I just didn't use that word. By seeing WSGI as being layers instead, first thing is that web frameworks such as web2py and CherryPy which merely use WSGI as the gateway interface would continue to work directly on this layer, regardless of whether they use Python 2.X or 3.X. Those frameworks are already going to translate what ever this interface defines into their own internal interface and effectively relegate WSGI from any higher levels of the application. We now get back to the unicode vs bytes argument we have been having. This argument will not vanish by virtue of doing this, but instead of pushing the unicode translation down into the gateway interface layer, we just apply it on top. I don't understand what you mean by layers here. To avoid conflict, one could as a minimal measure just add an additional 'wsgi.' variable which indicates whether interface is 'bytes' or 'unicode' and hope middleware validate they have been plugged in at the correct level. Dear please God, no. Anyway, that is the thought. Should we be looking at WSGI as a set of layers instead of assuming we have to push unicode into the gateway interface layer? These are not mutually exclusive options. However, the set of layers thing, if I'm understanding it correctly, is a big fat -1000 -- totally invalidates the whole point of WSGI. Honestly, I don't even like having two versions of the spec, which is why the idea of having a 3.0 really ticks me off. Standards don't benefit from having multiple versions, even in disguised forms like layers or options. FWIW, I thought of this because I was going to suggest at this point that overall we have a break from the discussion at this point. I'm not sure I follow you. Ian has put forth a proposal that I heartily support, with the possible exception of a part that I've asked for clarification on. Others have expressed support for that proposal as well, and I haven't seen any -1's on it yet. Perhaps you should take a look at it? (It's under the Proposal to remove SCRIPT_NAME/PATH_INFO thread, but it's really a complete proposal for moving forward with a single new 2.0 spec.) ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
2009/9/24 P.J. Eby p...@telecommunity.com: Anyway, that is the thought. Should we be looking at WSGI as a set of layers instead of assuming we have to push unicode into the gateway interface layer? These are not mutually exclusive options. However, the set of layers thing, if I'm understanding it correctly, is a big fat -1000 -- totally invalidates the whole point of WSGI. Honestly, I don't even like having two versions of the spec, which is why the idea of having a 3.0 really ticks me off. Standards don't benefit from having multiple versions, even in disguised forms like layers or options. This isn't really about multiple versions of the specification which shows you do perhaps simply don't understand. With such a negative response, seems no point in trying to explain to you though. FWIW, I thought of this because I was going to suggest at this point that overall we have a break from the discussion at this point. I'm not sure I follow you. Ian has put forth a proposal that I heartily support, with the possible exception of a part that I've asked for clarification on. Others have expressed support for that proposal as well, and I haven't seen any -1's on it yet. Perhaps you should take a look at it? (It's under the Proposal to remove SCRIPT_NAME/PATH_INFO thread, but it's really a complete proposal for moving forward with a single new 2.0 spec.) I have read it. Right now there is a mere 10 posts in that discussion which is nothing compared to the scrutiny that other proposals have got. Other proposals have also had Armin and Robert trying out actual code to either implement them or investigate the practicality of them actually working. I have seen no suggestion that this has been done in regard to Ian's proposal, so it is theoretical only whether other options have at least been tried to an extent. Putting aside the technical merits or otherwise of Ian's suggestions, it falls at a time akin to how governments will introduce new legislation very late in a sitting (ie., 3am) in the morning, when most have lost interest or don't care any more. The governments do this because they know it will not get the scrutiny it should. Now, I am not saying that Ian has done that deliberately, just that it has been unfortunate timing in that some of the participants in the discussions seemed to have faded away and at this point most are possibly just seeing this as all part of the original discussion instead of something new and so either not commenting or simply cant be bothered commenting. With that in mind, part of what I suggested was for us all to take a breather and try and get together some concise documentation on a web site somewhere, along with working code examples for all the different options, that can sit on top of existing WSGI implementations so that people can actually try out and experiment with the options and see what in practice works. This way people can see exactly what is being suggested rather than having to wade through huge amounts of emails to distil what is important and what isn't. Don't know why you didn't understand what I was suggesting in that part of my email. Anyway, I am at the point that if someone else wants to do that, ie., document the proposals and provide example implementation code that will run on top of WSGI 1.0, they can, but frankly I am not sure I can be bothered now even though I suggested it. Overall, I believe I have more or less worked out what I will probably now do in regard to mod_wsgi and its future, and that decision will mean I no longer have to care about what you all decide as I wouldn't necessarily have to implement it anyway. I'll be talking to a few others off list about if first and then make my final decision. After almost two years of trying to get WSGI for Python 3.0 to fly, I really do think it is time for me to give up. I did say a while back I would try one last push and this has been it. Graham ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
At 11:47 AM 9/24/2009 +1000, Graham Dumpleton wrote: After almost two years of trying to get WSGI for Python 3.0 to fly, I really do think it is time for me to give up. I did say a while back I would try one last push and this has been it. I'm sorry you feel that way, and I'm sorry if I contributed to any frustration on your part. It wasn't my intention, and I'm really taken by surprise, because before Ian's proposal, it sounded like you and I were converging towards something very much like what he ended up proposing. Anyway, just... sorry. I, for one, *really* appreciate the work you put into all of this, as I previously commented on your blog post. And I really hope you'll hang in there. Thanks for all your hard work. --PJ ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
P.J. Eby wrote: I, for one, *really* appreciate the work you put into all of this, as I previously commented on your blog post. And I really hope you'll hang in there. Thanks for all your hard work. +...15 or so :) My +s may not count for much, but they go to many others as well. I have been amazed at some of the detail and thoughtfulness you are all putting into this discussion. Thank you all for your hard work. -- Randy Syring Intelicom 502-644-4776 Whether, then, you eat or drink or whatever you do, do all to the glory of God. 1 Cor 10:31 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Getting back to WSGI grass roots.
Hi Graham, Me being an outsider who contributed nothing to the process, I hope you'll reconsider. I really appreciate your work and I trusted the process more with you in it. Massimo On Sep 23, 2009, at 9:11 PM, P.J. Eby wrote: At 11:47 AM 9/24/2009 +1000, Graham Dumpleton wrote: After almost two years of trying to get WSGI for Python 3.0 to fly, I really do think it is time for me to give up. I did say a while back I would try one last push and this has been it. ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
[Web-SIG] Getting back to WSGI grass roots.
Sorry, after having had a bit of think while eating lunch, I am going to throw up another point of view on this whole issue. So, sit back and be just a little bit concerned. WSGI stands for Web Server GATEWAY Interface. My understanding is that right back at the beginning WSGI was purely intended to only be used at the direct interface with the underlying web server. This is why I understand, in part at least, the term 'gateway is used in the acronym. The problem was that people discovered one could apply the same interface for use as middleware. As we all know, that has been used quite successfully, but has also been equally abused. With that in mind, maybe we should start instead to look more at WSGI being a series of layers. Yes people have talked about standardised request/response objects, but I am not thinking at that high of a level. What I am going to suggest is that there perhaps should still be a clear line between bytes and unicode. So, rather than throw away completely the idea of bytes everywhere, and rewrite the WSGI specification, we could instead say that the existing conceptual idea of WSGI 1.0 is still valid, and just build on top of it a translation interface to present that as unicode. We might still want to respecify WSGI as is now as per the bytes/unicode/native definitions I explained in my blog post at: http://blog.dscpl.com.au/2009/09/roadmap-for-python-wsgi-specification.html I'd suggest this would possibly be the same or quite similar to my original definition #2 in the blog post. To save you having to go back to the blog post, I include it here again. 1. The application is passed an instance of a Python dictionary containing what is referred to as the WSGI environment. All keys in this dictionary are native strings. For CGI variables, all names are going to be ISO-8859-1 and so where native strings are unicode strings, that encoding is used for the names of CGI variables 2. For the WSGI variable 'wsgi.url_scheme' contained in the WSGI environment, the value of the variable should be a native string. 3. For the CGI variables contained in the WSGI environment, the values of the variables are byte strings. 4. The WSGI input stream 'wsgi.input' contained in the WSGI environment and from which request content is read, should yield byte strings. 5. The status line specified by the WSGI application must be a byte string. 6. The list of response headers specified by the WSGI application must contain tuples consisting of two values, where each value is a byte string. 7. The iterable returned by the application and from which response content is derived, must yield byte strings. By seeing WSGI as being layers instead, first thing is that web frameworks such as web2py and CherryPy which merely use WSGI as the gateway interface would continue to work directly on this layer, regardless of whether they use Python 2.X or 3.X. Those frameworks are already going to translate what ever this interface defines into their own internal interface and effectively relegate WSGI from any higher levels of the application. We now get back to the unicode vs bytes argument we have been having. This argument will not vanish by virtue of doing this, but instead of pushing the unicode translation down into the gateway interface layer, we just apply it on top. There is possibly not even a need for the gateway interface layer to even implement the unicode translation layer, and instead this may instead be a documented standard convention that any web application which mounts directly on the gateway interface layer should implement. The danger in taking this approach is that you now risk having two types of so called middleware. These are bytes middleware and unicode middleware. Confusion obviously could come about if you accidentally mix the two, although some middleware may actually be able to operate on either bytes or unicode and so not care. To avoid conflict, one could as a minimal measure just add an additional 'wsgi.' variable which indicates whether interface is 'bytes' or 'unicode' and hope middleware validate they have been plugged in at the correct level. Alternatively you change the interface in some way that they couldn't be plugged together in the first place. Some may see this though as the opportunity to introduce a full request/response object. There is some merit to that as these may actually want to access the original bytes rather than deal with the result of the unicode translation layer. Anyway, that is the thought. Should we be looking at WSGI as a set of layers instead of assuming we have to push unicode into the gateway interface layer? I don't believe this is the same as the prior question of whether WSGI should be bytes or unicode as we are saying it encompasses both, but as separate layers. Previously in asking whether should be bytes or unicode, if the answer was yes to bytes, then the intention was that unicode would be out of scope and every man and his
