Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Armin Ronacher
Hi, Armin Ronacher schrieb: urllib.parse appears to be buggy with bytestrings: I'm pretty sure the latter is a bug and I will file one, however if there is broken behavior with bytestrings in Python 3.1 that's another thing we have to keep in mind. I have to correct myself, there are

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread René Dudfield
On Sat, Sep 19, 2009 at 11:59 AM, Armin Ronacher armin.ronac...@active-4.com wrote: Hi, Armin Ronacher schrieb: urllib.parse appears to be buggy with bytestrings: I'm pretty sure the latter is a bug and I will file one, however if there is broken behavior with bytestrings in Python 3.1

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Armin Ronacher
Hi, René Dudfield schrieb: I think that shows that they are being handled differently depending on type. Which is against polymorphism... but some people prefer to have separate functions for different types(in and out). I don't think other python functions do this though. So maybe this is

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Graham Dumpleton
2009/9/19 Armin Ronacher armin.ronac...@active-4.com: Graham's suggestion for URL encodings means that the URL encoding would ahve to be passed to the WSGI server from outside (he proposed the apache config as an example).  This means that the application behavior will change based on the

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread René Dudfield
On Sat, Sep 19, 2009 at 1:34 PM, Armin Ronacher armin.ronac...@active-4.com wrote: Hi, René Dudfield schrieb: I think that shows that they are being handled differently depending on type.  Which is against polymorphism... but some people prefer to have separate functions for different

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Georg Brandl
René Dudfield schrieb: Here is a snippet from the compat.py we used to port pygame to support python2.3 through 3.1 How is that related? Rather than using a 2to3 tool - which then makes you have two versions of your code, making the code work in python 2.x and 3.x. 2to3 outputs

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread René Dudfield
On Sat, Sep 19, 2009 at 1:54 PM, Graham Dumpleton graham.dumple...@gmail.com wrote: 2009/9/19 Armin Ronacher armin.ronac...@active-4.com: Graham's suggestion for URL encodings means that the URL encoding would ahve to be passed to the WSGI server from outside (he proposed the apache config as

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Armin Ronacher
Hi, Graham Dumpleton schrieb: So, no strict need to make the WSGI adapter do it differently. You may want to only do that if concerned about overhead of transcoding. Transcoding just these is most probably going to be less overhead than the WSGI adapter having to set up both unicode and raw

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Armin Ronacher
Hi, René Dudfield schrieb: Rather than using a 2to3 tool - which then makes you have two versions of your code, making the code work in python 2.x and 3.x. 2to3 outputs python2.x incompatible code - when it doesn't have to. 2to3 is intended to be run automatically for each release. You would

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread René Dudfield
On Sat, Sep 19, 2009 at 2:26 PM, Georg Brandl g.bra...@gmx.net wrote: René Dudfield schrieb: Here is a snippet from the compat.py we used to port pygame to support python2.3 through 3.1 How is that related? Rather than using a 2to3 tool - which then makes you have two versions of your

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Georg Brandl
René Dudfield schrieb: On Sat, Sep 19, 2009 at 2:26 PM, Georg Brandl g.brandl-hi6y0cq0...@public.gmane.org wrote: René Dudfield schrieb: Here is a snippet from the compat.py we used to port pygame to support python2.3 through 3.1 How is that related? Rather than using a 2to3 tool -

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread René Dudfield
Hello again, ok, getting back on topic... away from py3k porting methods... Using an API where the user can request the type wanted solves a lot of encoding issues. This is similar to Grahams suggestion, but instead allowing a user to request which encoding they want, and also get access to

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Massimo Di Pierro
I agree I was forced to write two files. The problems where 1) b'' vs '' (could be solved using eval('b') if python3 else eval() but ugly) 2) try:...:except Exeption,e: vs try:...except Exception as e. (no way around it) 3) it would have required a lot of if

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Armin Ronacher
Hi, René Dudfield schrieb: What is proposed: Where was that proposed? 1. Default utf-8 to be used. That's a possibility yes, but it has to be carefully be considered. 2. A buffer to be used for raw data. What is raw data? If you mean we keep the unencoded data around, I would

Re: [Web-SIG] Unicode in Python 3

2009-09-19 Thread Ian Bicking
I can't read all this thread carefully, too much stuff. I will note however that people are STILL ignoring surrogateescape (http://www.python.org/dev/peps/pep-0383/). This is like the third or fourth time I've brought it up. It was added to Python 3.1 for some of the exact issues we are