Re: [Python-Dev] PEP 414 - some numbers from the Django port
- Original Message - But the stuff you run is not really benchmarking anything. As far as I know django benchmarks benchmark something like mostly DB creation and deletion, although that might differ between CPython and PyPy. How about running *actual* django benchmarks, instead of the test suite? Not that proving anything is necessary, but if you try to prove something, make it right. But my point was only to show that in a reasonable body of Python code (as opposed to a microbenchmark), the overhead of using wrappers was not significant. All those wrapper calls in ported Django and its test suite were exercised. It was not exactly a benchmarking exercise in that it didn't matter what the actual numbers were, nor was any claim being made about absolute performance; just that they were the same for all three variants, within statistical variation. As I mentioned in my other post, I happened to have the Django test suite figures to hand, and to my mind they suited the purpose of showing that wrapper calls, in the overall mix, don't seem to have a noticeable impact (whereas they do, in a microbenchmark). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - some numbers from the Django port
On Wed, Mar 7, 2012 at 2:36 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Armin Ronacher armin.ronacher at active-4.com writes: What are you trying to argue? That the overall Django testsuite does not do a lot of string processing, less processing with native strings? I'm surprised you see a difference at all over the whole Django testsuite and I wonder why you get a slowdown at all for the ported Django on 2.7. The point of the figures is to show there is *no* difference (statistically speaking) between the three sets of samples. Of course, any individual run or set of runs could be higher or lower due to other things happening on the machine (not that I was running any background tasks), so the idea of the simple statistical analysis is to determine whether these samples could all have come from the same populations. According to ministat, they could have (with a 95% confidence level). But the stuff you run is not really benchmarking anything. As far as I know django benchmarks benchmark something like mostly DB creation and deletion, although that might differ between CPython and PyPy. How about running *actual* django benchmarks, instead of the test suite? Not that proving anything is necessary, but if you try to prove something, make it right. Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - some numbers from the Django port
Hi, On 3/3/12 2:28 AM, Vinay Sajip wrote: So, looking at a large project in a relevant problem domain, unicode_literals and native string markers would appear not to adversely impact readability or performance. What are you trying to argue? That the overall Django testsuite does not do a lot of string processing, less processing with native strings? I'm surprised you see a difference at all over the whole Django testsuite and I wonder why you get a slowdown at all for the ported Django on 2.7. Regards, Armin` ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - some numbers from the Django port
Armin Ronacher armin.ronacher at active-4.com writes: What are you trying to argue? That the overall Django testsuite does not do a lot of string processing, less processing with native strings? I'm surprised you see a difference at all over the whole Django testsuite and I wonder why you get a slowdown at all for the ported Django on 2.7. The point of the figures is to show there is *no* difference (statistically speaking) between the three sets of samples. Of course, any individual run or set of runs could be higher or lower due to other things happening on the machine (not that I was running any background tasks), so the idea of the simple statistical analysis is to determine whether these samples could all have come from the same populations. According to ministat, they could have (with a 95% confidence level). The Django test suite is pretty comprehensive, so it would presumably exercise every part of Django, including the parts that handle processing of requests and producing responses. I can't confirm this, not having done a coverage analysis of Django; but this seems like a more representative workload than any microbenchmark which just measures a single operation, like the overhead of a wrapper. And so my argument was that the microbenchmark numbers didn't give a meaningful indication of the actual performance in a real scenario, and they should be taken in that light. No doubt there are other, better (more useful) tests that could be performed (e.g. ab run against all three variants and requests/sec figures compared) but I had the Django test run figures to hand (since they're a byproduct of the porting work), and so presented them in my post. Anyway, it doesn't really matter now, since the latest version of the PEP no longer mentions those figures. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - some numbers from the Django port
On Thu, Mar 8, 2012 at 8:36 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Anyway, it doesn't really matter now, since the latest version of the PEP no longer mentions those figures. Indeed, I deliberately removed the part about performance concerns, since I considered it a distraction from what I see as the heart of the problem PEP 414 is designed to address (i.e. that the purely mechanical changes previously required to Unicode text that is already clearly marked as such in the Python 2 version are irrelevant noise when it comes to identifying and reviewing the *actual* changes needed for a successful Python 3 port). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
Nick Coghlan ncoghlan at gmail.com writes: My rewritten version of PEP 414 is now up (http://www.python.org/dev/peps/pep-0414/). It describes in detail a lot more of the historical background that was taken as read when Guido accepted the PEP. Nice work - thanks! I've implemented a first attempt at an import hook as mentioned in the PEP: https://bitbucket.org/vinay.sajip/uprefix/ It's used as follows: assume you have a simple package hierarchy of code containing u-prefixed literals: frob +-- __init__.py +-- subwob | +-- __init__.py | +-- subsubwob.py +-- wob.py with the following contents: # frob/subwob/__init__.py z = u'def' #- # frob/subwob/subsubwob.py w = u'tuv' #- # frob/__init__.py y = u'xyz' #- # frob/wob.py x = u'abc' #- You can now import these in Python 3.2 using the hook: Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type help, copyright, credits or license for more information. import uprefix; uprefix.register_hook() import frob.subwob.subsubwob frob.subwob.subsubwob.w 'tuv' frob.subwob module 'frob.subwob' from 'frob/subwob/__init__.py' frob.subwob.z 'def' import frob.wob frob.wob.x 'abc' frob module 'frob' from 'frob/__init__.py' frob.y 'xyz' The import uprefix; uprefix.register_hook() is all that's needed to enable the hook. You can also unregister the hook by calling uprefix.unregister_hook(). The project is set up with a setup.py and (basic) test suite, though it's too early to put it on PyPI. I have done basic testing with it, and it should also work as expected in virtual environments. The implementation uses lib2to3, though that could be changed if needed. You can also, of course, use it in Python 3.3 right now (before the PEP gets implemented). Please take a look at it, try it out, and give some feedback. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
On 03/04/2012 10:34 AM, Vinay Sajip wrote:
https://bitbucket.org/vinay.sajip/uprefix/
import uprefix; uprefix.register_hook()
import frob.subwob.subsubwob
frob.subwob.subsubwob.w
Hi,
it's pretty cool that 150 lines is enough to have this functionality.
This guard:
if sys.version_info[0] 3:
raise NotImplementedError('This hook is implemented for Python 3 only')
Wouldn't it be better if the hook did nothing when on python 2?
I think it'll make it necessary to use something like
import sys
if sys.version_info[0] 3:
import uprefix
uprefix.register_hook()
in the calling code to enable the code to run unchanged on both branches.
Also: have you though about providing a context manager which does
register_hook() in __enter__() and unregister_hook() in __exit__()?
I think that some code will want to enable the hook only for specific
modules. The number of lines could be minimized with something like this:
import uprefix
with uprefix.hook:
import abcde_with_u
import bcdef_with_u
import other_module_without_u
Regards,
Zbyszek
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
Zbigniew Jędrzejewski-Szmek zbyszek at in.waw.pl writes:
if sys.version_info[0] 3:
raise NotImplementedError('This hook is implemented for Python 3 only')
Wouldn't it be better if the hook did nothing when on python 2?
I think it'll make it necessary to use something like
Actually I've realised the guard won't be invoked on Python 2, anyway: I later
added a raise ImportError() from e in an exception handler, which leads to a
syntax error in Python 2 before the guard even gets executed.
So, I'll remove the guard (as it does nothing useful anyway) and think a bit
more about not failing on Python 2. Perhaps - not use the from syntax in the
exception handler, and do a no-op in register_hook if on Python 2.
Also: have you though about providing a context manager which does
register_hook() in __enter__() and unregister_hook() in __exit__()?
Of course, things like this can be added without too much trouble.
Thanks for the feedback.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
Hi, It should also be added that the Python 3.3 alpha will release with support: Python 3.3.0a0 (default:042e7481c7b4, Mar 4 2012, 12:37:26) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type help, copyright, credits or license for more information. uHello + ' World!' 'Hello World!' Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
On Sun, Mar 4, 2012 at 11:46 PM, Armin Ronacher armin.ronac...@active-4.com wrote: Hi, It should also be added that the Python 3.3 alpha will release with support: Python 3.3.0a0 (default:042e7481c7b4, Mar 4 2012, 12:37:26) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type help, copyright, credits or license for more information. uHello + ' World!' 'Hello World!' Nice :) Do you have any more updates left to do? I saw the change, the tests, the docs and the tokenizer updates go by on python-checkins, so if you're done we can mark the PEP as Final (at which point the inclusion in the first alpha is implied). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
Hi, On 3/4/12 2:01 PM, Nick Coghlan wrote: Nice :) Do you have any more updates left to do? I saw the change, the tests, the docs and the tokenizer updates go by on python-checkins, so if you're done we can mark the PEP as Final (at which point the inclusion in the first alpha is implied). Docs just have a minor notice regarding the reintroduced support for 'u' prefixes, someone might want to add more to it. Especially regarding the intended use for them. Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
On Sat, Mar 3, 2012 at 11:34 PM, Nick Coghlan ncogh...@gmail.com wrote: My rewritten version of PEP 414 is now up (http://www.python.org/dev/peps/pep-0414/). It describes in detail a lot more of the historical background that was taken as read when Guido accepted the PEP. Thanks very much! It looks great to me. Can we let the interminable discussion die now? Please? Regards, Nick. P.S. If you find an actual factual *error* in the PEP, let me know by private email. If you just disagree with Guido's acceptance of the PEP, or want to quibble about my personal choice of wording on a particular point, please just let it rest. +1 -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
On Mar 04, 2012, at 05:34 PM, Nick Coghlan wrote: My rewritten version of PEP 414 is now up (http://www.python.org/dev/peps/pep-0414/). It describes in detail a lot more of the historical background that was taken as read when Guido accepted the PEP. Nick, really great job with your rewrite of PEP 414. I think you nailed it from the technical side while bringing some much needed balance to the social side. Not to diminish Armin's contribution to the PEP - after all this, I'm really glad he was able to bring it up and despite the heat of the discussion, get this resolved to his satisfaction. One factual omission: In the section on WSGI native strings, you say * binary data: handled as str in Python 2 and bytes in Python 3 While true, this omits that binary data can *also* be handled as bytes in Python 2.6 and 2.7, where using `bytes` can be a more descriptive alias for `str`. If you can do it in a readable way within the context of that section I think it's worth mentioning this. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 updated
On Sun, 2012-03-04 at 17:34 +1000, Nick Coghlan wrote: My rewritten version of PEP 414 is now up (http://www.python.org/dev/peps/pep-0414/). It describes in detail a lot more of the historical background that was taken as read when Guido accepted the PEP. Just as support for string exceptions was eliminated from Python 2 using the normal deprecation process, support for redundant string prefix characters (specifically, B, R, u, U) may be eventually eliminated from Python 3, regardless of the current acceptance of this PEP. We might need to clarify the feature's longevity. I take the above to mean that use of u'' and/or U'' won't emit a deprecation warning in 3.3. But that doesn't necessarily mean its usage won't emit a deprecation warning in 3.4 or 3.5 or 3.6, or whenever it feels right? Does that sound about right? - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Sat, Mar 3, 2012 at 5:02 AM, Lennart Regebro rege...@gmail.com wrote: I'm not sure that's true at all. In most cases where you support both Python 2 and Python 3, most strings will be native, ie, without prefix in either Python 2 or Python 3. The native case is the most common case. Exactly. The reason native strings even exist as a concept in WSGI was to make it so that the idiomatic manipulation of header data in both Python 2 and 3 would use plain old string constants with no special wrappers or markings. What's thrown the monkey wrench in here for the WSGI case is the use of unicode_literals. If you simply skip using unicode_literals for WSGI code, you should be fine with a single 2/3 codebase. But then you need some way to mark some things as unicode... which is how we end up back at this PEP. I suppose WSGI could have gone the route of using byte strings for headers instead, but I'm not sure it would have helped. The design goals for PEP were to sanely support both 2to3 and 2+3 single codebases, and WSGI does actually do that... for the code that's actually doing WSGI stuff. Ironically enough, the effect of the WSGI API is that it's all the *non* WSGI-specific code in the same module that ends up needing to mark its strings as unicode... or else it has to use unicode_literals and mark all the WSGI code with str(). There's really no good way to deal with a *mixed* WSGI/non-WSGI module, except to use explicit markers on one side or the other. Perhaps the simplest solution of all might be to just isolate direct WSGI code in modules that don't import unicode_literals. Web frameworks usually hide WSGI stuff away from the user anyway, and many are already natively unicode in their app-facing APIs. So, if a framework or library encapsulates WSGI in a str-safe/unicode-friendly API, this really shouldn't be an issue for the library's users. But I suppose somebody's got to port the libraries first. ;-) If anyone's updating porting strategy stuff, a mention of this in the tips regarding unicode_literals would be a good idea. i.e., something like: If you have 2.x modules which work with WSGI and also contain explicit u'' strings, you should not use unicode_literals unless you are willing to explicitly mark all WSGI environment and header strings as native strings using 'str()'. This is necessary because WSGI headers and environment keys/values are defined as byte strings in Python 2.x, and unicode strings in 3.x. Alternatively, you may continue to use u'' strings if you are targeting Python 3.3+ only, or can use the import or install hooks provided for Python 3.2, or if you are using 2to3... but in this case you should not use unicode_literals. That could probably be written a lot more clearly. ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Lennart Regebro regebro at gmail.com writes: So the question is if you have any proposal that is *less* confusing while still being practical. Because we do need to distinguish between binary, Unicode and native strings. Isn't this the least confusing solution? It's a matter of the degree of confusion caused (hard to assess) and also a question of taste, so there will be differing views on this. Considering use of unicode_literals, 'xxx' for text, b'yyy' for bytes and with a function wrapper to mark native strings, it becomes clear that the native strings are special cases - much less encountered when looking at code compared to 'xxx' / b'yyy', so there are fewer opportunities for confusion. Where native strings need to be discussed, then it is not unexceptional, nor I believe incorrect, to explain that they are there to suit the requirements of legacy APIs which pre-date Python 3 and the latest versions of Python 2. In terms of practicality, it is IMO quite practical (assuming 2.5 / earlier support can be dropped) to move to a 2.6+/3.x-friendly codebase, e.g. by using Armin's python-modernize. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Sat, Mar 3, 2012 at 10:26, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Lennart Regebro regebro at gmail.com writes: So the question is if you have any proposal that is *less* confusing while still being practical. Because we do need to distinguish between binary, Unicode and native strings. Isn't this the least confusing solution? It's a matter of the degree of confusion caused (hard to assess) and also a question of taste, so there will be differing views on this. Considering use of unicode_literals, 'xxx' for text, b'yyy' for bytes and with a function wrapper to mark native strings, it becomes clear that the native strings are special cases - much less encountered when looking at code compared to 'xxx' / b'yyy', I'm not sure that's true at all. In most cases where you support both Python 2 and Python 3, most strings will be native, ie, without prefix in either Python 2 or Python 3. The native case is the most common case. In terms of practicality, it is IMO quite practical (assuming 2.5 / earlier support can be dropped) to move to a 2.6+/3.x-friendly codebase, e.g. by using Armin's python-modernize. I think there is some misunderstanding here. The binary/unicode/native separation is only possible on Python 2.6 and 2.7 at the moment, unless you use function wrappers like b(). //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Lennart Regebro regebro at gmail.com writes:
I'm not sure that's true at all. In most cases where you support both
Python 2 and Python 3, most strings will be native, ie, without
prefix in either Python 2 or Python 3. The native case is the most
common case.
Sorry, I didn't make myself clear. If you import unicode_literals, then in both
2.x and 3.x code, an unadorned literal string is text, and a b-adorned literal
string is bytes. My assertion was based on that assumption - the text (Unicode)
case then becomes the most common case.
In terms of practicality, it is
IMO quite practical (assuming 2.5 / earlier support can be dropped) to
move to a
2.6+/3.x-friendly codebase, e.g. by using Armin's python-modernize.
I think there is some misunderstanding here. The binary/unicode/native
separation is only possible on Python 2.6 and 2.7 at the moment,
unless you use function wrappers like b().
Right, and that is a possible option for 2.5 and earlier: though obviously not a
desirable one from an aesthetic point of view!
What I meant (and should have said) was: if you can drop support for 2.5 /
earlier, a lib2to3 fixer-based approach brings your 2.x code into the 3-friendly
region of 2.x - 2.6 and 2.7. You can then, using the unicode_literals approach,
arrive at a common codebase for 2.6+ and 3.x which is not slow to run (see my
other post on ported Django test run performance), and clean (looks just like 3
code, pretty much, and means the same, as far as string literals are concerned).
Where you hit native string requirements, apply the wrapper.
I don't actually use python-modernize, as I independently developed fixers when
doing the Django port late last year. I initially wrote a fixer to transform
u'xxx' to u('xxx') (as I was assuming 2.5 support was needed), and then, when it
appeared likely that Django would drop 2.5 support after 1.4, I wrote a fixer to
go from u('xxx') to 'xxx'. Once I learned to use lib2to3, with a few pointers
from Benjamin, it worked like a charm for me.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Sat, Mar 3, 2012 at 11:39, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Sorry, I didn't make myself clear. If you import unicode_literals, then in both 2.x and 3.x code, an unadorned literal string is text, and a b-adorned literal string is bytes. My assertion was based on that assumption - the text (Unicode) case then becomes the most common case. Absolutely. I think there is some misunderstanding here. The binary/unicode/native separation is only possible on Python 2.6 and 2.7 at the moment, unless you use function wrappers like b(). Right, and that is a possible option for 2.5 and earlier: though obviously not a desirable one from an aesthetic point of view! What I meant (and should have said) was: if you can drop support for 2.5 / earlier, a lib2to3 fixer-based approach brings your 2.x code into the 3-friendly region of 2.x - 2.6 and 2.7. You can then, using the unicode_literals approach, arrive at a common codebase for 2.6+ and 3.x which is not slow to run (see my other post on ported Django test run performance), and clean (looks just like 3 code, pretty much, and means the same, as far as string literals are concerned). Where you hit native string requirements, apply the wrapper. Yes, that's a doable solution. Just as the common solution of using b() and u() wrappers. But these are still more confusing and less aesthetically pleasing (and insignificantly slower) than supporting u'' in Python 3. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
2to3 should recognize the str(string_literal) (or nstr(), or native(), etc) as a native string and does not add prefix u to it. And you have to explicitly specify these tips. That is already implemented. 2to3 *never* adds a u prefix anywhere, including not for str(string_literal). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
04.03.12 00:12, Martin v. Löwis написав(ла): 2to3 should recognize the str(string_literal) (or nstr(), or native(), etc) as a native string and does not add prefix u to it. And you have to explicitly specify these tips. That is already implemented. 2to3 *never* adds a u prefix anywhere, including not for str(string_literal). Sorry, I mean *3to2*. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 414 updated
My rewritten version of PEP 414 is now up (http://www.python.org/dev/peps/pep-0414/). It describes in detail a lot more of the historical background that was taken as read when Guido accepted the PEP. Can we let the interminable discussion die now? Please? Regards, Nick. P.S. If you find an actual factual *error* in the PEP, let me know by private email. If you just disagree with Guido's acceptance of the PEP, or want to quibble about my personal choice of wording on a particular point, please just let it rest. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
[quoting Armin from Reddit] Because in all honesty, because string wrappers make a codebase horrible to work with. I will have to continue maintaining 2.x versions for at least another four or five years. The idea if having to use string wrappers for that long makes me puke. Reading this led me to think the following: * 2.5 is now available basically everywhere, and it was released almost 5 years ago (Sep 2006); * if it takes the same time for 3.3, it will be widespread after 4-5 years (i.e. 2016-2017) [0]; * if you want to target a Python 3 version that is widespread [1], you will want to support 3.1/3.2 too in the meanwhile; * therefore you will have to use the hook on 3.1/3.2; * in 2016-2017 you'll finally be able to drop 3.1/3.2 and use only 3.3 without hooks; * in 2016-2017 you'll also stop maintaining the 2.x version (according to that quote); * if you are not maintaining 2.x anymore, you won't need u'' -- right when you could finally stop using the hook; Now, if the hook doesn't get in the way (AFAIU you just have to install it and it will do its work automatically), wouldn't it be better to use it in 3.3 too (especially considering that you will probably have to use it already for 3.1/3.2)? If my reasoning is correct, by the time you will be able to use u without problems you will have to start phasing it out because you won't need to support 2.x anymore. Is this hook available somewhere? How difficult is the installation? Does it strip the u automatically or is there a further step that developers should do before testing on 3.1/3.2? Best Regards, Ezio Melotti [0]: ISTM that people think once you decide to switch to 3.x, there's really no reason to pick an older release, just pick the latest (3.3). While this might be true for single developers that install it by hand, I don't think it's the same for distros and I expect for 3.x the same time span between release and widespread availability that we have with 2.x (i.e. 4-5 years). However this is just an assumption, if you have more accurate information that can show that the time span will indeed be shorter for 3.x (e.g. 2-3 year), feel free to prove me wrong. [1]: I think most projects still support 2.5, some support even older versions, some support only newer ones, but 2.5 as minimum support version seems a good average to me. Targeting the same use base seems reasonable to me (albeit not strictly necessary). I know that this was just a comment on Reddit and was not in the PEP, but it smacks of you throwing all your toys out of the pram. It certainly wasn't a reasoned response to my point. And some of that toys-pram attitude bleeds through into the language of the PEP, leading others to make the criticisms that they have. A PEP is supposed to be balanced, reasonable and thought through. It's not supposed to gloss over things in a hand-wavy sort of way - there's still uncertainty in my mind, for example, whether the 3.2 hook will be a 2to3-style tool that runs over a chunk of the whole project's codebase between editing and running a test, or whether it's an import-time hook which only kicks in on files that have just been edited in a development session. Which of these it is might affect crucially the experience of someone wanting to support 3.2 and 3.3 and 2.6+ - but that clearly isn't you, and you don't seem to have much sympathy or patience with that constituency - we're all stick-in-the-muds who want to work with Ubuntu LTS, rather than people subject to constraints imposed by employers, clients, projects we depend on etc. In contrast, Nick made a more reasonable case when commenting ion my preference for unicode_literals (on this list, not on Reddit), by reminding me about how unicode_literals changes the semantics of string literals, and this increases the cognitive burden on developers. I'm not whinging about the PEP in this post - I've said elsewhere that I wasn't opposed to it. I'm just trying to respond to your apparent bewilderment at some of the reaction to the PEP. I have confidence that with your continued input and Nick's input, the wording of the PEP can be made such that it doesn't ruffle quite so many feathers. I'm looking forward to seeing the updates. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Yury Selivanov writes: Otherwise, many of its points are moot, and we need to raise a question of rejecting it somehow. Yury, that's not going to happen. Guido made it quite clear that he agrees with those who consider this PEP useful, obvious, and safe, and the PEP *is* approved. There has been no hint of second thoughts, and AFAICS no reason why there would be. Please be patient, as Nick has taken on the next revision of this PEP with Armin's approval, and has indicated multiple times that he may take some time to actually do it because of other personal commitments. Regards, ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Wiadomość napisana przez Ezio Melotti w dniu 2 mar 2012, o godz. 10:33: Now, if the hook doesn't get in the way (AFAIU you just have to install it and it will do its work automatically), wouldn't it be better to use it in 3.3 too (especially considering that you will probably have to use it already for 3.1/3.2)? +1 -- Best regards, Łukasz Langa Senior Systems Architecture Engineer IT Infrastructure Department Grupa Allegro Sp. z o.o. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hi Ezio, Am 02.03.2012 um 10:33 schrieb Ezio Melotti: [quoting Armin from Reddit] Because in all honesty, because string wrappers make a codebase horrible to work with. I will have to continue maintaining 2.x versions for at least another four or five years. The idea if having to use string wrappers for that long makes me puke. Reading this led me to think the following: * 2.5 is now available basically everywhere, and it was released almost 5 years ago (Sep 2006); * if it takes the same time for 3.3, it will be widespread after 4-5 years (i.e. 2016-2017) [0]; * if you want to target a Python 3 version that is widespread [1], you will want to support 3.1/3.2 too in the meanwhile; * therefore you will have to use the hook on 3.1/3.2; * in 2016-2017 you'll finally be able to drop 3.1/3.2 and use only 3.3 without hooks; * in 2016-2017 you'll also stop maintaining the 2.x version (according to that quote); * if you are not maintaining 2.x anymore, you won't need u'' -- right when you could finally stop using the hook; I don't think you can compare 2.5 and 3.2 like that. Although 3.2 is/will be shipped with some distributions, it never has, and never will have, the adoption of 2.5 that was mainstream for a quite long time. 3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :) Re the language thingie (not directed towards Ezio): It's true that Armin tends to be opinionated – maybe even polemic. However I can't recall a case where he personally attacked people like it happened here. Regards, Hynek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hi Ezio, Am 02.03.2012 um 10:33 schrieb Ezio Melotti: Reading this led me to think the following: * 2.5 is now available basically everywhere, and it was released almost 5 years ago (Sep 2006); * if it takes the same time for 3.3, it will be widespread after 4-5 years (i.e. 2016-2017) [0]; * if you want to target a Python 3 version that is widespread [1], you will want to support 3.1/3.2 too in the meanwhile; * therefore you will have to use the hook on 3.1/3.2; * in 2016-2017 you'll finally be able to drop 3.1/3.2 and use only 3.3 without hooks; * in 2016-2017 you'll also stop maintaining the 2.x version (according to that quote); * if you are not maintaining 2.x anymore, you won't need u'' -- right when you could finally stop using the hook; I don't think you can compare 2.5 and 3.2 like that. Although 3.2 is/will be shipped with some distributions, it never has, and never will have, the adoption of 2.5 that was mainstream for a quite long time. But I don't think the adoption of 3.2 will affect the decisions that distros will take about 3.3. Even in the unlikely case that e.g. Debian/RHEL make Python 3.3 available as soon as it's released, not everyone will immediately upload to the latest Debian or RHEL version. The point is that regardless the current Python 3 situation, it will take a few years before 3.3 will be widely available on most of the machine. For example I work on a server where I have 3.1. When/if it will be updated it will probably get 3.2, not 3.3 -- and this might happen in a couple of years. If I want 3.3 I will probably have to wait another couple of years. Other people might have to wait less time, others more. 3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :) I heard this about 3.1 and 3.2 too, and indeed they are both perfectly valid releases. The fact that 3.3 is even cooler doesn't mean that 3.1/3.2 are not cool. (I'm perfectly fine with the aforementioned server and 3.1, and currently I don't miss anything that is new in 3.2/3.3.) Best Regards, Ezio Melotti ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Just my 2 cents on the PEP rewrite: u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or native. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
02.03.12 15:49, Lennart Regebro написав(ла): Just my 2 cents on the PEP rewrite: u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or native. What native means in context Python 3 only? Native strings only have meaning if we consider Python 2 and Python 3 together. Native string is a text string, which was binary in Python 2. There is a flag for such strings -- str(). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Mar 02, 2012, at 02:48 PM, Nick Coghlan wrote: Consider: an application that uses 8-bit strings everywhere and blows up on non-ASCII data in Python 2 has at least a fighting chance to run unmodified *and* handle Unicode properly on Python 3. Because unicode literals are gone, a Unicode-aware Python 2 application currently has *no* chance to run unmodified on Python 3. On its face, this statement is incorrect. It *might* be accurate if qualified by saying a Unicode-aware Python 2 *web* application. I say might because I'm not an expert on web frameworks so I defer to those who are. It certainly can't be applied to the entire universe of Unicode-aware Python 2 applications. Accordingly, I'd like to ask folks not to stress too much about the precise wording until I get a chance to update it over the weekend :) /me takes a deep breath. :) -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Mar 02, 2012, at 12:58 PM, Hynek Schlawack wrote: 3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :) Although I disagree with the premise (I think Python 3.2 is a fine platform to build many applications on) it's probably likely what we'll have backports of stable Python 3 releases to 12.04, at the very least in semi-official PPAs. Just like today we're trying to provide a smoother path for LTS-LTS upgrades where 10.04 had only Python 2.6 but 12.04 has only Python 2.7. We have a semi-official Lucid PPA providing Python 2.7, though afaict very few people have actually used or tested it. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Fri, 2012-03-02 at 14:41 -0500, Barry Warsaw wrote: On Mar 02, 2012, at 02:48 PM, Nick Coghlan wrote: Consider: an application that uses 8-bit strings everywhere and blows up on non-ASCII data in Python 2 has at least a fighting chance to run unmodified *and* handle Unicode properly on Python 3. Because unicode literals are gone, a Unicode-aware Python 2 application currently has *no* chance to run unmodified on Python 3. On its face, this statement is incorrect. It *might* be accurate if qualified by saying a Unicode-aware Python 2 *web* application. I say might because I'm not an expert on web frameworks so I defer to those who are. It certainly can't be applied to the entire universe of Unicode-aware Python 2 applications. FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase). - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Am 02.03.2012 um 20:44 schrieb Barry Warsaw: 3.3 is the IMHO the first 3.x release that brings really cool stuff to the table and might be the tipping point for people to start embracing Python 3 – despite the fact that Ubuntu LTS will alas ship 3.2 for the next 10 years. I hope for some half-official back port there. :) Although I disagree with the premise (I think Python 3.2 is a fine platform to build many applications on) Just to be clear: I didn't say 3.2 is “bad” or “not fine”. It's just the fact that people need more than “fine” to feel urged to switch to Python 3. I sincerely hope 3.3 fulfills that and if PEP 414 even makes porting easier we might have a perfect storm. :) it's probably likely what we'll have backports of stable Python 3 releases to 12.04, at the very least in semi-official PPAs. That's what I've been hoping for. Maybe it will work the other way around too: People like 3.3, target it first and port back later to reach more users. It's all about encouraging people to try the nectar of Python 3 – once they're caught it's sticky sweetness[1]… ;) Cheers, Hynek [1] disclaimer: sticky sweetness only applies if you're not a maintainer of wsgi-related middleware/framework ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Mar 02, 2012, at 03:13 PM, Chris McDonough wrote: FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase). Okay, I just want to be very careful about the message we're sending here, because I think many libraries and applications will work fine with the facilities available in today's stable releases, i.e. unicode_literals and b-prefixes. For these, there's no need to define native strings, nor do they require language constructs above what's already available. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Mar 02, 2012, at 09:23 PM, Hynek Schlawack wrote: Just to be clear: I didn't say 3.2 is “bad” or “not fine”. It's just the fact that people need more than “fine” to feel urged to switch to Python 3. I sincerely hope 3.3 fulfills that and if PEP 414 even makes porting easier we might have a perfect storm. :) Cool, and yes reaching that tipping point is what it's all about. :) it's probably likely what we'll have backports of stable Python 3 releases to 12.04, at the very least in semi-official PPAs. That's what I've been hoping for. Maybe it will work the other way around too: People like 3.3, target it first and port back later to reach more users. It's all about encouraging people to try the nectar of Python 3 – once they're caught it's sticky sweetness[1]… ;) Indeed! -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Fri, 2012-03-02 at 15:39 -0500, Barry Warsaw wrote: On Mar 02, 2012, at 03:13 PM, Chris McDonough wrote: FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If e.g. wxPython had already been ported, I think you'd be hearing the same sorts of things from folks that had investments in existing Python-2-compatible code when trying to port stuff to Py3 (at least if they wanted to run on both Python 2 and Python 3 within the same codebase). Okay, I just want to be very careful about the message we're sending here, because I think many libraries and applications will work fine with the facilities available in today's stable releases, i.e. unicode_literals and b-prefixes. For these, there's no need to define native strings, nor do they require language constructs above what's already available. Although the change makes it possible, and it is very useful for very low level WSGI apps, the issue this change addresses really isn't really 100% about needing to define native strings. It's also just preservation of a resource in pretty short supply: developer energy. You will probably need to modify less code when taking piece of software that currently runs on Python 2 and changing it so that it runs on both Python 2 and Python 3, without needing to worry over the unintended consequences of using a unicode_literals future import or replacing existing u'' with a function call. This, IMO, can only be a good thing, because the nominal impact of some future user who must now understand u'' syntax is (again IMO) not as consequential as that user having less software to choose from because porting to Python 3 was just that much harder for existing Python 2 developers. - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Fri, Mar 2, 2012 at 15:26, Serhiy Storchaka storch...@gmail.com wrote: 02.03.12 15:49, Lennart Regebro написав(ла): Just my 2 cents on the PEP rewrite: u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or native. What native means in context Python 3 only? I don't understand your question. Native strings only have meaning if we consider Python 2 and Python 3 together. Native string is a text string, which was binary in Python 2. There is a flag for such strings -- str(). Yes. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Zitat von Lennart Regebro rege...@gmail.com: Just my 2 cents on the PEP rewrite: u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or native. How so? In the Python 3 code, the u prefix would not appear, even if appears in the original source, as 2to3 eliminates it. So you surely need the u prefix to distinguish binary, unicode, or native strings in your source - but with 2to3, the PEP 414 change is unnecessary. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Chris McDonough chrism at plope.com writes:
FWIW, I think this issue's webness may be overestimated. There happens
to be lots and lots of existing UI code which contains complex
interactions between unicode literals and nonliterals in web apps, but
there's also likely lots of nonweb code that has the same issue. If
e.g. wxPython had already been ported, I think you'd be hearing the same
sorts of things from folks that had investments in existing
Python-2-compatible code when trying to port stuff to Py3 (at least if
they wanted to run on both Python 2 and Python 3 within the same
codebase).
As I understand it, WSGI happens to explicitly expect str in certain places,
even places where conceptually text should be acceptable. The perception of
webness seems to be substantiated by Nick's comment about endorsement from you,
Armin, Jacob Kaplan-Moss, and Kenneth Reitz for this change. Not that webness is
a bad thing, of course - it's a very important part of the ecosystem.
It would be good to hear from other constituencies about where else (apart from
WSGI and the other uses mentioned in the APIs and Concepts Using Native
Strings section of the PEP) native strings are needed. I have encountered such
needs sometimes, but not uncommonly, they appear to be broken APIs that just
expect str even though text should be OK (e.g. cookie APIs, or the sqlite
adapter's insisting on accepting datetimes in text format, but only as native
strings). It would be a shame to leave these APIs as they are indefinitely, and
perhaps using a marker like n('xxx') for native strings would help to remind us
that these areas need addressing at some point.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 414 - some numbers from the Django port
PEP 414 mentions the use of function wrappers and talks about both their obtrusiveness and performance impact on Python code. In the Django Python 3 port, I've used unicode_literals, and hence have no u prefixes in the ported code, and use a function wrapper to adorn native strings where they are needed. Though the port is still work in progress, it passes all tests on 2.x and 3.x with the SQLite adapter, with only a small number skipped specifically during the porting exercise (generally due to representational differences). I'd like to share some numbers from this port to see what people here think about them. Firstly, on obtrusiveness: Out of a total of 1872 source files, the native string marker only appears in 30 files - 18 files in Django itself, and 12 files in the test suite. This is less than 2% of files, so the native string markers are not especially invasive when looking at code. There are only 76 lines in the ported Django which contain native string markers. Secondly, on performance. I ran the following steps 6 times: Run the test suite on unported Django using Python 2.7.2 (vanilla) Run the test suite on the ported Django using Python 2.7.2 (ported) Run the test suite on the ported Django using Python 3.2.2 (ported3) Django skips some tests because dependencies aren't installed (e.g. PIL for Python 3.2). The raw numbers, in seconds elapsed for the test run, are given below: vanilla (4659 tests): 468.586 486.231 467.584 464.916 480.530 475.457 ported (4655 tests): 467.350 480.902 479.276 478.748 478.115 486.044 ported3 (4609 tests): 463.161 470.423 463.833 448.097 456.727 504.402 If we allow for the different numbers of tests run by dividing by the number of tests and multiplying by 100, we get: vanilla-weighted: 10.057 10.436 10.036 9.979 10.314 10.205 ported-weighted: 10.040 10.331 10.296 10.285 10.271 10.441 ported3-weighted: 10.049 10.207 10.064 9.722 9.909 10.944 If I run these through ministat, it tells me there is no significant difference in these data sets, with a 95% confidence level: $ ministat -w 74 vanilla-weighted ported-weighted ported3-weighted x vanilla-weighted + ported-weighted * ported3-weighted +--+ |* + | |* * x *** ++x+ * *| ||___|___M|AA_M___AM___|__|_| | +--+ N Min MaxMedian AvgStddev x 6 9.97910.43610.205 10.1711670.17883782 + 6 10.0410.44110.296 10.2773330.13148485 No difference proven at 95.0% confidence * 6 9.72210.94410.064 10.1491670.42250274 No difference proven at 95.0% confidence So, looking at a large project in a relevant problem domain, unicode_literals and native string markers would appear not to adversely impact readability or performance. Your comments would be appreciated. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Chris McDonough chrism at plope.com writes: Although the change makes it possible, and it is very useful for very low level WSGI apps, the issue this change addresses really isn't really 100% about needing to define native strings. It's also just preservation of a resource in pretty short supply: developer energy. Apparently developer energy is a limitless resource when it comes to arguing over PEPs ;-) This, IMO, can only be a good thing, because the nominal impact of some It can also have some downsides, at least according to some points of view. For example, I regard elevating native strings to undue prominence, and the continued use of u'xxx' in Python 3 code, as unfortunate consequences. For example, with PEP 414, it will be possible to mix Unicode with and without prefix - how would that not be at least a little confusing for users new to Python? Remember, native strings are a Python-only concept. future user who must now understand u'' syntax is (again IMO) not as consequential as that user having less software to choose from because porting to Python 3 was just that much harder for existing Python 2 developers. I don't believe it's because porting to Python 3 is especially hard. I'm not saying it's trivial, but it isn't rocket surgery ;-) Even if porting were trivially easy to do technically at the level the PEP addresses, there would still be additional tests, and perhaps documentation, and perhaps release-related work to be done. Since Python 2.x is a very good platform for software development, where's the incentive to move over to 3.x? It's the chicken and egg effect. Many people are waiting for other people to move over (perhaps projects they depend upon), and while the transition is happening, it's not as quick as it could be. I think a lot of it is down to inertia. Possibly another factor was the just use 2to3 message, which we now know doesn't work well in all scenarios. However, I don't believe that the use a single codebase, use six or six-like techniques, use unicode_literals, use the 2to3 fixer to remove unicode prefixes, and use native string markers where you need to message has received anything like the same level of airplay. If you talk to people who have *actually tried* this approach (say Barry, or me) you'll hear that it's not been all that rough a ride. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Fri, Mar 2, 2012 at 7:22 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Apparently developer energy is a limitless resource when it comes to arguing over PEPs ;-) Aren't *you* the one who keeps kicking this dead horse? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Guido van Rossum guido at python.org writes: Aren't *you* the one who keeps kicking this dead horse? From looking at the overall thread, I'm just one of many people posting on it. Which dead horse am I kicking? It's not as if I'm opposing anything or anyone - just putting my point of view forward about porting from 2.x - 3.x, as others have done - that's not OT, is it? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Sat, Mar 3, 2012 at 01:49, mar...@v.loewis.de wrote: Zitat von Lennart Regebro rege...@gmail.com: Just my 2 cents on the PEP rewrite: u'' support is not just if you want to write code that doesn't use 2to3. Even when you use 2to3 it is useful to be able to flag strings s binary, unicode or native. How so? In the Python 3 code, the u prefix would not appear, even if appears in the original source, as 2to3 eliminates it. Well, not if you disable that fixer. ;-) But you are right, it isn't necessary. I was thinking of 3to2, actually. That was one of the objections I had to the usefulness of 3to2, there is no way to make the distinction between unicode and native strings. (The u'' prefix hence actually makes 3to2 a realistic option, and that's good.) So everyone can ignore this, I mixed up two issues. :-) //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Sat, Mar 3, 2012 at 04:22, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: It can also have some downsides, at least according to some points of view. For example, I regard elevating native strings to undue prominence, and the continued use of u'xxx' in Python 3 code, as unfortunate consequences. For example, with PEP 414, it will be possible to mix Unicode with and without prefix - how would that not be at least a little confusing for users new to Python? Remember, native strings are a Python-only concept. This is true, new users will see 'foo', r'foo', b'foo', and will naturally assume u'foo' is something special too, and will have to be told it is not. But that's an unfortunate effect of Python 3 making the change to Unicode strings, a change that *removed* a lot of other much more confusing things. So the question is if you have any proposal that is *less* confusing while still being practical. Because we do need to distinguish between binary, Unicode and native strings. Isn't this the least confusing solution? The only way we could have avoided this three strings situation is by actually removing native strings from Python for at least five years, and only used b'' or u''. That would not have been any less confusing. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Chris McDonough writes: FWIW, I think this issue's webness may be overestimated. There happens to be lots and lots of existing UI code which contains complex interactions between unicode literals and nonliterals in web apps, but there's also likely lots of nonweb code that has the same issue. If we generalize web to wire protocols, I would say that nonweb code that has the same issue is poorly coded. I suppose there may be some similar issues in say XML handling, because XML can be used in binary applications as well as for structuring text (ie, XML is really a wire protocol too). But pure user interface modules like wxPython? Text should be handled as text, not as bytes that probably are ASCII-encoded or locale-specifically-encoded (or are magic numbers that happen to be mnemonic when interpreted as ASCII). I don't say that we should ignore the pain of the nonweb users -- but it is a different issue, with different solutions. In particular, using native strings (and distinguishing them by the absence of u'') is usually a non-solution for non-web applications, because it propagates the bad practice of pretending that unknown encodings can be assumed to be well-behaved into an environment where good practice is designed in. This is quite different from the case for webby usage, where it often makes sense to handle many low-level operations without ever converting to text, while the same literal strings may be useful in both wire and text contexts (and so should be present only once according to DRY). (N.B. I suspect that it is probably also generally possible for webby applications to avoid native strings without much cost, as Nick showed in urlparse. But at least manipulations of the wire protocol without conversion to text are a plausible optimization.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
03.03.12 08:20, Lennart Regebro написав(ла): But you are right, it isn't necessary. I was thinking of 3to2, actually. That was one of the objections I had to the usefulness of 3to2, there is no way to make the distinction between unicode and native strings. (The u'' prefix hence actually makes 3to2 a realistic option, and that's good.) 2to3 should recognize the str(string_literal) (or nstr(), or native(), etc) as a native string and does not add prefix u to it. And you have to explicitly specify these tips. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Mar 01, 2012, at 09:42 AM, Guido van Rossum wrote: I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? Not yet, but I believe Nick volunteered to do a rewrite. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Guido van Rossum guido at python.org writes: I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? I'm not sure if Nick has finished his updates, but I for one would like to see some improvements in a few places: Many thought that the unicode_literals future import might make a common source possible, but it turns out that it's doing more harm than good. Rather than talking about it doing more harm than good, it would be better to say that unicode_literals is not the best solution in some scenarios (specifically, WSGI, but any other scenarios can also be mentioned). The more harm than good is not true in all scenarios, but as it's worded now, it seems like it is always a bad approach. (either by having a u function that marks things as unicode without future imports or the inverse by having a n function that marks strings as native). Unfortunately, this has the side effect of slowing down the runtime performance of Python and makes for less beautiful code. The use of u() and n() are not equivalent in the sense that n() only has to be used when unicode_literals are in effect, and the incidence of n() calls in an application would be much lower than using u() in the absence of unicode_literals. In at least some cases, it is possible that some of the APIs which fail unless native strings are provided may be broken (e.g. some database adapters expect datetimes in ISO format as native strings, where there is no apparent reason why they couldn't accept them as text). As far as less beautiful code is concerned, it's subjective: I see nothing especially ugly about 'xxx' for text, and certainly don't find u'xxx' more beautiful - and I doubt if I'm the only person with that view. The point about the added cognitive burden of semantic-changing __future__ imports is, however, quite valid. As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. This looks to become a self-fulfilling prophecy, if you take it seriously. You would expect that, if Python 3 is the future of Python, then Python 3 is *precisely* the choice for *long*-term investments. The ecosystem is not yet fully developed, true: but that is because some people aren't ready to grasp the nettle and undergo the short-term pain required to get things in order. By things, I mean places in existing 2.x code where no distinction was made between bytes and text, which you could get away with because of 2.x's forgiving nature. Whether you're using unicode_literals and 'xxx' or u'xxx', these things will need to be sorted out, and the syntax element is only one possible focus. If that entire sentence is removed, it does the PEP no harm, and the PEP will antagonise fewer people. A valid point is that this would encourage people to become dependent on Python 3.3 for their ports. Fortunately that is not a big problem since that could be fixed at installation time similar to how many projects are currently invoking 2to3 as part of their installation process. Yes, but avoiding the very pain of running 2to3 is what (at least in part) motivates the PEP in the first place. This appears to be moving the pain that 2.x developers feel when trying to move to 3.x, to people who want to support 3.2 and 3.3 and 2.6+ in the same codebase. For Python 3.1 and Python 3.2 (even 3.0 if necessary) a simple on-installation hook could be provided that tokenizes all source files and strips away the otherwise unnecessary u prefix at installation time. There's some confusion about this hook - The PEP calls it an on-installation hook (like 2to3) but Nick said it was an import-time hook. I'm more comfortable with the latter - it has a chance of providing an acceptable performance for a large codebase, as it will only kick in when .py files are newer than their .pyc. A 2to3 like hook, when working with a large codebase like Django, is likely to be about as painful as people are finding 2to3 now (when used in an edit-test-edit-test workflow). Possible Downsides does not mention any possible adverse impact on single codebase for 3.2/3.3, which I mention only because it's still not clear how the hook which is to make 3.2 development easier will work (in terms of its impact on development workflow). In the section on Modernizing code, but to make strings cheap for both 2.x and 3.x it is nearly impossible. The way it currently works is by abusing the unicode-escape codec on Python 2.x native strings. IIUC, the unicode-escape codec is only needed if you don't use unicode_literals - am I wrong about that? How are strings not equally cheap (near enough) on 2.x and 3.x if you use unicode_literals? In the Runtime overhead of wrappers, the times may be valid, but a rider should be added to the
Re: [Python-Dev] PEP 414
Vinay, Thank you for the comprehensive summary. Big +1. I really do hope that Nick and Armin will rectify the PEP. Otherwise, many of its points are moot, and we need to raise a question of rejecting it somehow. On 2012-03-01, at 2:00 PM, Vinay Sajip wrote: Guido van Rossum guido at python.org writes: I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? I'm not sure if Nick has finished his updates, but I for one would like to see some improvements in a few places: Many thought that the unicode_literals future import might make a common source possible, but it turns out that it's doing more harm than good. Rather than talking about it doing more harm than good, it would be better to say that unicode_literals is not the best solution in some scenarios (specifically, WSGI, but any other scenarios can also be mentioned). The more harm than good is not true in all scenarios, but as it's worded now, it seems like it is always a bad approach. (either by having a u function that marks things as unicode without future imports or the inverse by having a n function that marks strings as native). Unfortunately, this has the side effect of slowing down the runtime performance of Python and makes for less beautiful code. The use of u() and n() are not equivalent in the sense that n() only has to be used when unicode_literals are in effect, and the incidence of n() calls in an application would be much lower than using u() in the absence of unicode_literals. In at least some cases, it is possible that some of the APIs which fail unless native strings are provided may be broken (e.g. some database adapters expect datetimes in ISO format as native strings, where there is no apparent reason why they couldn't accept them as text). As far as less beautiful code is concerned, it's subjective: I see nothing especially ugly about 'xxx' for text, and certainly don't find u'xxx' more beautiful - and I doubt if I'm the only person with that view. The point about the added cognitive burden of semantic-changing __future__ imports is, however, quite valid. As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. This looks to become a self-fulfilling prophecy, if you take it seriously. You would expect that, if Python 3 is the future of Python, then Python 3 is *precisely* the choice for *long*-term investments. The ecosystem is not yet fully developed, true: but that is because some people aren't ready to grasp the nettle and undergo the short-term pain required to get things in order. By things, I mean places in existing 2.x code where no distinction was made between bytes and text, which you could get away with because of 2.x's forgiving nature. Whether you're using unicode_literals and 'xxx' or u'xxx', these things will need to be sorted out, and the syntax element is only one possible focus. If that entire sentence is removed, it does the PEP no harm, and the PEP will antagonise fewer people. A valid point is that this would encourage people to become dependent on Python 3.3 for their ports. Fortunately that is not a big problem since that could be fixed at installation time similar to how many projects are currently invoking 2to3 as part of their installation process. Yes, but avoiding the very pain of running 2to3 is what (at least in part) motivates the PEP in the first place. This appears to be moving the pain that 2.x developers feel when trying to move to 3.x, to people who want to support 3.2 and 3.3 and 2.6+ in the same codebase. For Python 3.1 and Python 3.2 (even 3.0 if necessary) a simple on-installation hook could be provided that tokenizes all source files and strips away the otherwise unnecessary u prefix at installation time. There's some confusion about this hook - The PEP calls it an on-installation hook (like 2to3) but Nick said it was an import-time hook. I'm more comfortable with the latter - it has a chance of providing an acceptable performance for a large codebase, as it will only kick in when .py files are newer than their .pyc. A 2to3 like hook, when working with a large codebase like Django, is likely to be about as painful as people are finding 2to3 now (when used in an edit-test-edit-test workflow). Possible Downsides does not mention any possible adverse impact on single codebase for 3.2/3.3, which I mention only because it's still not clear how the hook which is to make 3.2 development easier will work (in terms of its impact on development workflow). In the section on Modernizing code, but to make strings cheap for both 2.x and 3.x it is nearly impossible. The way it currently
Re: [Python-Dev] PEP 414
Hi, On 2/29/12 12:30 PM, Yury Selivanov wrote: I see you've (or somebody) changed: Yes, I reworded that. Could you just remove the statement completely? I will let Nick handle the PEP wording. I don't think that PEPs are the right place to put such polemic and biased statements. Why call it polemic? If you want to use ubuntu LTS you're forcing yourself to stick to a particular Python version for a longer time. Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. That is not a personal-feeling-PEP. If people would be 100% happy with Python 3 we would not have these discussions, would we. Why is it that I'm getting attacked on this mailinglist for writing this PEP, or the wording etc. Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Thu, 01 Mar 2012 21:12:48 +, Armin Ronacher armin.ronac...@active-4.com wrote: Hi, On 2/29/12 12:30 PM, Yury Selivanov wrote: I see you've (or somebody) changed: Yes, I reworded that. Could you just remove the statement completely? I will let Nick handle the PEP wording. I don't think that PEPs are the right place to put such polemic and biased statements. Why call it polemic? If you want to use ubuntu LTS you're forcing Presumably because it comes across to him that way. Perception aside, I do think it matches the dictionary meaning of the term (One who writes in support of one opinion, doctrine, or system, in opposition to another), which Nick's edits will presumably fix (by addressing all sides of the argument, as a finished PEP should). yourself to stick to a particular Python version for a longer time. Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact. Wow. I never would have guessed that from the sentence in question. I don't think I agree with your that means statement either, I can imagine other motivations for using an LTS. But I don't think that discussion is worth getting in to or matters for the PEP. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. That is not a personal-feeling-PEP. If people would be 100% happy with Python 3 we would not have these discussions, would we. Why is it that I'm getting attacked on this mailinglist for writing this PEP, or the wording etc. I think it is because people are *perceiving* that you are attacking Python3 and arguing (out of your personal experience) that porting is harder than other people (out of their personal experience) have found it to be. This presumably reflects the different problem domains people are working in. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hopefully, I can say the following in a constructive way. I certainly don't mean to attack anyone personally for their closely held beliefs, though I might disagree with them. And you have the right to those beliefs and to express them in a respectful and constructive manner on this mailing list, which I think you've done. No criticisms there. However, PEPs *are* official documents from the Python developer community, so I think it's required of us to present technical issues in an honest light, yet devoid of negative connotations which harm Python. On Mar 01, 2012, at 09:12 PM, Armin Ronacher wrote: Why call it polemic? If you want to use ubuntu LTS you're forcing yourself to stick to a particular Python version for a longer time. Not just a particular Python 3 version, but a particular Python 2 version too. And a particular kernel version, and version of Perl, Ruby, Java, gcc, etc. etc. That's kind of the whole point of an LTS. :) Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact. I don't agree with the conclusion. But none of that is germane to the PEP anyway. The PEP could simply say that for some domains, the ability to port code from Python 2 to Python 3 would be enhanced by the reintroduction of the u-prefix. It could even explain why WSGI applications in particular would benefit from this. That would be enough to justify Guido's acceptance of the PEP. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hi Armin, Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP polemic. Thanks, - Yury On 2012-03-01, at 4:12 PM, Armin Ronacher wrote: Hi, On 2/29/12 12:30 PM, Yury Selivanov wrote: I see you've (or somebody) changed: Yes, I reworded that. Could you just remove the statement completely? I will let Nick handle the PEP wording. I don't think that PEPs are the right place to put such polemic and biased statements. Why call it polemic? If you want to use ubuntu LTS you're forcing yourself to stick to a particular Python version for a longer time. Which means you don't want to have to adjust your code. Which again means that you're better of with the Python 2.x ecosystem which is proven, does not change nearly as quickly as the Python 3 one (hopefully) so if you have the choice between those two you would chose 2.x over 3.x. That's what this sentence is supposed to say. That's not polemic, that's just a fact. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. That is not a personal-feeling-PEP. If people would be 100% happy with Python 3 we would not have these discussions, would we. Why is it that I'm getting attacked on this mailinglist for writing this PEP, or the wording etc. Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote: Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other communication channels and by peers). You were just the unlucky mail I responded to :-) But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP polemic. I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals :-) Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On 2012-03-01, at 5:52 PM, Armin Ronacher wrote: Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote: Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other communication channels and by peers). You were just the unlucky mail I responded to :-) It's OK ;) But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP polemic. I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals :-) Well, unfortunately it's not that simple from the standpoint of how this change will be perceived by the community. If we have u'' syntax in python 3, will people even understand what is the key difference from python 2? Will the internet be polluted with weird source-code targeted only for python3, but with the wide use of u''? When to deprecate it, and will it ever be deprecated (as everybody is already tired with all this)? Will it further strengthen the common misbelief the porting is hard (as for the many of the projects it is not), or that the right way it to have one code-base for all versions? And that's just the beginning of such questions. And when this PEP was suddenly approved, many of us felt that all those questions are not answered and were not even discussed. - Yury ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Armin Ronacher armin.ronacher at active-4.com writes:
I tried my best but obviously it was not good enough to please
everybody. In all honesty I did not expect that such a small change
would spawn such a great discussion. After all what we're discussing
here is the introduction of one letter to literals
The objections are not to the introduction of one letter to literals. It is the
extent to which, in your presentation of the PEP, a narrow set of concerns and a
specific addressing of those concerns has been represented as if it is the only
possible view of all right-thinking people in the Python community. What is
obvious to you may not be so to others - au contraire. A PEP author obviously
will promote their specific views - it is an instrument of advocacy - but an
author should not, in my view, arrogate to themselves the presumption of
speaking for everyone in the community; rather, they should respect that others
may have different sensibilities.
The PEP comes across as being primarily motivated by WSGI concerns: Nick
mentioned that he would update the PEP to name drop and indicate support from
you, Jacob Kaplan-Moss and Chris McDonough - all authors of Web frameworks.
While I completely acknowledge the importance and ubiquity of these web
frameworks in the overall ecosystem, I think Python the language is about more
than just Web development. There is a bit of a sense of the tail wagging the
dog. Let's remember, it's possible to do Web development without the concept of
native strings - this doesn't exist AFAIK in many other languages which allow
Web applications to be developed - the concept is a sort of historical accident
arising in part out of how the WSGI spec was written and evolved, interacting
with how 3.x differs from 2.x, and how some legacy APIs expect native strings
because they are broken.
There are a number of possible ways of addressing the concerns which motivated
the PEP, but in my view you have given some of them short shrift because of what
come across as personal prejudices. An example - on a Reddit thread about PEP
414, I commented:
The PEP does not (for example) consider the possibility of leaving literals as
they are and using a n('xxx') callable for native strings. Since there are very
few places where native strings are needed, this approach is potentially less
obtrusive than either u'xxx' or u('xxx').
Your response to this was:
Because in all honesty, because string wrappers make a codebase horrible to
work with. I will have to continue maintaining 2.x versions for at least another
four or five years. The idea if having to use string wrappers for that long
makes me puke.
I know that this was just a comment on Reddit and was not in the PEP, but it
smacks of you throwing all your toys out of the pram. It certainly wasn't a
reasoned response to my point. And some of that toys-pram attitude bleeds
through into the language of the PEP, leading others to make the criticisms that
they have. A PEP is supposed to be balanced, reasonable and thought through.
It's not supposed to gloss over things in a hand-wavy sort of way - there's
still uncertainty in my mind, for example, whether the 3.2 hook will be a
2to3-style tool that runs over a chunk of the whole project's codebase between
editing and running a test, or whether it's an import-time hook which only kicks
in on files that have just been edited in a development session. Which of these
it is might affect crucially the experience of someone wanting to support 3.2
and 3.3 and 2.6+ - but that clearly isn't you, and you don't seem to have much
sympathy or patience with that constituency - we're all stick-in-the-muds who
want to work with Ubuntu LTS, rather than people subject to constraints imposed
by employers, clients, projects we depend on etc.
In contrast, Nick made a more reasonable case when commenting ion my preference
for unicode_literals (on this list, not on Reddit), by reminding me about how
unicode_literals changes the semantics of string literals, and this increases
the cognitive burden on developers.
I'm not whinging about the PEP in this post - I've said elsewhere that I wasn't
opposed to it. I'm just trying to respond to your apparent bewilderment at some
of the reaction to the PEP.
I have confidence that with your continued input and Nick's input, the wording
of the PEP can be made such that it doesn't ruffle quite so many feathers. I'm
looking forward to seeing the updates.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/01/2012 05:52 PM, Armin Ronacher wrote: Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote: Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other communication channels and by peers). You were just the unlucky mail I responded to :-) Several responses on the list *have* been offensive, not criticizing the PEP on its own merits but on your (presumed) motives for introducing it. Such attacks are wildly off-base. But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all porting scenarios with well-thought set of benchmarks, nobody would ever call your PEP polemic. I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one letter to literals :-) Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk9QE5AACgkQ+gerLs4ltQ6m7wCdHufuDMrplgg0+MQr4M10Y4Oy N74AoJW5UKbUjPOdrreeTT38C9Cl2eFk =DkBX -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hello, Le 02/03/2012 00:15, Yury Selivanov a écrit : And that's just the beginning of such questions. And when this PEP was suddenly approved, many of us felt that all those questions are not answered and were not even discussed. Let me comment on that “suddenly”. We joke about Guido being the dictator for Python, but it’s actually not a joke. The point of the PEP process is to help Guido make an informed decision on a proposed change. (There are also side benefits like providing a record of design or implementation choices, or documenting once and for all why some idea will never be accepted, but let’s ignore them here.) I can’t read Guido’s mind, but I think that here he pronounced somewhat quickly because he was convinced by the arguments in the PEP, while choosing to ignore the problems therein, knowing that they could be fixed later. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Fri, Mar 2, 2012 at 2:08 PM, Éric Araujo mer...@netwok.org wrote: I can’t read Guido’s mind, but I think that here he pronounced somewhat quickly because he was convinced by the arguments in the PEP, while choosing to ignore the problems therein, knowing that they could be fixed later. It's also the case that this particular point has been the subject of debate for a *long* time. When the decision was first made to offer the unicode_literals future import, one of the other contenders was just to allow the u/U prefix in Python 3 and not worry about it, and while the purity side carried the day back then, it was a close run thing. While that approach turned out to work pretty well for many users that didn't use unicode literals all that much, the web community understandably feel like they're being actively *punished* for doing Unicode right in Python 2. Consider: an application that uses 8-bit strings everywhere and blows up on non-ASCII data in Python 2 has at least a fighting chance to run unmodified *and* handle Unicode properly on Python 3. Because unicode literals are gone, a Unicode-aware Python 2 application currently has *no* chance to run unmodified on Python 3. So even though the PEP doesn't currently do a good job of *presenting* that history to new readers, it's still very much a factor in the decision making process. Accordingly, I'd like to ask folks not to stress too much about the precise wording until I get a chance to update it over the weekend :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Armin, I see you've (or somebody) changed: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. to: As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. Could you just remove the statement completely? Again, my understanding of what is the best choice for certain *long-term* investments is drastically different from yours. In my opinion, python 3 is much more suitable for anything *long-term* than python 2. I don't think that PEPs are the right place to put such polemic and biased statements. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. There are blogs for that. Thank you. - Yury On 2012-02-28, at 11:29 AM, Yury Selivanov wrote: Hi Armin, Could you please remove from the PEP the following statement: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org. The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc. I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3. Thanks, -Yury On 2012-02-27, at 5:38 PM, Armin Ronacher wrote: Hi, On 2/27/12 10:18 PM, Terry Reedy wrote: I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of except (((ExceptionType),),), the comparision operator and the removal of nonlocal, the latter to make Python 2.x developers feel better about themselves. :-) Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Armin, I see you've (or somebody) changed: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. to: As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. Could you just remove the statement completely? Again, my understanding of what is the best choice for certain *long-term* investments is drastically different from yours. In my opinion, python 3 is much more suitable for anything *long-term* than python 2. I don't think that PEPs are the right place to put such polemic and biased statements. Nobody asked you to express your *personal* feelings and thoughts about applicability or state of python3 in the PEP. There are blogs for that. Thank you. - Yury On 2012-02-28, at 11:29 AM, Yury Selivanov wrote: Hi Armin, Could you please remove from the PEP the following statement: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org. The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc. I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3. Thanks, -Yury On 2012-02-27, at 5:38 PM, Armin Ronacher wrote: Hi, On 2/27/12 10:18 PM, Terry Reedy wrote: I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of except (((ExceptionType),),), the comparision operator and the removal of nonlocal, the latter to make Python 2.x developers feel better about themselves. :-) Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Feb 29, 2012, at 07:30 AM, Yury Selivanov wrote: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. to: As it stands, when chosing between 2.7 and Python 3.2, Python 3 is currently not the best choice for certain long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. Could you just remove the statement completely? If I read correctly, Nick is undertaking a rewrite of PEP 414, which should help a lot. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
A couple of people have said that 'native string' is spelt 'str', but I'm not sure that's the right answer. For example, 2.x's cString.StringIO expects native strings, not Unicode: Your counter-example is non-ASCII characters/bytes. I doubt that this is a valid use case; in a native string, these shouldn't occur (i.e. native strings should always be ASCII), since the semantics of non-ASCII changes drastically between 2.x and 3.x. So whoever defines some API to take native strings can't have defined a valid use of non-ASCII in that interface. I'm not saying this is the right thing to do for all cases - just that str() may not be, either. This should be elaborated in the PEP. Indeed it should. If there is a known application of non-ASCII native strings, I surely would like to know what that is. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Hi,
On 2/28/12 12:16 AM, mar...@v.loewis.de wrote:
Armin, I propose that you correct the *factual* deficits of the PEP
(i.e. remove all claims that cannot be supported by facts, or are otherwise
incorrect or misleading). Many readers here would be more open to accepting
the PEP if it was factual rather than polemic.
Please don't call this PEP polemic.
The PEP author is supposed to collect all arguments, even the ones he
doesn't agree with, and refute them.
I brought up all the arguments that were I knew about before I submitted
this mailinglist thread and I had since not updated it.
In this specific issue, the PEP states the unicode_literals import the
native string type is no longer available and has to be incorrectly
labeled as bytestring
This is incorrect: even though the native string type indeed is no longer
available, it is *not* consequential that it has to be labeled as byte
string. Instead, you can use the str() function.
Obviously it means not available by syntax.
It may be that you don't like that solution for some reason. If so, please
mention the approach in the PEP, along with your reason for not liking it.
If by str() you mean using str('x') as replacement for 'x' in both 2.x
and 3.x with __future__ imports as a replacement for native string
literals, please mention why this is better than u(), s(), n() etc. It
would be equally slow than a custom wrapper function and it would not
support non-ascii characters.
Regards,
Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Hi, On 2/27/12 11:54 PM, Steven D'Aprano wrote: That would be one order of magnitude. I am aware of that :-) Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
The PEP author is supposed to collect all arguments, even the ones he
doesn't agree with, and refute them.
I brought up all the arguments that were I knew about before I submitted
this mailinglist thread and I had since not updated it.
This is fine, of course. I still hope you will update it now, even
though it has been accepted.
This is incorrect: even though the native string type indeed is no longer
available, it is *not* consequential that it has to be labeled as byte
string. Instead, you can use the str() function.
Obviously it means not available by syntax.
I agree that the native string type is no longer supported by syntax in
that approach.
It may be that you don't like that solution for some reason. If so, please
mention the approach in the PEP, along with your reason for not liking it.
If by str() you mean using str('x') as replacement for 'x' in both 2.x
and 3.x with __future__ imports as a replacement for native string
literals, please mention why this is better than u(), s(), n() etc. It
would be equally slow than a custom wrapper function and it would not
support non-ascii characters.
That's not the point. The point is that the PEP ought to mention it as
an alternative, instead of making the false claim that it has to be
labeled as byte string (which I take as using a b prefix). Feel free
to write something like
... it either has to be labelled as a byte string, or wrapped into
a function call, e.g. using the str() function. This would be slow and
would not support non-ascii characters
My whole point here is that I want the PEP to mention it, not this
email thread.
In addition, if you are using this very phrasing that I propose,
I would then claim that
a) it is not slow (certainly not as slow as a custom wrapper (*)), and
b) it's not a problem that it is ASCII-only, since native strings
are *practically* restricted to ASCII, anyway (even though
not theoretically)
In turn, I would ask that this counter-argument of mine is also
reflected in the PEP.
The whole point of the PEP process is that it settles disputes.
Part of that settling is to avoid arguments which go in circles.
To that effect, the PEP author ideally should *quickly* update
the PEP, along with writing responses, so that anybody repeating
an argument could be pointed to the PEP in order to shut up.
HTH,
Martin
(*) This is also something that Guido requested at some point
from the PEP: that it fairly analyses efficient implementations
of potential wrapper functions, taking C implementations into
account as well.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Am 27.02.2012 22:35, schrieb Armin Ronacher: Hi, On 2/27/12 4:44 PM, mar...@v.loewis.de wrote: Maybe I'm missing something, but there doesn't seem to be a benchmark that measures the 2to3 performance, supporting the claim that it runs two orders of magnitude slower (which I'd interpret as a factor of 100). My Jinja2+Werkzeug's testsuite combined takes 2 seconds to run (Werkzeug actually takes 3 because it pauses for two seconds in a cache expiration test). 2to3 takes 45 seconds to run. And those are small code bases (15K lines combined). I'm not quite able to reproduce that. I don't know how to run the Jinja2 and Werkzeug test suites combined (Werkzeug's setup.py install gives SyntaxError on Python3). So taking Jinja2 alone, this is what I get: - test suite run: 0.86s (python setup.py test) - 2to3 run: 6.7s (python3 setup.py build, using default:3328e388cb28) So this is less than a factor of ten, but more importantly, much shorter than 45s. I also claim that the example is atypical, in that the test suite completes so quickly. Taking distribute 0.6.24 as a counter-example: - test suite run: 9s - 2to3 run: 7s So the test suite runs longer than the build process. Therefore, even a claim In many cases 2to3 runs 20 times slower than the testsuite for the library or application it's testing cannot be substantiated, as cannot the claim This for instance is the case for the Jinja2 library. On the contrary, I'd expect that the build time using 2to3 is significantly shorter than the test suite run times, *in particular* for large projects. For example, for Django, 2to3 takes less than 3 minutes (IIRC), and the test suite runs an hour or so (depending on how many tests get skipped). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 08:51, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Lennart Regebro regebro at gmail.com writes: I'm +1 on the PEP, for reasons already repeated here. We need three types of strings when supporting both Python 2 and Python 3. A binary string, a unicode string and a native string, ie one that is the old 8-bit str in python 2 but a Unicode str in Python 3. Well it's a done deal, and as I said elsewhere on the thread, I wasn't opposing the PEP, but wanting some improvements in it. ISTM that given the PEP as it is, working across 3.2 and 3.3 on a single codebase may not always be the easiest process (IIUC you have to run a mini2to3 process, and it'll need to be cleverer than 2to3 about running over the entire codebase if it's to appear seamless), Distribute helps with this. I think we might have to add a support in distribute to easily exclude the fixer that removes u''-prefixes, I don't remember if there is an exclude feature. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 28.2.2012 01:16, mar...@v.loewis.de wrote: Armin, I propose that you correct the *factual* deficits of the PEP He cannot, because he would have to throw away whole PEP ... it is all based on non-sensical concept of native string. There is no such animal (there are only strings and bytes, although they are incorrectly named Unicode strings and strings in Python 2), and whole PEP is just I don't like Python 3 and I want it to be reverted back to Python 2. It doesn't matter anymore now, but I just needed to put it off my chest. Matěj ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, 28 Feb 2012 10:02:46 +0100 Martin v. Löwis mar...@v.loewis.de wrote: On the contrary, I'd expect that the build time using 2to3 is significantly shorter than the test suite run times, *in particular* for large projects. For example, for Django, 2to3 takes less than 3 minutes (IIRC), and the test suite runs an hour or so (depending on how many tests get skipped). In the end, that's not particularly relevant, because you don't have to run the test suite entirely; when working on small changes, you usually re-run the impacted parts of the test suite until everything goes fine; on the other hand, 2to3 *has* to run on the entire code base. So, really, it's a couple of seconds to run a single bunch of tests vs. several minutes to run 2to3 on the code base. And it's not just the test suite: every concrete experiment with the library you're porting has a serial dependency on running 2to3. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 5:56 PM, Matej Cepl mc...@redhat.com wrote: He cannot, because he would have to throw away whole PEP ... it is all based on non-sensical concept of native string. There is no such animal (there are only strings and bytes, although they are incorrectly named Unicode strings and strings in Python 2), and whole PEP is just I don't like Python 3 and I want it to be reverted back to Python 2. It doesn't matter anymore now, but I just needed to put it off my chest. If you don't know what a native string is, then you need to study more to understand why Armin's PEP exists and why it is useful. I suggest starting with PEP (the WSGI update to v1.0.1 that first clearly defined the concept of a native string: http://www.python.org/dev/peps/pep-/#a-note-on-string-types). There are concrete, practical reasons why the lack of Unicode literals in Python 3 makes porting harder than it needs to be. Are they insurmountable? No, of course not - there are plenty of successful ports already that demonstate porting it quite feasible with existing tools. But the existing approaches require that, in order to be forward compatible with Python 3, a program must be made *worse* in Python 2 (i.e. harder to read and harder to write correctly for someone that hasn't learned Python 3 yet). Restoring unicode literal support in 3.3 is a pragmatic step that allows a lot of code to *just work* on Python 3. Most 2.6+ code that still doesn't work on Python 3 even after this change will be made *better* (or at least not made substantially worse) by the additional changes necessary for forward compatibility. Unicode literals are somewhat unique in their impact on porting efforts, as they show up *everywhere* in Unicode correct code in Python 2. The diffs that will be needed to correctly tag bytestrings in such code under Python 2 are tiny compared to those that would be needed to strip the u prefixes. Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, 28 Feb 2012 21:42:54 +1000 Nick Coghlan ncogh...@gmail.com wrote: But the existing approaches require that, in order to be forward compatible with Python 3, a program must be made *worse* in Python 2 (i.e. harder to read and harder to write correctly for someone that hasn't learned Python 3 yet). Wrong. The separate branches approach allows you to have a clean Python 3 codebase without crippling the Python 2 codebase. Of course that approach was downplayed from the start in favour of using 2to3 on a single codebase, and now we discover that this approach is cumbersome. Note that 2to3 is actually helpful when you choose the dual branches approach, and it isn't a serial dependency in that case. (see https://bitbucket.org/pitrou/t3k/) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Lennart Regebro regebro at gmail.com writes: Distribute helps with this. I think we might have to add a support in distribute to easily exclude the fixer that removes u''-prefixes, I don't remember if there is an exclude feature. We might be at cross purposes here. I don't see how Distribute helps, because the use case I'm talking about is not about distributing or installing stuff, but iteratively changing and testing code which needs to work on 2.6+, 3.2 and 3.3+. If the 2.x code depends on having u'xxx' literals, then 3.2 testing will potentially involve running a fixer on all files in the project every time a change is made, writing to a separate directory, or else a fixer which is integrated into the editing environment so it knows what changed. This is painful, and what motivated PEP 314 in the first place - which seems ironic. The PEP 314 approach seems to assume that that if things work on 3.3, they will work on 3.2/3.1/3.0 without any changes other than replacing u'xxx' with 'xxx'. In other words, you aren't supposed to want to e.g. test 3.2 and 3.3 iteratively, using a workflow which intersperses edits with running tests using 3.2 and running tests with 3.3. In any case, a single code base seems not to be possible across 2.6+/3.0/3.1/3.2/3.3+ using the PEP 314 approach, though of course one will be possible for just 2.6+/3.3+. Early adopters of 3.x seem to be penalised by this approach: I for one will try to use the unicode_literals approach wherever I can. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 9:52 PM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 28 Feb 2012 21:42:54 +1000 Nick Coghlan ncogh...@gmail.com wrote: But the existing approaches require that, in order to be forward compatible with Python 3, a program must be made *worse* in Python 2 (i.e. harder to read and harder to write correctly for someone that hasn't learned Python 3 yet). Wrong. The separate branches approach allows you to have a clean Python 3 codebase without crippling the Python 2 codebase. Of course that approach was downplayed from the start in favour of using 2to3 on a single codebase, and now we discover that this approach is cumbersome. If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. You've once again raised the barrier to entry: either people contribute two patches, or they accept that their patch may languish until someone else writes the patch for the other version. Again, as with 2to3, that approach obviously *works* (we've done it ourselves for years with the standard library), but it's hardly a low friction approach to porting. That's all PEP 414 is about - lowering the friction of porting to Python 3. Is it *necessary*? No, there are already enough successful ports to prove that, if sufficiently motivated, porting to Python 3 is feasible with the current toolset. However, that's the wrong question. The right question is Does PEP 414 make porting substantially *easier*, by significantly reducing the volume of code that needs to change in order to attain Python 3 compatibility?. And the answer to *that* question is Absolutely. Porting the web frameworks themselves to Python 3 is only the first step in migrating those ecosystems to Python 3, and because the web APIs exposed by those frameworks are so heavily Unicode based this is an issue that will hit pretty much every Python web app and library on the planet. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 10:10 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: If the 2.x code depends on having u'xxx' literals, then 3.2 testing will potentially involve running a fixer on all files in the project every time a change is made, writing to a separate directory, or else a fixer which is integrated into the editing environment so it knows what changed. This is painful, and what motivated PEP 314 in the first place - which seems ironic. No, the real idea behind PEP 414 is that most ports that rely on it simply won't support 3.2 - they will only target 3.3+. The u fixer will just be one more tool in the arsenal of those that *do* want to support 3.2 (either because they want to target Ubuntu's LTS 3.2 stack, or for their own reasons). All of the other alternatives (such as separate branches or the unicode_literals future import) will also remain available to them. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. You've once again raised the barrier to entry: either people contribute two patches, or they accept that their patch may languish until someone else writes the patch for the other version. Again that's wrong. If you cleverly use 2to3 to port between branches, patches only have to be written against the 2.x version. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 28 febbraio 2012 13:19, Antoine Pitrou solip...@pitrou.net ha scritto: Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. Would that mean distributing 2 separate tarballs? How would tools such as easy_install and pip work in respect of that? Is there a naming convention they can rely on? --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, Feb 28, 2012 at 10:19 PM, Antoine Pitrou solip...@pitrou.net wrote: Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. Aside from the unicode literal problem, I find that the Python 2.6+/3.2+ subset is still a fairly nice language for an application level web program. Most of the rest of the bytes/text ugliness is hidden away below the framework layer where folks like Chris, Armin and Jacob have to deal with it, but it doesn't affect me as a framework user. You've once again raised the barrier to entry: either people contribute two patches, or they accept that their patch may languish until someone else writes the patch for the other version. Again that's wrong. If you cleverly use 2to3 to port between branches, patches only have to be written against the 2.x version. Apparently *you* know how to do that, but I don't. If I, as a CPython core developer, don't know how to do that, how is it reasonable to expect J. Random Hacker to become a Python 3 porting export? PEP 414 is all about lowering the barrier to entry for successful Python 3 ports. OK, fine some very clever people have invested a lot of time in finding ways to deal with the status quo that make it less painful. That doesn't mean it isn't painful - it just means the early adopters have steeled themselves against the pain and learned to suck it up and cope. Now that we've discovered some of the key sources of pain, we can live with a few pragmatic concessions in the purity of Python 3's language definition to ease the transition for the vast number of Python 3 ports which have yet to begin. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Nick Coghlan ncoghlan at gmail.com writes: On Tue, Feb 28, 2012 at 10:10 PM, Vinay Sajip vinay_sajip at yahoo.co.uk wrote: If the 2.x code depends on having u'xxx' literals, then 3.2 testing will potentially involve running a fixer on all files in the project every time a change is made, writing to a separate directory, or else a fixer which is integrated into the editing environment so it knows what changed. This is painful, and what motivated PEP 314 in the first place - which seems ironic. No, the real idea behind PEP 414 is that most ports that rely on it simply won't support 3.2 - they will only target 3.3+. Well, yes in that the PEP will only be implemented in 3+, but the motivation was to make a single codebase easier to achieve. It does that if you take the narrow view of 2.6+/3.3+, but not if you factor 3.2 into the mix. Maybe 3.2 adoption is too low for us to worry about here, but I for one certainly wish it hadn't been relegated to being a 2nd-class citizen. The u fixer will just be one more tool in the arsenal of those that *do* want to support 3.2 (either because they want to target Ubuntu's LTS 3.2 stack, or for their own reasons). All of the other alternatives (such as separate branches or the unicode_literals future import) will also remain available to them. Right, I get that - as I said, unicode_literals is my preferred path of the options available. It's a shame to see this sort of Balkanisation, though. For example, if Django retains u'xxx' literals (even though I've ported it using unicode_literals, they may choose a different path officially), users wanting to work with it using 2.6/2.7/3.2/3.3 (as I do now) are SOL as far as a single codebase is concerned. Of course, when you're working on your own project, you can call the shots. But problems can arise if you have to work with an external project, as many of us frequently do. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
martin at v.loewis.de writes: A couple of people have said that 'native string' is spelt 'str', but I'm not sure that's the right answer. For example, 2.x's cString.StringIO expects native strings, not Unicode: Your counter-example is non-ASCII characters/bytes. I doubt that this is a valid use case; in a native string, these shouldn't occur (i.e. native strings should always be ASCII), since the semantics of non-ASCII changes drastically between 2.x and 3.x. So whoever defines some API to take native strings can't have defined a valid use of non-ASCII in that interface. It might not be a valid usage, but the 2.x ecosystem has numerous occurrences of invalid usages, which tend to crop up when porting because of 3.x's increased strictness. In the example I gave, cStringIO.StringIO should be able to cope with text strings, but doesn't. Of course there are StringIO.StringIO and io.StringIO in 2.6, but when porting a project, you can't be sure which of these you might run into. Indeed it should. If there is a known application of non-ASCII native strings, I surely would like to know what that is. I can't think of a specific instance off-hand, but I seem to recall having problems with some of the cookie APIs insisting on native strings (rather than text, which is validated against ASCII where appropriate). Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
28.02.12 14:14, Nick Coghlan написав(ла): However, that's the wrong question. The right question is Does PEP 414 make porting substantially *easier*, by significantly reducing the volume of code that needs to change in order to attain Python 3 compatibility?. Another pertinent question: What are disadvantages of PEP 414 is adopted? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Tue, 28 Feb 2012 22:21:11 +1000, Nick Coghlan ncogh...@gmail.com wrote: On Tue, Feb 28, 2012 at 10:10 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: If the 2.x code depends on having u'xxx' literals, then 3.2 testing will potentially involve running a fixer on all files in the project every time a change is made, writing to a separate directory, or else a fixer which is integrated into the editing environment so it knows what changed. This is painful, and what motivated PEP 314 in the first place - which seems ironic. No, the real idea behind PEP 414 is that most ports that rely on it simply won't support 3.2 - they will only target 3.3+. Hmm. It seems to me that this argument implies that PEP 414 is just as likely to *slow down* adoption of Python3 as it is to speed it up, since if this issue is as big a barrier as indicated, many potential porters may choose to wait until OS vendors are supporting 3.3 widely before starting their ports. We are clearly expecting that the reality is that the impact will be at worse neutral, and hopefully positive. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 28/02/2012 14.19, Antoine Pitrou wrote: Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. +10 For every CPython bug that I fix I first apply the patch on 2.7, then on 3.2 and then on 3.3. Most of the time I don't even need to change anything while applying the patch to 3.2, sometimes I have to do some trivial fixes. This is also true for another personal 12kloc project* where I'm using the two-branches approach. For me, the costs of having two branches are: 1) a one-time conversion when the Python3-compatible branch is created (can be done easily with 2to3); 2) merging the fix I apply to the Python2 branch (and with modern DVCS this is not really an issue). With the shared code base approach, the costs are: 1) a one-time conversion to fix the code base and make it run on both 2.x and 3.x; 2) keep using and having to deal with hacks in order to keep it running. With the first approach, you also have two clean and separate code bases, with no hacks; when you stop using Python 2, you end up with a clean Python 3 branch. The one-time conversion also seems easier in the first case. (Note: there are also other costs -- e.g. releasing -- that I haven't considered because they don't affect me personally, but I'm not sure they are big enough to make the two-branches approach worse.) You've once again raised the barrier to entry: either people contribute two patches, or they accept that their patch may languish until someone else writes the patch for the other version. Again that's wrong. If you cleverly use 2to3 to port between branches, patches only have to be written against the 2.x version. After the initial conversion of the code base, the fixes are mostly trivial, so people don't need to write two patches (most of the patches we get for CPython are either against 2.7 or 3.2, and sometimes they even apply clearly to both). Using 2to3 to generate the 3.x code automatically for every change applied to the 2.x branch (or to convert everything when a new package is installed) sounds wrong to me. I wouldn't trust generated code even if 2to3 was a better tool. That said, I successfully used the shared code base approach with print_function, unicode_literals, and a couple of try/except for the imports for a few one-file scripts (for 2.7/3.2) that I wrote recently. TL;DR the two-branches approach usually works better (at least IME) than the shared code base approach, doesn't necessarily require more work, and doesn't need ugly hacks to work. * in this case all the string literals I had were already text (rather than bytes) and even without using unicode_literals they worked out of the box when I moved the code to 3.x. There was however a place where it didn't work, and that turned out to be a bug even in Python 2 because I was mixing bytes and text. Best Regards, Ezio Melotti Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Armin Ronacher armin.ronacher at active-4.com writes:
If by str() you mean using str('x') as replacement for 'x' in both 2.x
and 3.x with __future__ imports as a replacement for native string
literals, please mention why this is better than u(), s(), n() etc. It
would be equally slow than a custom wrapper function and it would not
support non-ascii characters.
Well, you can give it any name you like, but
if PY3:
def n(literal): return literal
else:
# used along with from __future__ import unicode_literals in client code
def n(literal): return literal.encode('utf-8')
will support non-ASCII characters. You have not provided anything other than a
microbenchmark regarding performance - as you are well aware, this does not
illustrate what the performance might be on a representative workload. While
there might be the odd percent in it, I didn't see any major degradation when
running the Django test suite - which I would think is a more balanced workload
than just benchmarking the wrapper. Of course, I don't claim to have studied the
performance characteristics closely - I haven't.
AFAICT, the incidence of native strings in an application is not that great (of
course there can be pathological cases), so the number of calls to n() or
whatever it's called is unlikely to have any significant impact. Even when I was
using u() calls with the 2.5 port - which are of course much more common - the
performance impact was unremarkable.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On Feb 28, 2012, at 10:49 PM, Nick Coghlan wrote: On Tue, Feb 28, 2012 at 10:19 PM, Antoine Pitrou solip...@pitrou.net wrote: Again that's wrong. If you cleverly use 2to3 to port between branches, patches only have to be written against the 2.x version. Apparently *you* know how to do that, but I don't. If I, as a CPython core developer, don't know how to do that, how is it reasonable to expect J. Random Hacker to become a Python 3 porting export? They don't need to, but *we* do, and it's incumbent on us to educate our users. I strongly believe that *now* is the time to be porting to Python 3. It's critical to the long-term health of Python. It's up to us to learn the strategies for accomplishing this, spread the message that it is not only possible, but usually easy (and yes even, from my own experience, fun!). Oh and here's how in three easy steps, 1, 2, 3. I've blogged about my own porting experiences extensively. My strategies may not work for everyone, but they will work for a great many projects. If they work for yours, spread the word. If they don't, figure out something better, write about it, and spread the word. We really need to stop saying that porting to Python 3 is hard, or should be delayed. It's not in the vast majority of cases. Yes, there are warts, and we should continue to improve Python 3 so it gets easier, but by no means is it impossible for most code to be working very nicely on Python 3 today. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Nick Coghlan ncoghlan at gmail.com writes: tools. But the existing approaches require that, in order to be forward compatible with Python 3, a program must be made *worse* in Python 2 (i.e. harder to read and harder to write correctly for someone that hasn't learned Python 3 yet). Restoring unicode literal How so? In the case of string literals, are you saying that it's worse in that you use 'xxx' instead of u'xxx' for text, and have to add a unicode_literals import? I don't feel that either of those make a 2.x program worse. support in 3.3 is a pragmatic step that allows a lot of code to *just work* on Python 3. Most 2.6+ code that still doesn't work on Python 3 even after this change will be made *better* (or at least not made substantially worse) by the additional changes necessary for forward compatibility. Remember, the PEP advocates what it does in the name of a single codebase. If you want to (or have to) support 3.2 in addition to 3.3, 2.6, 2.7, the PEP does not work for you. It only works for you if you're interested in 2.6+ and 3.3+. Unicode literals are somewhat unique in their impact on porting efforts, as they show up *everywhere* in Unicode correct code in Python 2. The diffs that will be needed to correctly tag bytestrings in such code under Python 2 are tiny compared to those that would be needed to strip the u prefixes. But that's a one-time operation using a lib2to3 fixer, and even for a big project like Django, we're not talking about a lot of time spent on this (at least, in my experience). Having a good test suite helps catch those byte-string cases more easily, of course. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Antoine Pitrou solipsis at pitrou.net writes: Wrong. The separate branches approach allows you to have a clean Python 3 codebase without crippling the Python 2 codebase. There may be warts in a single codebase (you usually can't have something for nothing), but it's not necessarily *crippled* when running in 2.x. Of course two branches allow you to have a no-compromise approach for the code style, but you might pay for that in time spent doing merges etc. Note that 2to3 is actually helpful when you choose the dual branches approach, and it isn't a serial dependency in that case. (see https://bitbucket.org/pitrou/t3k/) Yes, 2to3 is very useful when doing an initial porting exercise. I've used it just once in each port I've done. It also works well for a single codebase approach, only I just follow its advice rather than letting it do the conversion automatically. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 28 febbraio 2012 15:20, Ezio Melotti ezio.melo...@gmail.com ha scritto: On 28/02/2012 14.19, Antoine Pitrou wrote: Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : If you're using separate branches, then your Python 2 code isn't being made forward compatible with Python 3. Yes, it avoids making your Python 2 code uglier, but it means maintaining two branches in parallel until you drop Python 2 support. IMO, maintaining two branches shouldn't be much more work than maintaining hacks so that a single codebase works with two different programming languages. +10 For every CPython bug that I fix I first apply the patch on 2.7, then on 3.2 and then on 3.3. Most of the time I don't even need to change anything while applying the patch to 3.2, sometimes I have to do some trivial fixes. This is also true for another personal 12kloc project* where I'm using the two-branches approach. For me, the costs of having two branches are: 1) a one-time conversion when the Python3-compatible branch is created (can be done easily with 2to3); 2) merging the fix I apply to the Python2 branch (and with modern DVCS this is not really an issue). With the shared code base approach, the costs are: 1) a one-time conversion to fix the code base and make it run on both 2.x and 3.x; 2) keep using and having to deal with hacks in order to keep it running. With the first approach, you also have two clean and separate code bases, with no hacks; when you stop using Python 2, you end up with a clean Python 3 branch. The one-time conversion also seems easier in the first case. (Note: there are also other costs -- e.g. releasing -- that I haven't considered because they don't affect me personally, but I'm not sure they are big enough to make the two-branches approach worse.) They are. With that kind of approach you're basically forced to include the python version number as part of the tarball name (e.g. foo-0.3.1-py2.tar.gz and foo-0.3.1-py3.tar.gz). Just to name one, that means foo can't be installed via pip/easy_install. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Serhiy Storchaka storchaka at gmail.com writes: Another pertinent question: What are disadvantages of PEP 414 is adopted? It's moot, but as I see it: the purpose of PEP 414 is to facilitate a single codebase across 2.x and 3.x. However, it only does this if your 3.x interest is 3.3+. If you also want to or need to support 3.0 - 3.2, it makes your workflow more painful, because you can't run tests on 2.x or 3.3 and then run them on 3.2 without an intermediate source conversion step - just like the 2to3 step that people find painful when it's part of maintenance workflow, and which in part prompted the PEP in the first place. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Vinay Sajip wrote: Serhiy Storchaka storchaka at gmail.com writes: Another pertinent question: What are disadvantages of PEP 414 is adopted? It's moot, but as I see it: the purpose of PEP 414 is to facilitate a single codebase across 2.x and 3.x. However, it only does this if your 3.x interest is 3.3+. If you also want to or need to support 3.0 - 3.2, it makes your workflow more painful, because you can't run tests on 2.x or 3.3 and then run them on 3.2 without an intermediate source conversion step - just like the 2to3 step that people find painful when it's part of maintenance workflow, and which in part prompted the PEP in the first place. I don't think it's fair to say it makes it *more* painful. Fair to say it doesn't make it less painful, but adding u'' to 3.3+ doesn't make it harder to port from 2.x to 3.1+. You're merely no better off with it than without it. Aside: in my opinion, people shouldn't actively support 3.0, or at least not advertise support for it, as it was end-of-lifed on the release of 3.1. As I see it, it is best to pretend that 3.0 never existed :) -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Ezio Melotti ezio.melotti at gmail.com writes: For every CPython bug that I fix I first apply the patch on 2.7, then on 3.2 and then on 3.3. Most of the time I don't even need to change anything while applying the patch to 3.2, sometimes I have to do some trivial fixes. This is also true for another personal 12kloc project* where I'm using the two-branches approach. I hear what you say about the personal project, but IMO CPython is atypical (as far as this discussion is concerned), not least because it's not a pure-Python project. For me, the costs of having two branches are: 1) a one-time conversion when the Python3-compatible branch is created (can be done easily with 2to3); Yes, but the amount of ease is project-dependent. For example, 2to3 wraps values() method calls with list(), which is a reasonable thing to do for dicts; when presented Django's querysets, which have a values() method which should not be wrapped, then you have to go through and sort things out. I'm not knocking 2to3, which I think is great. Just that things go well sometimes, and less well at other times, With the shared code base approach, the costs are: 1) a one-time conversion to fix the code base and make it run on both 2.x and 3.x; 2) keep using and having to deal with hacks in order to keep it running. Which hacks do you mean, if you're only interested in 2.6+? With the first approach, you also have two clean and separate code bases, with no hacks; when you stop using Python 2, you end up with a clean Python 3 branch. The one-time conversion also seems easier in the first case. (Note: there are also other costs -- e.g. releasing -- that I haven't considered because they don't affect me personally, but I'm not sure they are big enough to make the two-branches approach worse.) I don't believe there's a one-size-fits-all. The two branches approach is appealing, and I have no quarrel with it: but I contend that big projects like Django would be reluctant to switch, or take much longer to switch to 3.x, if they had to maintain separate branches. Given the size of their user community, they have to follow strict release procedures, which (even with just running on 2.x) smaller projects can be more relaxed about. You forgot to mention the part which is most time-consuming day-to-day: making changes and testing. For the two-branch approach, its 1. Change on 2.x 2. Test on 2.x 3. Commit on 2.x 4. Merge to 3.x 5. Possibly change on 3.x 6. Test on 3.x 7. Commit on 3.x where each test step, if failures occur, might take you back to a previous change step. For the single codebase, that's 1. Change 2. Test on 2.x 3. Test on 3.x 4. Commit This, to me, is the single big advantage of the single codebase approach, and the productivity improvements outweigh code purity issues which are, in the grand scheme of things, not all that large. Another advantage is DRY: you don't have to worry about forgetting to merge some changes from 2.x to 3.x. Haven't we all been there one time or another? I know I have, though I try not to make a habit of it ;-) After the initial conversion of the code base, the fixes are mostly trivial, so people don't need to write two patches (most of the patches we get for CPython are either against 2.7 or 3.2, and sometimes they even apply clearly to both). Fixes may be trivial, but new features might not always be so. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
Hi Armin, Could you please remove from the PEP the following statement: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org. The python ecosystem is not just limited to WSGI apps, Django and Flask. Yes, we don't have all the packages on pypi support python 3, but many of those are portable within 10 minutes to couple of hours of work (and I did many of such ports for our internal systems.) And many of the essential packages do exist for python 3, like numpy, zeromq etc. I know several sturt-ups, including mine that develop huge commercial applications entirely on python 3. Thanks, -Yury On 2012-02-27, at 5:38 PM, Armin Ronacher wrote: Hi, On 2/27/12 10:18 PM, Terry Reedy wrote: I would like to know if you think that this one change is enough to do agile development and testing, etc, or whether, as Chris McDonough hopes, this is just the first of a series of proposals you have planned. Indeed I have three other PEPs in the work. The reintroduction of except (((ExceptionType),),), the comparision operator and the removal of nonlocal, the latter to make Python 2.x developers feel better about themselves. :-) Regards, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Steven D'Aprano steve at pearwood.info writes: I don't think it's fair to say it makes it *more* painful. Fair to say it doesn't make it less painful, but adding u'' to 3.3+ doesn't make it harder to port from 2.x to 3.1+. You're merely no better off with it than without it. No, it actually does make it *more* painful in some scenarios. Let's say Django decides to move to 3.x using a single codebase starting with 3.3, using PEP 414 to avoid changing u'xxx' in their source code. This is dandy for 3.3, and say I have to work with Django on 2.6, 2.7 and 3.3. Great - I make some changes, I run tests on 2.x, 3.3 - make changes as needed to fix failures, then commit. And on to the next set of changes. Now, suppose I also need to support 3.2, in addition to the other versions. I don't get the same easy workflow I had before: for 3.2, I have to run Armin's hook to remove the u'' prefixes between making changes and running tests, *every time*, but the output will be written to a separate directory, and I may have to maintain a separate test environment there in terms of test data files etc. It's exactly the complaint the PEP makes about having to have 2to3 in the workflow, and how that hurts your productivity! Though the experience may differ in degree because Armin's tool is faster, it's not going to make for a seamless workflow. Especially not if it has to run over all the files in the Django codebase. And if it's going to know only which files have changed and run only on those, how does it propose to do that, independently of my editing tools? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414
On Feb 28, 2012, at 11:29 AM, Yury Selivanov wrote: Could you please remove from the PEP the following statement: As it stands, Python 3 is currently a bad choice for long-term investments, since the ecosystem is not yet properly developed, and libraries are still fighting with their API decisions for Python 3. While it may be as such for you, I think it is incorrect for the rest. Moreover, it is harmful for the python 3 adoption, to put such documents on python.org. +∞ -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
In the end, that's not particularly relevant, because you don't have to run the test suite entirely; when working on small changes, you usually re-run the impacted parts of the test suite until everything goes fine; on the other hand, 2to3 *has* to run on the entire code base. Not at all. If you are working on the code, 2to3 only needs to run on the parts of the code that you changed, since the unmodified parts will not need to be re-transformed using 2to3. So, really, it's a couple of seconds to run a single bunch of tests vs. several minutes to run 2to3 on the code base. Not in my experience. The incremental run-time of 2to3 after a single change is in the order of fractions of a second. And it's not just the test suite: every concrete experiment with the library you're porting has a serial dependency on running 2to3. Therefore, your build process should support incremental changes. Fortunately, distribute does support this approach. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 28/02/2012 18.08, Vinay Sajip wrote:
Ezio Melottiezio.melottiat gmail.com writes:
For every CPython bug that I fix I first apply the patch on 2.7, then on
3.2 and then on 3.3.
Most of the time I don't even need to change anything while applying the
patch to 3.2, sometimes I have to do some trivial fixes. This is also
true for another personal 12kloc project* where I'm using the
two-branches approach.
I hear what you say about the personal project, but IMO CPython is atypical (as
far as this discussion is concerned), not least because it's not a pure-Python
project.
Most of the things I fix are pure Python, I wasn't considering the C
patches and doc fixes here.
For me, the costs of having two branches are:
1) a one-time conversion when the Python3-compatible branch is created
(can be done easily with 2to3);
Yes, but the amount of ease is project-dependent. For example, 2to3 wraps
values() method calls with list(), which is a reasonable thing to do for dicts;
when presented Django's querysets, which have a values() method which should not
be wrapped, then you have to go through and sort things out. I'm not knocking
2to3, which I think is great. Just that things go well sometimes, and less well
at other times,
With the personal project this is what I did:
1) make a separate branch;
2) run 2to3 and let it overwrite the file;
3) review the changes as I would do with any other patch before
committing;
4) fix things that 2to3 missed and other minor glitches;
5) fix a few bugs that surfaced after the port (and were in the
original code too);
The fixes made by 2to3 were mostly:
* removing u'' from strings;
* renaming imports, methods (like the .iteritems);
* adding 'as' in the excepts;
* adding () for a few prints;
These changes affected about 500 lines of code (out of 12kloc).
The changes I did manually after running 2to3 were (some where not
strictly necessary):
* removing 'object' from classes;
* removing ord() in a few places;
* removing the content of super(...);
* removing codecs.open() and use open() instead;
* removing a few .decode('utf-8');
* adding a couple of b'';
After a couple of days almost everything was working fine.
With the shared code base approach, the costs are:
1) a one-time conversion to fix the code base and make it run on
both 2.x and 3.x;
2) keep using and having to deal with hacks in order to keep it running.
Which hacks do you mean, if you're only interested in 2.6+?
Things like try/except for names that changed and wrappers for
bytes/strings.
Of course the situation is worse for projects that have to support
earlier versions.
With the first approach, you also have two clean and separate code
bases, with no hacks; when you stop using Python 2, you end up with a
clean Python 3 branch.
The one-time conversion also seems easier in the first case.
(Note: there are also other costs -- e.g. releasing -- that I haven't
considered because they don't affect me personally, but I'm not sure
they are big enough to make the two-branches approach worse.)
I don't believe there's a one-size-fits-all. The two branches approach is
appealing, and I have no quarrel with it: but I contend that big projects like
Django would be reluctant to switch, or take much longer to switch to 3.x, if
they had to maintain separate branches.
I would actually feel safer doing the port in a separate branch and keep
it there.
Changing all the code in the main branch just to make it work for 3.x
too doesn't strike like a really good idea to me.
Given the size of their user community,
they have to follow strict release procedures, which (even with just running on
2.x) smaller projects can be more relaxed about.
I don't have much experience regarding releases, but developing on a
separate branch shouldn't affect the release of the 2.x version. The
developers will have to merge the changes to the py3 branch too, and
eventually they will be able to ship an additional release for py3 too.
Sure, there's more work for the developers, but that's no news.
You forgot to mention the part which is most time-consuming day-to-day: making
changes and testing. For the two-branch approach, its
1. Change on 2.x
2. Test on 2.x
3. Commit on 2.x
4. Merge to 3.x
5. Possibly change on 3.x
6. Test on 3.x
7. Commit on 3.x
where each test step, if failures occur, might take you back to a previous
change step.
For the single codebase, that's
1. Change
2. Test on 2.x
3. Test on 3.x
4. Commit
And if something fails here, you will have to repeat both step 2 and 3,
until you get it right for both at the same time.
The step 1 of the single codebase is in the end more or less equivalent
to steps 1+4+5, just in a different way. The remaining extra commit
takes no time, and since the branches are independent, if you find a
problem with py3 you don't have to run the test suite for 2.x again.
In my experience with CPython, the most time-consuming part is
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
On 2/28/2012 7:10 AM, Vinay Sajip wrote: The PEP 314 approach seems to assume that that if things work on 3.3, they will work on 3.2/3.1/3.0 without any changes other than replacing u'xxx' with 'xxx'. (Delete 3.0. 3.1 is also less of a concern.) It actually assumes that if things work on 3.3 *and* 2.7 (or .6), then ... . At first glance, this seems reasonable. If the code works on 2.7, then it does not use any new 3.3 features. Nor does it depend on any 3.3-only bug fixes that were part of a feature patch. 2.6, of course, is essentially not getting any bugfixes. In other words, you aren't supposed to want to e.g. test 3.2 and 3.3 iteratively, using a workflow which intersperses edits with running tests using 3.2 and running tests with 3.3. Anyone who is also targeting 3.2 could run a test32 script whenever they need to take a break. Or it could be run in the background (perhaps on a different core) while editing continues. People will work this out on a project by project basis, or use one of the other solutions. In any case, a single code base seems not to be possible across 2.6+/3.0/3.1/3.2/3.3+ using the PEP 314 approach, though of course one will be possible for just 2.6+/3.3+. Early adopters of 3.x seem to be penalised by this approach: I for one will try to use the unicode_literals approach wherever I can. Early adoption of new tech typically has costs as well as benefits ;-). -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 414 - Unicode Literals for Python 3
In http://mail.python.org/pipermail/python-dev/2012-February/117070.html Vinay Sajip wrote: It's moot, but as I see it: the purpose of PEP 414 is to facilitate a single codebase across 2.x and 3.x. However, it only does this if your 3.x interest is 3.3+ For many people -- particularly those who haven't ported yet -- 3.x will mean 3.3+. There are some who will support 3.2 because it is a LTS release on some distribution, just as there were some who supported Python 1.5 (but not 1.6) long into the 2.x cycle, but I expect them to be the minority. I certainly don't expect 3.2 to remain a primary development target, the way that 2.7 is. IIRC, the only ways to use 3.2 even today are: (a) Make an explicit choice to use something other than the default (b) Download directly and choose 3.x without OS support (c) Use Arch Linux These are the sort of people who can be expected to upgrade. Now also remember that we're talking specifically about projects that have *not* been ported to 3.x (== no existing users to support), and that won't be ported until 3.2 is already in maintenance mode. If you also want to or need to support 3.0 - 3.2, it makes your workflow more painful, Compared to dropping 3.2, yes. Compared to supporting 3.2 today? I don't see how. because you can't run tests on 2.x or 3.3 and then run them on 3.2 without an intermediate source conversion step - just like the 2to3 step that people find painful when it's part of maintenance workflow, and which in part prompted the PEP in the first place. So the only differences compared to today are that: (a) Fewer branches are after the auto-conversion. (b) No current branches are after the auto-conversion. (c) The auto-conversion is much more limited in scope. -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
