Re: [Python-Dev] str.format implementation
On Dec 12, 2011, at 7:56 PM, Ben Wolfson wrote: I personally would prefer (1) to (2) or (3), and (3) to (2), had I my druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of them to nothing (or to changing the docs to reflect the current batty behavior). +1 on changing the batty behavior. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str.format implementation
On Tue, Dec 13, 2011 at 6:37 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On Dec 12, 2011, at 7:56 PM, Ben Wolfson wrote: I personally would prefer (1) to (2) or (3), and (3) to (2), had I my druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of them to nothing (or to changing the docs to reflect the current batty behavior). +1 on changing the batty behavior. Skimming my comments from last time this came up, +1 on just going with what the docs say. The PEP underspecified it, so taking the docs as the spec for this aspect seems like a reasonable course of action. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str.format implementation
Ben Wolfson wrote:
Hi,
I'm hoping to get some kind of consensus about the divergences between
the implementation and documentation of str.format
(http://mail.python.org/pipermail/python-dev/2011-June/111860.html and
the linked bug report contain examples of the divergences). These
pertain to the arg_name, attribute_name, and element_index fields of
the grammar in the docs:
replacement_field ::= { [field_name] [! conversion] [:
format_spec] }
field_name::= arg_name (. attribute_name | [
element_index ])*
arg_name ::= [identifier | integer]
attribute_name::= identifier
element_index ::= integer | index_string
index_string ::= any source character except ] +
Nothing definitive emerged from the last round of discussion, and as
far as I can recall there are now three proposals for what kind of
changes might be worth making:
(1) the implementation should conform to the docs;*
(2) like (1) with the change that element_index should be changed to
integer | identifier (rendering index_string otiose);
(3) like (1) with the change that index_string should be changed to
'any source character except ], }, or {'.
* the docs link integer to
http://docs.python.org/reference/lexical_analysis.html#grammar-token-integer
but the current implementation only allows decimal integers, which
seems reasonable and worth retaining.
(2) was suggested by Greg Ewing on python-dev and (3) by Petri
Lehtinen in the bug report. (Petri actually suggested that braces be
disallowed except for the nesting in the format_spec, but it comes to
the same thing.)
None of these should be difficult to implement; patches exist for (1)
and (2). (2) and (3) would lead to format strings that are easier to
for the programmer to visually parse; (1) would make the indexing part
of the replacement field conform more closely to the way indexing with
strings behaves in Python generally, where arbitrary strings can be
used. (It wouldn't conform exactly, obviously, since ']' would still
be excluded.)
I personally would prefer (1) to (2) or (3), and (3) to (2), had I my
druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of
them to nothing (or to changing the docs to reflect the current batty
behavior).
+1 for changing. And as I've said before, I prefer proposal (3).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] IEEE/ISO draft on Python vulnerabilities
2011/12/12 Victor Stinner victor.stin...@haypocalc.com When sorting a list using the sort() method, attempting to inspect or mutate the content of the list will result in undefined behaviour. But is this even true? in listobject.c::listsort(), since 2002, /* The list is temporarily made empty, so that mutations performed * by comparison functions can't affect the slice of memory we're * sorting (allowing mutations during sorting is a core-dump * factory, since ob_item may change). */ So behaviour is not undefined at all... maybe this report is only based on note #10 of the documentation: http://docs.python.org/library/stdtypes.html#mutable-sequence-types and only considers python 2.2 or older... -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] IEEE/ISO draft on Python vulnerabilities
Hi, On Tue, Dec 13, 2011 at 11:37, Amaury Forgeot d'Arc amaur...@gmail.com wrote: When sorting a list using the sort() method, attempting to inspect or mutate the content of the list will result in undefined behaviour. (...) So behaviour is not undefined at all... No, the behavior _is_ undefined. The comment you cited says that it cannot crash the Python interpreter; additionally, it makes a best-effort attempt at catching such accesses and raising ValueError. But I think I can build a strange-looking example where you mutate a list during sorting and don't get a ValueError (although admittedly it needs a lot of hacking to do that nowadays, e.g. multiple threads). A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] IEEE/ISO draft on Python vulnerabilities
2011/12/13 Armin Rigo ar...@tunes.org No, the behavior _is_ undefined. The comment you cited says that it cannot crash the Python interpreter; additionally, it makes a best-effort attempt at catching such accesses and raising ValueError. But I think I can build a strange-looking example where you mutate a list during sorting and don't get a ValueError (although admittedly it needs a lot of hacking to do that nowadays, e.g. multiple threads). I'm interested to see how! The current implementation installs an empty array in the list, and the initial array is only held by a local variable in listsort(). even gc.get_referrers() can return the empty list... -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough chr...@plope.com wrote: On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote: As someone who ported WebOb and other stuff built on top of it to Python 3 without using from __future__ import unicode_literals, I'm kinda sad that to be using best practice I'll have to go back and flip the polarity on everything. Eh? If you don't need unicode_literals, what's the problem? Porting the WebOb code sucked. It's only about 5K lines of code but the porting effort took me about 80 hours. Some of the problem is certainly my own idiocy, but some of it is just because straddling code across Python 2 and Python 3 currently requires that you change lots and lots of code for suspect benefit. Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? That would seem to avoid most of the drawbacks to the current 2to3 approach. Laurence ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 13/12/2011 13:33, Laurence Rowe wrote: On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough chr...@plope.com wrote: On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote: As someone who ported WebOb and other stuff built on top of it to Python 3 without using from __future__ import unicode_literals, I'm kinda sad that to be using best practice I'll have to go back and flip the polarity on everything. Eh? If you don't need unicode_literals, what's the problem? Porting the WebOb code sucked. It's only about 5K lines of code but the porting effort took me about 80 hours. Some of the problem is certainly my own idiocy, but some of it is just because straddling code across Python 2 and Python 3 currently requires that you change lots and lots of code for suspect benefit. Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? That would seem to avoid most of the drawbacks to the current 2to3 approach. I'm not sure what you mean, but it *reads* as if you mean a version of 2to3 that only converts code that doesn't need converting. Could you clarify? Thanks, Michael Laurence ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
Input = normal 2.x code; Output = code that runs on both 2.x and 3.x. That is, tinkering with what 2to3 produces, not what it accepts. -- Nick Coghlan (via Gmail on Android, so likely to be more terse than usual) On Dec 13, 2011 11:46 PM, Michael Foord fuzzy...@voidspace.org.uk wrote: On 13/12/2011 13:33, Laurence Rowe wrote: On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough chr...@plope.com wrote: On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote: As someone who ported WebOb and other stuff built on top of it to Python 3 without using from __future__ import unicode_literals, I'm kinda sad that to be using best practice I'll have to go back and flip the polarity on everything. Eh? If you don't need unicode_literals, what's the problem? Porting the WebOb code sucked. It's only about 5K lines of code but the porting effort took me about 80 hours. Some of the problem is certainly my own idiocy, but some of it is just because straddling code across Python 2 and Python 3 currently requires that you change lots and lots of code for suspect benefit. Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? That would seem to avoid most of the drawbacks to the current 2to3 approach. I'm not sure what you mean, but it *reads* as if you mean a version of 2to3 that only converts code that doesn't need converting. Could you clarify? Thanks, Michael Laurence __**_ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/**mailman/listinfo/python-devhttp://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/**mailman/options/python-dev/** fuzzyman%40voidspace.org.ukhttp://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/**different.htmlhttp://www.sqlite.org/different.html __**_ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/**mailman/listinfo/python-devhttp://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/**mailman/options/python-dev/** ncoghlan%40gmail.comhttp://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 13/12/2011 14:24, Nick Coghlan wrote: Input = normal 2.x code; Output = code that runs on both 2.x and 3.x. That is, tinkering with what 2to3 produces, not what it accepts. How is that different from what 2to3 currently does? Are you agreeing with Laurence, suggesting an alternative, or something else? Michael -- Nick Coghlan (via Gmail on Android, so likely to be more terse than usual) On Dec 13, 2011 11:46 PM, Michael Foord fuzzy...@voidspace.org.uk mailto:fuzzy...@voidspace.org.uk wrote: On 13/12/2011 13:33, Laurence Rowe wrote: On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough chr...@plope.com mailto:chr...@plope.com wrote: On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote: As someone who ported WebOb and other stuff built on top of it to Python 3 without using from __future__ import unicode_literals, I'm kinda sad that to be using best practice I'll have to go back and flip the polarity on everything. Eh? If you don't need unicode_literals, what's the problem? Porting the WebOb code sucked. It's only about 5K lines of code but the porting effort took me about 80 hours. Some of the problem is certainly my own idiocy, but some of it is just because straddling code across Python 2 and Python 3 currently requires that you change lots and lots of code for suspect benefit. Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? That would seem to avoid most of the drawbacks to the current 2to3 approach. I'm not sure what you mean, but it *reads* as if you mean a version of 2to3 that only converts code that doesn't need converting. Could you clarify? Thanks, Michael Laurence ___ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, 13 Dec 2011 14:42:12 +0100, Michael Foord fuzzy...@voidspace.org.uk wrote: On 13/12/2011 13:33, Laurence Rowe wrote: On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough chr...@plope.com wrote: On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote: As someone who ported WebOb and other stuff built on top of it to Python 3 without using from __future__ import unicode_literals, I'm kinda sad that to be using best practice I'll have to go back and flip the polarity on everything. Eh? If you don't need unicode_literals, what's the problem? Porting the WebOb code sucked. It's only about 5K lines of code but the porting effort took me about 80 hours. Some of the problem is certainly my own idiocy, but some of it is just because straddling code across Python 2 and Python 3 currently requires that you change lots and lots of code for suspect benefit. Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? That would seem to avoid most of the drawbacks to the current 2to3 approach. I'm not sure what you mean, but it *reads* as if you mean a version of 2to3 that only converts code that doesn't need converting. Could you clarify? The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. If I understand correctly, Chris' experience of porting WebOb was that there is a large amount of manual work required in this approach in part because of the many u'' strings in libraries that extensively use unicode. It should be possible to automate this with the same approach as 2to3, but instead of a transform from 2-3 it would transform code from 2-(2 3). In this case the transform would only have to be run once (rather than on every setup.py install) and would avoid the difficulties of debugging with tracebacks that do not exactly match the source code. Laurence ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 13/12/2011 14:28, Laurence Rowe wrote: On Tue, 13 Dec 2011 14:42:12 +0100, Michael Foord fuzzy...@voidspace.org.uk wrote: On 13/12/2011 13:33, Laurence Rowe wrote: On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough chr...@plope.com wrote: On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote: As someone who ported WebOb and other stuff built on top of it to Python 3 without using from __future__ import unicode_literals, I'm kinda sad that to be using best practice I'll have to go back and flip the polarity on everything. Eh? If you don't need unicode_literals, what's the problem? Porting the WebOb code sucked. It's only about 5K lines of code but the porting effort took me about 80 hours. Some of the problem is certainly my own idiocy, but some of it is just because straddling code across Python 2 and Python 3 currently requires that you change lots and lots of code for suspect benefit. Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? That would seem to avoid most of the drawbacks to the current 2to3 approach. I'm not sure what you mean, but it *reads* as if you mean a version of 2to3 that only converts code that doesn't need converting. Could you clarify? The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. If I understand correctly, Chris' experience of porting WebOb was that there is a large amount of manual work required in this approach in part because of the many u'' strings in libraries that extensively use unicode. It should be possible to automate this with the same approach as 2to3, but instead of a transform from 2-3 it would transform code from 2-(2 3). In this case the transform would only have to be run once (rather than on every setup.py install) and would avoid the difficulties of debugging with tracebacks that do not exactly match the source code. Ah, you mean a 2toPython3compatible2 converter. Not a bad idea. Michael Laurence ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
Laurence Rowe l at lrowe.co.uk writes:
The approach that most people seem to have settled on for porting
libraries to Python 3 is to make a single codebase that is compatible with
both Python 2 and Python 3, perhaps making use of the six library. If I
understand correctly, Chris' experience of porting WebOb was that there is
a large amount of manual work required in this approach in part because of
the many u'' strings in libraries that extensively use unicode. It should
be possible to automate this with the same approach as 2to3, but instead
of a transform from 2-3 it would transform code from 2-(2 3). In this
case the transform would only have to be run once (rather than on every
setup.py install) and would avoid the difficulties of debugging with
tracebacks that do not exactly match the source code.
I started writing a tool today, tentatively called '2to23', which aims to do
this. It's basically 2to3, but with a package of custom fixers in a package
'lib2to23.fixers' adapted from the corresponding fixers in lib2to3. It's
experimental work in progress at the moment. With a sample file like
import anything
import dummy
class CustomException(Exception):
pass
def func1():
a = u'abc'
b = b'def'
c = 'unchanged'
c1 = u'abc' u'def'
def func2():
try:
d = 5L
e = (int, long)
f = (long, int)
g = func3()
if isinstance(g, basestring):
print 'a string'
elif isinstance(g, bytes):
print 'some bytes'
elif isinstance(g, unicode):
print 'a unicode string'
else:
print
for i in xrange(3):
pass
except Exception:
e = sys.exc_info()
raise CustomException, e[1], e[2]
class BaseClass:
pass
class OtherBaseClass:
pass
class MetaClass:
pass
class DerivedClass(BaseClass, OtherBaseClass):
__metaclass__ = MetaClass
2to23 gives the following suggested changes:
--- sample.py (original)
+++ sample.py (refactored)
@@ -1,34 +1,41 @@
import anything
import dummy
+from django.utils.py3 import long_type
+from django.utils.py3 import string_types
+from django.utils.py3 import binary_type
+from django.utils.py3 import b
+from django.utils.py3 import text_type
+from django.utils.py3 import u
+from django.utils.py3 import xrange
class CustomException(Exception):
pass
def func1():
-a = u'abc'
-b = b'def'
+a = u('abc')
+b = b('def')
c = 'unchanged'
-c1 = u'abc' u'def'
+c1 = u('abc') u('def')
def func2():
try:
-d = 5L
+d = long_type(5)
e = (int, long)
f = (long, int)
g = func3()
-if isinstance(g, basestring):
-print 'a string'
-elif isinstance(g, bytes):
-print 'some bytes'
-elif isinstance(g, unicode):
-print 'a unicode string'
+if isinstance(g, string_types):
+print('a string')
+elif isinstance(g, binary_type):
+print('some bytes')
+elif isinstance(g, text_type):
+print('a unicode string')
else:
-print
+print()
for i in xrange(3):
pass
except Exception:
e = sys.exc_info()
-raise CustomException, e[1], e[2]
+raise CustomException(e[1]).with_traceback(e[2])
class BaseClass:
pass
@@ -39,8 +46,8 @@
class MetaClass:
pass
-class DerivedClass(BaseClass, OtherBaseClass):
-__metaclass__ = MetaClass
+class DerivedClass(with_metaclass(MetaClass, BaseClass, OtherBaseClass)):
+pass
As you can see, there's still a bit of work to do, and the sample doesn't cover
all use cases yet. I'll be cross-checking it using my recent Django porting work
to confirm that it covers everything at least needed for that port, which is why
the fixers currently generate imports from django.utils.py3. Obviously, I'll
change this in due course.
Regards,
Vinay Sajip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, 13 Dec 2011 15:28:31 +0100 Laurence Rowe l...@lrowe.co.uk wrote: The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. Do you have evidence that most people have settled on that approach? (besides the couple of library writers who have commented on this thread) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Dec 13, 2011, at 05:24 PM, Antoine Pitrou wrote: On Tue, 13 Dec 2011 15:28:31 +0100 Laurence Rowe l...@lrowe.co.uk wrote: The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. Do you have evidence that most people have settled on that approach? (besides the couple of library writers who have commented on this thread) I'm not sure there's any settling at all when it comes to Python 3 porting yet. ;) Sometimes, one code base works better, other times 2to3 works well. I tend to use the latter on pure-Python setuptools-based projects, and the former on projects with C extensions, autoconf-based libraries. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, Dec 13, 2011 at 14:33, Laurence Rowe l...@lrowe.co.uk wrote: Could this manual work be cut down if there was a version of 2to3 that targeted the subset of the language that is compatible with both 2 and 3? Not really, but a 2to6, ie something that tries to keep Python 2 compatibility by using the six library, might be useful. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, Dec 13, 2011 at 11:24 AM, Antoine Pitrou solip...@pitrou.netwrote: On Tue, 13 Dec 2011 15:28:31 +0100 Laurence Rowe l...@lrowe.co.uk wrote: The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. Do you have evidence that most people have settled on that approach? (besides the couple of library writers who have commented on this thread) I've seen more projects doing it that way than maintaining dual code bases. In retrospect, it seems way more attractive than having to run a converter all the time, especially if I could run a 2to6 tool *once* and then simply write new code using six-isms Among other things, it means that: * There's only one codebase * If the conversion isn't perfect, you only have to fix it once * Line numbers are the same * There's no conversion step slowing down development So, I expect that if the approach is at all viable, it'll quickly become the One Obvious Way to do it. In effect, 2to3 is a purity solution, but six is more like a practicality solution. And if there's official support for it, so much the better. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str.format implementation
On 12/12/2011 10:56 PM, Ben Wolfson wrote:
Hi,
I'm hoping to get some kind of consensus about the divergences between
the implementation and documentation of str.format
(http://mail.python.org/pipermail/python-dev/2011-June/111860.html and
the linked bug report contain examples of the divergences). These
pertain to the arg_name, attribute_name, and element_index fields of
the grammar in the docs:
replacement_field ::= { [field_name] [! conversion] [:
format_spec] }
field_name::= arg_name (. attribute_name | [
element_index ])*
arg_name ::= [identifier | integer]
attribute_name::= identifier
element_index ::= integer | index_string
index_string ::= any source character except ] +
Nothing definitive emerged from the last round of discussion, and as
far as I can recall there are now three proposals for what kind of
changes might be worth making:
(1) the implementation should conform to the docs;*
(2) like (1) with the change that element_index should be changed to
integer | identifier (rendering index_string otiose);
I've now learned what otiose means. Thanks!
(3) like (1) with the change that index_string should be changed to
'any source character except ], }, or {'.
This is still on my plate. I just haven't had a lot of Python time
recently. But I do plan to address this.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 12/13/2011 2:02 PM, PJ Eby wrote: On Tue, Dec 13, 2011 at 11:24 AM, Antoine Pitrou solip...@pitrou.net mailto:solip...@pitrou.net wrote: On Tue, 13 Dec 2011 15:28:31 +0100 Laurence Rowe l...@lrowe.co.uk mailto:l...@lrowe.co.uk wrote: The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. Do you have evidence that most people have settled on that approach? (besides the couple of library writers who have commented on this thread) I think there is clearly enough 'some' people to justify official support of a 2to23 (or more obscurely, 2to6, but I just got the point that 6=2*3). Beyond that, we don't know and don't need to know. I've seen more projects doing it that way than maintaining dual code bases. In retrospect, it seems way more attractive than having to run a converter all the time, especially if I could run a 2to6 tool *once* and then simply write new code using six-isms Among other things, it means that: * There's only one codebase * If the conversion isn't perfect, you only have to fix it once * Line numbers are the same * There's no conversion step slowing down development So, I expect that if the approach is at all viable, it'll quickly become the One Obvious Way to do it. In effect, 2to3 is a purity solution, but six is more like a practicality solution. 2to3 is the practical solution for someone converting private Python 2 code to run on Python 3 *instead* of Python 2, without looking back. By the nature of things, such conversions will be private and scattered over the next decade or so. If 2to3 works well, we will never hear about them, except for the rare praise. Ditto for public code whose author wishes to abandon Py 2. But that seems to rare so far. So we are really talking about upgrading public libraries and apps to work with Python 3 *as well as* 'recent' Python 2 versions. For that, a 'Python23' bridge seems to work for some. Looking ahead, there will in the future be a need for a 23to3 converter to remove the then extraneous bridge code. But that will need a semi-standard 'Python23' to work from. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions
On Tue, Dec 13, 2011 at 2:55 AM, Martin v. Löwis mar...@v.loewis.de wrote: (1) Why is PyObject_HEAD used instead of PyObject_VAR_HEAD? The unicode object is not a var object. In a var object, tp_itemsize gives the element size, which is not possible for unicode objects, since the itemsize may vary by instance. In addition, not all instances have the items after the base object (plus the size of the base object in tp_basicsize is also not always correct). That makes perfect sense. Any chance of adding the rationale to the code? Either inline, such as changing unicodeobject.h line 291 from PyObject_HEAD to something like: PyObject_HEAD /* Not VAR_HEAD, because tp_itemsize varies, and data may be elsewhere. */ or in the large comments around line 288: Note that Strings use PyObject_HEAD and a length field instead of PyObject_VAR_HEAD, because the tp_itemsize varies by instance, and the actual data is not always immediately after the PyASCIIObject header. (2) Why does PyASCIIObject have a wstr member, and why does PyCompactUnicodeObject have wstr_length? As best I can tell from the PEP or header file, wstr is only meaningful when either: No. wstr is most of all relevant if someone calls PyUnicode_AsUnicode(AndSize); any unicode object might get the wstr pointer filled out at some point. I am willing to believe that requests for a wchar_t (or utf-8 or System Locale charset) representation are common enough to justify caching the data after the first request. But then why throw it away in the first place? Wouldn't programs that create unicode from wchar_t data also be the most likely to request wchar_t data back? wstr_length is only relevant if wstr is not NULL. For a pure ASCII string (and also for Latin-1 and other BMP strings), the wstr length will always equal the canonical length (number of code points). wstr_length != length exactly when: 2==sizeof(wchar_t) PyUnicode_4BYTE_KIND == PyUnicode_KIND( str ) which can sometimes be eliminated at compile-time, and always by string creation time. In all other cases, (wstr_length == length), and wstr can be generated by widening the data without having to inspect it. Is it worth eliminating wstr_length (or even wstr) in those cases, or is that too much complexity? (3) I would feel much less nervous if the remaining 4 values of PyUnicode_Kind were explicitly reserved, and the macros raised an error when they showed up. ... If people use C, they can construct all kinds of illegal ... kind values: many places will either work incorrectly, or have an assertion in debug mode already if an unexpected kind is encountered. What I'm asking is that (1) The other values be documented as reserved, rather than as illegal. (2) The macros produce an error rather than silently corrupting data. This allows at least the possibility of a later change such that (3) The macros handle the new values correctly, if only by delegating back to type-supplied functions. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 12/13/2011 10:54 AM, Vinay Sajip wrote: I started writing a tool today, tentatively called '2to23', which aims to do this. It's basically 2to3, but with a package of custom fixers in a package 'lib2to23.fixers' adapted from the corresponding fixers in lib2to3. When, some year in the future, people want to drop Python 2 compatibility from their Python23 code, they will need a 23to3 tool. You might keep this in mind when designing and documenting a bridge language. For each 2to23 fixer, is there a 23to3 fixer so that 23to3(2to23(code)) == 2to3(code) or close enough. (23to3 can and should assume that its input is the output of 2to23, and only look to convert the ultimately temporary scaffolding inserted by 2to23.) The point about documentation is to list the names that 2to23 introduces (with its special meanings) and that 23to3 will remove (assuming the special meanings). So these names should neither be in the 2 code before running 2to23 nor added to 23 code (with a different meaning) before running 23to3. If 2to23 were paired with a 23to3, so people knew that its output is not a deadend cul-de-sac, but a stepping stone to the future, it would be even more attractive. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 13/12/2011 21:10, Terry Reedy wrote: On 12/13/2011 2:02 PM, PJ Eby wrote: On Tue, Dec 13, 2011 at 11:24 AM, Antoine Pitrou solip...@pitrou.net mailto:solip...@pitrou.net wrote: On Tue, 13 Dec 2011 15:28:31 +0100 Laurence Rowe l...@lrowe.co.uk mailto:l...@lrowe.co.uk wrote: The approach that most people seem to have settled on for porting libraries to Python 3 is to make a single codebase that is compatible with both Python 2 and Python 3, perhaps making use of the six library. Do you have evidence that most people have settled on that approach? (besides the couple of library writers who have commented on this thread) I think there is clearly enough 'some' people to justify official support of a 2to23 (or more obscurely, 2to6, but I just got the point that 6=2*3). More specifically six [1] is the name of Benjamin Peterson's support package to help write code that works on both 2 and 3. So the idea is that the conversion isn't just a straight syntax conversion - but that it [could] generate code using this library. All the best, Michael [1] http://packages.python.org/six/ Beyond that, we don't know and don't need to know. I've seen more projects doing it that way than maintaining dual code bases. In retrospect, it seems way more attractive than having to run a converter all the time, especially if I could run a 2to6 tool *once* and then simply write new code using six-isms Among other things, it means that: * There's only one codebase * If the conversion isn't perfect, you only have to fix it once * Line numbers are the same * There's no conversion step slowing down development So, I expect that if the approach is at all viable, it'll quickly become the One Obvious Way to do it. In effect, 2to3 is a purity solution, but six is more like a practicality solution. 2to3 is the practical solution for someone converting private Python 2 code to run on Python 3 *instead* of Python 2, without looking back. By the nature of things, such conversions will be private and scattered over the next decade or so. If 2to3 works well, we will never hear about them, except for the rare praise. Ditto for public code whose author wishes to abandon Py 2. But that seems to rare so far. So we are really talking about upgrading public libraries and apps to work with Python 3 *as well as* 'recent' Python 2 versions. For that, a 'Python23' bridge seems to work for some. Looking ahead, there will in the future be a need for a 23to3 converter to remove the then extraneous bridge code. But that will need a semi-standard 'Python23' to work from. -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Wed, Dec 14, 2011 at 8:17 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: More specifically six [1] is the name of Benjamin Peterson's support package to help write code that works on both 2 and 3. So the idea is that the conversion isn't just a straight syntax conversion - but that it [could] generate code using this library. The thing is, the code you want to generate varies depending on whether you want to target 2.6+, or include 2.5 and earlier. For 2.6+, you can just use the print_function and unicode_literal __future__ imports to minimise overhead. But if 2.5 and earlier is in the mix, you need to lean more heavily on six (for u(), b() and print_()) String translation is also an open question. For some codebases, you want both u and to translate to a Unicode (either in Py3k or via the future import), but if a code base deals with WSGI-style native strings (by means of u for text, for native, b for binary), then the more appropriate translation is to use the future import and map them to , str() and b respectively. So, rather than an overall 2to6, it may be better to focus on *specific* fixers that can be tweaked or added to help with: 2.4+ - 2.4+, 3.2+ 2.4+ - 2.6+, 3.2+ 2.6+ - 2.6+, 3.2+ 2.6+, 3.2+ - 3.2+ (with handling of string literals being the most significant, and likely most complicated) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Dec 14, 2011, at 08:38 AM, Nick Coghlan wrote: String translation is also an open question. For some codebases, you want both u and to translate to a Unicode (either in Py3k or via the future import) I have a fixer for this: http://bazaar.launchpad.net/~barry/flufl.i18n/devel/view/head:/myfixers/fix_ugettext.py (or maybe by translation you don't mean gettext). Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
14.12.11 00:38, Nick Coghlan написав(ла):
String translation is also an open question. For some codebases, you
want both u and to translate to a Unicode (either in Py3k or
via the future import), but if a code base deals with WSGI-style
native strings (by means of u for text, for native, b for
binary), then the more appropriate translation is to use the future
import and map them to , str() and b respectively.
There are other place for native strings -- sys.argv.
if sys.argv[1] == str('-'):
f = sys.stdin
else:
f = open(sys.argv[1], 'r')
Yet another pitfall -- sys.stdin is bytes stream in 2.x and text stream
in 3.x. For reading binary data:
if sys.argv[1] == str('-'):
if sys.version_info[0] = 3:
f = sys.stdin.buffer.raw
else:
f = sys.stdin
else:
f = open(sys.argv[1], 'r')
Reading text data is even more complicated in Python 2.x.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 09:37 pm, tjre...@udel.edu wrote: On 12/13/2011 10:54 AM, Vinay Sajip wrote: I started writing a tool today, tentatively called '2to23', which aims to do this. It's basically 2to3, but with a package of custom fixers in a package 'lib2to23.fixers' adapted from the corresponding fixers in lib2to3. When, some year in the future, people want to drop Python 2 compatibility from their Python23 code, they will need a 23to3 tool. No, they will not. They only need a 2to3 or 2to6 tool because Python 2 and Python 3 are not compatible with each other, but they want one program to be valid in Python 2 and Python 3 simultaneously. When they decide they no longer care about Python 2, they can just stop taking care to keep their program valid as Python 2 and only take care to keep it a valid Python 3 program. There's no specific change to make, just a different approach to take with future maintenance. You might say that they will *want* to immediately discard all of their legacy Python 2 support code. I suspect many of them will not want this; but either way it's a want, not a need. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions
Any chance of adding the rationale to the code? I'm really short of time right now, so you need to find somebody else to make such a change. I am willing to believe that requests for a wchar_t (or utf-8 or System Locale charset) representation are common enough to justify caching the data after the first request. That's not the issue; the real issue is memory management. But then why throw it away in the first place? Wouldn't programs that create unicode from wchar_t data also be the most likely to request wchar_t data back? Perhaps. But are they likely to access the string they just created again at all? They know what's in it, so why look at it again? In all other cases, (wstr_length == length), and wstr can be generated by widening the data without having to inspect it. Is it worth eliminating wstr_length (or even wstr) in those cases, or is that too much complexity? It's too little saving. What I'm asking is that (1) The other values be documented as reserved, rather than as illegal. How is that different? (2) The macros produce an error rather than silently corrupting data. In debug mode, or release mode? -1 on release mode. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, 13 Dec 2011 14:02:45 -0500 PJ Eby p...@telecommunity.com wrote: Among other things, it means that: * There's only one codebase * If the conversion isn't perfect, you only have to fix it once * Line numbers are the same * There's no conversion step slowing down development So, I expect that if the approach is at all viable, it'll quickly become the One Obvious Way to do it. Well, with all due respect, this is hand-waving. Sure, if it's viable, then fine. The question is if it's viable, precisely. That depends on which project we're talking about. In effect, 2to3 is a purity solution, but six is more like a practicality solution. This sounds like your personal interpretation. I see nothing pure in 2to3. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, Dec 13, 2011 at 7:30 PM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 13 Dec 2011 14:02:45 -0500 PJ Eby p...@telecommunity.com wrote: Among other things, it means that: * There's only one codebase * If the conversion isn't perfect, you only have to fix it once * Line numbers are the same * There's no conversion step slowing down development So, I expect that if the approach is at all viable, it'll quickly become the One Obvious Way to do it. Well, with all due respect, this is hand-waving. Sure, if it's viable, then fine. The question is if it's viable, precisely. That depends on which project we're talking about. What I'm saying is that it has many characteristics that are desirable for people who need to support Python 2 and 3 - which is likely the most common use case for library developers. In effect, 2to3 is a purity solution, but six is more like a practicality solution. This sounds like your personal interpretation. I see nothing pure in 2to3. It's pure in being optimized for a world where you just stop using Python 2 one day, and start using 3 the next, without any crossover support. As someone else pointed out, this is a more common case for application developers than for library developers. However, until the libraries are ported, it's harder for the app developers to port their apps. Anyway, if you're supporting both 2 and 3, a common code base offers many attractions, so if it can be done, it will. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 12/13/2011 6:36 PM, exar...@twistedmatrix.com wrote: On 09:37 pm, tjre...@udel.edu wrote: On 12/13/2011 10:54 AM, Vinay Sajip wrote: I started writing a tool today, tentatively called '2to23', which aims to do this. It's basically 2to3, but with a package of custom fixers in a package 'lib2to23.fixers' adapted from the corresponding fixers in lib2to3. When, some year in the future, people want to drop Python 2 compatibility from their Python23 code, they will need a 23to3 tool. No, they will not. Yes they will, if you read my conditional statement properly. Anyway, quibbling over the meaning of 'need' is quite useless. It has two shades of meaning: lack of something required, and lack of something desired. You could have made the valid part of your point without starting off as you did. But I already implied that removal is less urgent when I wrote When, some year in the future They only need a 2to3 or 2to6 tool because Python 2 and Python 3 are not compatible with each other, but they want one program to be valid in Python 2 and Python 3 simultaneously. They *need* the extra stuff inserted. They do not *want* to insert by hand. So by your narrow meaning of 'need', one could say that having the insertion done by program is a want, not a need. When they decide they no longer care about Python 2, they can just stop taking care to keep their program valid as Python 2 and only take care to keep it a valid Python 3 program. There's no specific change to make, just a different approach to take with future maintenance. You might say that they will *want* to immediately discard all of their legacy Python 2 support code. I suspect many of them will not want this; but either way it's a want, not a need. If and when someone wants the extra stuff removed to eliminated both the extra run-time and mental overhead of having it around, and they do not want to remove it by hand, they will want and therefore need in the more general sense to have it done automatically. In both cases, addition and removal, the process is tedious and error-prone if done by hand. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions
On 12/13/2011 7:01 PM, Martin v. Löwis wrote: What I'm asking is that (1) The other values be documented as reserved, rather than as illegal. How is that different? (2) The macros produce an error rather than silently corrupting data. In debug mode, or release mode? -1 on release mode. These two requests seem slight contradictory. Non-official __xxx__ names are reserved for future use but not illegal now for user-use, and user-generated examples do not raise an exception. They simply do not get any special attention unless and until given an official meaning. Then too bad if that breaks code. So by analogy, reserved type value would be ignored, neither corrupting data or raising errors, until put in use. But I don't know how easy/practical that would be. Or maybe more to the point, how expensive a check would be. Not checking names for reservedness is the easiest thing to do. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On Tue, Dec 13, 2011 at 23:38, Nick Coghlan ncogh...@gmail.com wrote: On Wed, Dec 14, 2011 at 8:17 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: More specifically six [1] is the name of Benjamin Peterson's support package to help write code that works on both 2 and 3. So the idea is that the conversion isn't just a straight syntax conversion - but that it [could] generate code using this library. The thing is, the code you want to generate varies depending on whether you want to target 2.6+, or include 2.5 and earlier. Sure. This is different fixers, and then script to run it could have a parameter for version. I'd expect though that a 2to6 first targets 2.6+, and possibly never end up supporting 2.5 at all. I do realize there still is 2.4 out in the wild, but fewer and fewer people need to support it, and the effort to support it is much higher. String translation is also an open question. For some codebases, you want both u and to translate to a Unicode (either in Py3k or via the future import), but if a code base deals with WSGI-style native strings (by means of u for text, for native, b for binary), then the more appropriate translation is to use the future import and map them to , str() and b respectively. Yeah, that can't be done automatically. There is no generic way to determine if a string should be binary, unicode or native. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
