Re: [Python-Dev] str.format implementation

2011-12-13 Thread Raymond Hettinger 

On Dec 12, 2011, at 7:56 PM, Ben Wolfson wrote:

> I personally would prefer (1) to (2) or (3), and (3) to (2), had I my
> druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of
> them to nothing (or to changing the docs to reflect the current batty
> behavior).

+1 on changing the batty behavior.


Raymond

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str.format implementation

2011-12-13 Thread Nick Coghlan 
On Tue, Dec 13, 2011 at 6:37 PM, Raymond Hettinger
 wrote:
>
> On Dec 12, 2011, at 7:56 PM, Ben Wolfson wrote:
>
> I personally would prefer (1) to (2) or (3), and (3) to (2), had I my
> druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of
> them to nothing (or to changing the docs to reflect the current batty
> behavior).
>
>
> +1 on changing the batty behavior.

Skimming my comments from last time this came up, +1 on just going
with what the docs say. The PEP underspecified it, so taking the docs
as the spec for this aspect seems like a reasonable course of action.

Cheers,
Nick.


-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str.format implementation

2011-12-13 Thread Petri Lehtinen 
Ben Wolfson wrote:
> Hi,
> 
> I'm hoping to get some kind of consensus about the divergences between
> the implementation and documentation of str.format
> (http://mail.python.org/pipermail/python-dev/2011-June/111860.html and
> the linked bug report contain examples of the divergences). These
> pertain to the arg_name, attribute_name, and element_index fields of
> the grammar in the docs:
> 
> replacement_field ::=  "{" [field_name] ["!" conversion] [":"
> format_spec] "}"
> field_name::=  arg_name ("." attribute_name | "["
> element_index "]")*
> arg_name  ::=  [identifier | integer]
> attribute_name::=  identifier
> element_index ::=  integer | index_string
> index_string  ::=   +
> 
> Nothing definitive emerged from the last round of discussion, and as
> far as I can recall there are now three proposals for what kind of
> changes might be worth making:
> 
>  (1) the implementation should conform to the docs;*
>  (2) like (1) with the change that element_index should be changed to
> "integer | identifier" (rendering index_string otiose);
>  (3) like (1) with the change that index_string should be changed to
> ''.
> 
> * the docs link "integer" to
> http://docs.python.org/reference/lexical_analysis.html#grammar-token-integer
> but the current implementation only allows decimal integers, which
> seems reasonable and worth retaining.
> 
> (2) was suggested by Greg Ewing on python-dev and (3) by Petri
> Lehtinen in the bug report. (Petri actually suggested that braces be
> disallowed except for the nesting in the format_spec, but it comes to
> the same thing.)
> 
> None of these should be difficult to implement; patches exist for (1)
> and (2). (2) and (3) would lead to format strings that are easier to
> for the programmer to visually parse; (1) would make the indexing part
> of the replacement field conform more closely to the way indexing with
> strings behaves in Python generally, where arbitrary strings can be
> used. (It wouldn't conform exactly, obviously, since ']' would still
> be excluded.)
> 
> I personally would prefer (1) to (2) or (3), and (3) to (2), had I my
> druthers, but it doesn't matter a *whole* lot to me; I'd prefer any of
> them to nothing (or to changing the docs to reflect the current batty
> behavior).

+1 for changing. And as I've said before, I prefer proposal (3).
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IEEE/ISO draft on Python vulnerabilities

2011-12-13 Thread Amaury Forgeot d'Arc 
2011/12/12 Victor Stinner 

> "When sorting a list using the sort() method, attempting to inspect or
> mutate the content of the list will result in undefined behaviour."


But is this even true? in listobject.c::listsort(), since 2002,
/* The list is temporarily made empty, so that mutations performed
 * by comparison functions can't affect the slice of memory we're
 * sorting (allowing mutations during sorting is a core-dump
 * factory, since ob_item may change).
 */
So behaviour is not undefined at all... maybe this report is only based on
note #10 of the documentation:
http://docs.python.org/library/stdtypes.html#mutable-sequence-types
and only considers python 2.2 or older...

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IEEE/ISO draft on Python vulnerabilities

2011-12-13 Thread Armin Rigo 
Hi,

On Tue, Dec 13, 2011 at 11:37, Amaury Forgeot d'Arc  wrote:
>> "When sorting a list using the sort() method, attempting to inspect or
>> mutate the content of the list will result in undefined behaviour."
>
> (...)
> So behaviour is not undefined at all...

No, the behavior _is_ undefined.  The comment you cited says that it
cannot crash the Python interpreter; additionally, it makes a
best-effort attempt at catching such accesses and raising ValueError.
But I think I can build a strange-looking example where you mutate a
list during sorting and don't get a ValueError (although admittedly it
needs a lot of hacking to do that nowadays, e.g. multiple threads).


A bientôt,

Armin.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IEEE/ISO draft on Python vulnerabilities

2011-12-13 Thread Amaury Forgeot d'Arc 
2011/12/13 Armin Rigo 

> No, the behavior _is_ undefined.  The comment you cited says that it
> cannot crash the Python interpreter; additionally, it makes a
> best-effort attempt at catching such accesses and raising ValueError.
> But I think I can build a strange-looking example where you mutate a
> list during sorting and don't get a ValueError (although admittedly it
> needs a lot of hacking to do that nowadays, e.g. multiple threads).
>

I'm interested to see how!
The current implementation installs an empty array in the list,
and the initial array is only held by a local variable in listsort().
even gc.get_referrers() can return the empty list...

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Laurence Rowe 
On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough   
wrote:



On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote:



As someone who ported WebOb and other stuff built on top of it
to Python
3 without using "from __future__ import unicode_literals", I'm
kinda sad
that to be using best practice I'll have to go back and flip
the
polarity on everything.


Eh?  If you don't need unicode_literals, what's the problem?


Porting the WebOb code sucked.  It's only about 5K lines of code but the
porting effort took me about 80 hours.  Some of the problem is certainly
my own idiocy, but some of it is just because straddling code across
Python 2 and Python 3 currently requires that you change lots and lots
of code for suspect benefit.


Could this manual work be cut down if there was a version of 2to3 that  
targeted the subset of the language that is compatible with both 2 and 3?  
That would seem to avoid most of the drawbacks to the current 2to3  
approach.


Laurence

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Michael Foord 

On 13/12/2011 13:33, Laurence Rowe wrote:
On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough  
wrote:



On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote:



As someone who ported WebOb and other stuff built on top of it
to Python
3 without using "from __future__ import unicode_literals", I'm
kinda sad
that to be using best practice I'll have to go back and flip
the
polarity on everything.


Eh?  If you don't need unicode_literals, what's the problem?


Porting the WebOb code sucked.  It's only about 5K lines of code but the
porting effort took me about 80 hours.  Some of the problem is certainly
my own idiocy, but some of it is just because straddling code across
Python 2 and Python 3 currently requires that you change lots and lots
of code for suspect benefit.


Could this manual work be cut down if there was a version of 2to3 that 
targeted the subset of the language that is compatible with both 2 and 
3? That would seem to avoid most of the drawbacks to the current 2to3 
approach.


I'm not sure what you mean, but it *reads* as if you mean "a version of 
2to3 that only converts code that doesn't need converting". Could you 
clarify?


Thanks,

Michael


Laurence

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk





--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Nick Coghlan 
Input = normal 2.x code; Output = code that runs on both 2.x and 3.x.

That is, tinkering with what 2to3 produces, not what it accepts.

--
Nick Coghlan (via Gmail on Android, so likely to be more terse than usual)
On Dec 13, 2011 11:46 PM, "Michael Foord"  wrote:

> On 13/12/2011 13:33, Laurence Rowe wrote:
>
>> On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough 
>> wrote:
>>
>>  On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote:
>>>
>>>
>>> As someone who ported WebOb and other stuff built on top of it
to Python
3 without using "from __future__ import unicode_literals", I'm
kinda sad
that to be using best practice I'll have to go back and flip
the
polarity on everything.


 Eh?  If you don't need unicode_literals, what's the problem?

>>>
>>> Porting the WebOb code sucked.  It's only about 5K lines of code but the
>>> porting effort took me about 80 hours.  Some of the problem is certainly
>>> my own idiocy, but some of it is just because straddling code across
>>> Python 2 and Python 3 currently requires that you change lots and lots
>>> of code for suspect benefit.
>>>
>>
>> Could this manual work be cut down if there was a version of 2to3 that
>> targeted the subset of the language that is compatible with both 2 and 3?
>> That would seem to avoid most of the drawbacks to the current 2to3 approach.
>>
>>  I'm not sure what you mean, but it *reads* as if you mean "a version of
> 2to3 that only converts code that doesn't need converting". Could you
> clarify?
>
> Thanks,
>
> Michael
>
>  Laurence
>>
>> __**_
>> Python-Dev mailing list
>> [email protected]
>> http://mail.python.org/**mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
>> fuzzyman%40voidspace.org.uk
>>
>>
>
> --
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing 
> http://www.sqlite.org/**different.html
>
> __**_
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> ncoghlan%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Michael Foord 

On 13/12/2011 14:24, Nick Coghlan wrote:


Input = normal 2.x code; Output = code that runs on both 2.x and 3.x.

That is, tinkering with what 2to3 produces, not what it accepts.



How is that different from what 2to3 currently does? Are you agreeing 
with Laurence, suggesting an alternative, or something else?


Michael


--
Nick Coghlan (via Gmail on Android, so likely to be more terse than usual)

On Dec 13, 2011 11:46 PM, "Michael Foord" > wrote:


On 13/12/2011 13:33, Laurence Rowe wrote:

On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough
mailto:[email protected]>> wrote:

On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote:


   As someone who ported WebOb and other stuff
built on top of it
   to Python
   3 without using "from __future__ import
unicode_literals", I'm
   kinda sad
   that to be using best practice I'll have to go
back and flip
   the
   polarity on everything.


Eh?  If you don't need unicode_literals, what's the
problem?


Porting the WebOb code sucked.  It's only about 5K lines
of code but the
porting effort took me about 80 hours.  Some of the
problem is certainly
my own idiocy, but some of it is just because straddling
code across
Python 2 and Python 3 currently requires that you change
lots and lots
of code for suspect benefit.


Could this manual work be cut down if there was a version of
2to3 that targeted the subset of the language that is
compatible with both 2 and 3? That would seem to avoid most of
the drawbacks to the current 2to3 approach.

I'm not sure what you mean, but it *reads* as if you mean "a
version of 2to3 that only converts code that doesn't need
converting". Could you clarify?

Thanks,

Michael

Laurence

___
Python-Dev mailing list
[email protected] 
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:

http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



-- 
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected] 
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com




--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Laurence Rowe 
On Tue, 13 Dec 2011 14:42:12 +0100, Michael Foord  
 wrote:



On 13/12/2011 13:33, Laurence Rowe wrote:
On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough   
wrote:



On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote:



As someone who ported WebOb and other stuff built on top of it
to Python
3 without using "from __future__ import unicode_literals", I'm
kinda sad
that to be using best practice I'll have to go back and flip
the
polarity on everything.


Eh?  If you don't need unicode_literals, what's the problem?


Porting the WebOb code sucked.  It's only about 5K lines of code but  
the
porting effort took me about 80 hours.  Some of the problem is  
certainly

my own idiocy, but some of it is just because straddling code across
Python 2 and Python 3 currently requires that you change lots and lots
of code for suspect benefit.


Could this manual work be cut down if there was a version of 2to3 that  
targeted the subset of the language that is compatible with both 2 and  
3? That would seem to avoid most of the drawbacks to the current 2to3  
approach.


I'm not sure what you mean, but it *reads* as if you mean "a version of  
2to3 that only converts code that doesn't need converting". Could you  
clarify?




The approach that most people seem to have settled on for porting  
libraries to Python 3 is to make a single codebase that is compatible with  
both Python 2 and Python 3, perhaps making use of the six library. If I  
understand correctly, Chris' experience of porting WebOb was that there is  
a large amount of manual work required in this approach in part because of  
the many u'' strings in libraries that extensively use unicode. It should  
be possible to automate this with the same approach as 2to3, but instead  
of a transform from 2->3 it would transform code from 2->(2 & 3). In this  
case the transform would only have to be run once (rather than on every  
setup.py install) and would avoid the difficulties of debugging with  
tracebacks that do not exactly match the source code.


Laurence

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Michael Foord 

On 13/12/2011 14:28, Laurence Rowe wrote:
On Tue, 13 Dec 2011 14:42:12 +0100, Michael Foord 
 wrote:



On 13/12/2011 13:33, Laurence Rowe wrote:
On Mon, 12 Dec 2011 22:18:40 +0100, Chris McDonough 
 wrote:



On Mon, 2011-12-12 at 09:50 -0500, PJ Eby wrote:


As someone who ported WebOb and other stuff built on top 
of it

to Python
3 without using "from __future__ import unicode_literals", 
I'm

kinda sad
that to be using best practice I'll have to go back and flip
the
polarity on everything.


Eh?  If you don't need unicode_literals, what's the problem?


Porting the WebOb code sucked.  It's only about 5K lines of code 
but the
porting effort took me about 80 hours.  Some of the problem is 
certainly

my own idiocy, but some of it is just because straddling code across
Python 2 and Python 3 currently requires that you change lots and lots
of code for suspect benefit.


Could this manual work be cut down if there was a version of 2to3 
that targeted the subset of the language that is compatible with 
both 2 and 3? That would seem to avoid most of the drawbacks to the 
current 2to3 approach.


I'm not sure what you mean, but it *reads* as if you mean "a version 
of 2to3 that only converts code that doesn't need converting". Could 
you clarify?




The approach that most people seem to have settled on for porting 
libraries to Python 3 is to make a single codebase that is compatible 
with both Python 2 and Python 3, perhaps making use of the six 
library. If I understand correctly, Chris' experience of porting WebOb 
was that there is a large amount of manual work required in this 
approach in part because of the many u'' strings in libraries that 
extensively use unicode. It should be possible to automate this with 
the same approach as 2to3, but instead of a transform from 2->3 it 
would transform code from 2->(2 & 3). In this case the transform would 
only have to be run once (rather than on every setup.py install) and 
would avoid the difficulties of debugging with tracebacks that do not 
exactly match the source code.


Ah, you mean a 2toPython3compatible2  converter. Not a bad idea.

Michael



Laurence

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk





--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Vinay Sajip 
Laurence Rowe  lrowe.co.uk> writes:

> The approach that most people seem to have settled on for porting  
> libraries to Python 3 is to make a single codebase that is compatible with  
> both Python 2 and Python 3, perhaps making use of the six library. If I  
> understand correctly, Chris' experience of porting WebOb was that there is  
> a large amount of manual work required in this approach in part because of  
> the many u'' strings in libraries that extensively use unicode. It should  
> be possible to automate this with the same approach as 2to3, but instead  
> of a transform from 2->3 it would transform code from 2->(2 & 3). In this  
> case the transform would only have to be run once (rather than on every  
> setup.py install) and would avoid the difficulties of debugging with  
> tracebacks that do not exactly match the source code.

I started writing a tool today, tentatively called '2to23', which aims to do
this. It's basically 2to3, but with a package of custom fixers in a package
'lib2to23.fixers' adapted from the corresponding fixers in lib2to3. It's
experimental work in progress at the moment. With a sample file like

import anything
import dummy

class CustomException(Exception):
pass

def func1():
a = u'abc'
b = b'def'
c = 'unchanged'
c1 = u'abc' u'def'

def func2():
try:
d = 5L
e = (int, long)
f = (long, int)
g = func3()
if isinstance(g, basestring):
print 'a string'
elif isinstance(g, bytes):
print 'some bytes'
elif isinstance(g, unicode):
print 'a unicode string'
else:
print
for i in xrange(3):
pass
except Exception:
e = sys.exc_info()
raise CustomException, e[1], e[2]

class BaseClass:
pass

class OtherBaseClass:
pass

class MetaClass:
pass

class DerivedClass(BaseClass, OtherBaseClass):
__metaclass__ = MetaClass


2to23 gives the following suggested changes:

--- sample.py   (original)
+++ sample.py   (refactored)
@@ -1,34 +1,41 @@
 import anything
 import dummy
+from django.utils.py3 import long_type
+from django.utils.py3 import string_types
+from django.utils.py3 import binary_type
+from django.utils.py3 import b
+from django.utils.py3 import text_type
+from django.utils.py3 import u
+from django.utils.py3 import xrange
 
 class CustomException(Exception):
 pass
 
 def func1():
-a = u'abc'
-b = b'def'
+a = u('abc')
+b = b('def')
 c = 'unchanged'
-c1 = u'abc' u'def'
+c1 = u('abc') u('def')
 
 def func2():
 try:
-d = 5L
+d = long_type(5)
 e = (int, long)
 f = (long, int)
 g = func3()
-if isinstance(g, basestring):
-print 'a string'
-elif isinstance(g, bytes):
-print 'some bytes'
-elif isinstance(g, unicode):
-print 'a unicode string'
+if isinstance(g, string_types):
+print('a string')
+elif isinstance(g, binary_type):
+print('some bytes')
+elif isinstance(g, text_type):
+print('a unicode string')
 else:
-print
+print()
 for i in xrange(3):
 pass
 except Exception:
 e = sys.exc_info()
-raise CustomException, e[1], e[2]
+raise CustomException(e[1]).with_traceback(e[2])
 
 class BaseClass:
 pass
@@ -39,8 +46,8 @@
 class MetaClass:
 pass
 
-class DerivedClass(BaseClass, OtherBaseClass):
-__metaclass__ = MetaClass
+class DerivedClass(with_metaclass(MetaClass, BaseClass, OtherBaseClass)):
+pass

As you can see, there's still a bit of work to do, and the sample doesn't cover
all use cases yet. I'll be cross-checking it using my recent Django porting work
to confirm that it covers everything at least needed for that port, which is why
the fixers currently generate imports from django.utils.py3. Obviously, I'll
change this in due course.

Regards,

Vinay Sajip

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Antoine Pitrou 
On Tue, 13 Dec 2011 15:28:31 +0100
"Laurence Rowe"  wrote:
> 
> The approach that most people seem to have settled on for porting  
> libraries to Python 3 is to make a single codebase that is compatible with  
> both Python 2 and Python 3, perhaps making use of the six library.

Do you have evidence that "most" people have settled on that approach?
(besides the couple of library writers who have commented on this
thread)


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Barry Warsaw 
On Dec 13, 2011, at 05:24 PM, Antoine Pitrou wrote:

>On Tue, 13 Dec 2011 15:28:31 +0100
>"Laurence Rowe"  wrote:
>> 
>> The approach that most people seem to have settled on for porting  
>> libraries to Python 3 is to make a single codebase that is compatible with  
>> both Python 2 and Python 3, perhaps making use of the six library.
>
>Do you have evidence that "most" people have settled on that approach?
>(besides the couple of library writers who have commented on this
>thread)

I'm not sure there's any settling at all when it comes to Python 3 porting
yet. ;)

Sometimes, one code base works better, other times 2to3 works well.  I tend to
use the latter on pure-Python setuptools-based projects, and the former on
projects with C extensions, autoconf-based libraries.

-Barry
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Lennart Regebro 
On Tue, Dec 13, 2011 at 14:33, Laurence Rowe  wrote:
> Could this manual work be cut down if there was a version of 2to3 that
> targeted the subset of the language that is compatible with both 2 and 3?

Not really, but a 2to6, ie something that tries to keep Python 2
compatibility by using the six library, might be useful.

//Lennart
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread PJ Eby 
On Tue, Dec 13, 2011 at 11:24 AM, Antoine Pitrou wrote:

> On Tue, 13 Dec 2011 15:28:31 +0100
> "Laurence Rowe"  wrote:
> >
> > The approach that most people seem to have settled on for porting
> > libraries to Python 3 is to make a single codebase that is compatible
> with
> > both Python 2 and Python 3, perhaps making use of the six library.
>
> Do you have evidence that "most" people have settled on that approach?
> (besides the couple of library writers who have commented on this
> thread)
>

I've seen more projects doing it that way than maintaining dual code bases.
 In retrospect, it seems way more attractive than having to run a converter
all the time, especially if I could run a "2to6" tool *once* and then
simply write new code using six-isms

Among other things, it means that:

* There's only one codebase
* If the conversion isn't perfect, you only have to fix it once
* Line numbers are the same
* There's no conversion step slowing down development

So, I expect that if the approach is at all viable, it'll quickly become
the One Obvious Way to do it.  In effect, 2to3 is a "purity" solution, but
six is more like a "practicality" solution.

And if there's official support for it, so much the better.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] str.format implementation

2011-12-13 Thread Eric V. Smith 
On 12/12/2011 10:56 PM, Ben Wolfson wrote:
> Hi,
> 
> I'm hoping to get some kind of consensus about the divergences between
> the implementation and documentation of str.format
> (http://mail.python.org/pipermail/python-dev/2011-June/111860.html and
> the linked bug report contain examples of the divergences). These
> pertain to the arg_name, attribute_name, and element_index fields of
> the grammar in the docs:
> 
> replacement_field ::=  "{" [field_name] ["!" conversion] [":"
> format_spec] "}"
> field_name::=  arg_name ("." attribute_name | "["
> element_index "]")*
> arg_name  ::=  [identifier | integer]
> attribute_name::=  identifier
> element_index ::=  integer | index_string
> index_string  ::=   +
> 
> Nothing definitive emerged from the last round of discussion, and as
> far as I can recall there are now three proposals for what kind of
> changes might be worth making:
> 
>  (1) the implementation should conform to the docs;*
>  (2) like (1) with the change that element_index should be changed to
> "integer | identifier" (rendering index_string otiose);

I've now learned what "otiose" means. Thanks!

>  (3) like (1) with the change that index_string should be changed to
> ''.

This is still on my plate. I just haven't had a lot of Python time
recently. But I do plan to address this.

Eric.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Terry Reedy 

On 12/13/2011 2:02 PM, PJ Eby wrote:

On Tue, Dec 13, 2011 at 11:24 AM, Antoine Pitrou mailto:[email protected]>> wrote:

On Tue, 13 Dec 2011 15:28:31 +0100
"Laurence Rowe" mailto:[email protected]>> wrote:
 >
 > The approach that most people seem to have settled on for porting
 > libraries to Python 3 is to make a single codebase that is
compatible with
 > both Python 2 and Python 3, perhaps making use of the six library.

Do you have evidence that "most" people have settled on that approach?
(besides the couple of library writers who have commented on this
thread)


I think there is clearly enough 'some' people to justify official 
support of a 2to23 (or more obscurely, 2to6, but I just got the point 
that 6=2*3). Beyond that, we don't know and don't need to know.



I've seen more projects doing it that way than maintaining dual code
bases.  In retrospect, it seems way more attractive than having to run a
converter all the time, especially if I could run a "2to6" tool *once*
and then simply write new code using six-isms

Among other things, it means that:

* There's only one codebase
* If the conversion isn't perfect, you only have to fix it once
* Line numbers are the same
* There's no conversion step slowing down development

So, I expect that if the approach is at all viable, it'll quickly become
the One Obvious Way to do it.  In effect, 2to3 is a "purity" solution,
but six is more like a "practicality" solution.


2to3 is the practical solution for someone converting private Python 2 
code to run on Python 3 *instead* of Python 2, without looking back. By 
the nature of things, such conversions will be private and scattered 
over the next decade or so. If 2to3 works well, we will never hear about 
them, except for the rare praise. Ditto for public code whose author 
wishes to abandon Py 2. But that seems to rare so far.


So we are really talking about upgrading public libraries and apps to 
work with Python 3 *as well as* 'recent' Python 2 versions. For that, a 
'Python23' bridge seems to work for some.


Looking ahead, there will in the future be a need for a 23to3 converter 
to remove the then extraneous bridge code. But that will need a 
semi-standard 'Python23' to work from.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions

2011-12-13 Thread Jim Jewett 
On Tue, Dec 13, 2011 at 2:55 AM, "Martin v. Löwis"  wrote:
>> (1)  Why is PyObject_HEAD used instead of PyObject_VAR_HEAD?

> The unicode object is not a var object. In a var object, tp_itemsize
> gives the element size, which is not possible for unicode objects,
> since the itemsize may vary by instance. In addition, not all instances
> have the items after the base object (plus the size of the base object
> in tp_basicsize is also not always correct).

That makes perfect sense.

Any chance of adding the rationale to the code?  Either inline, such
as changing unicodeobject.h line 291 from

PyObject_HEAD
to something like:
PyObject_HEAD   /* Not VAR_HEAD, because tp_itemsize
varies, and data may be elsewhere. */

or in the large comments around line 288:

Note that Strings use PyObject_HEAD and a length field instead of
PyObject_VAR_HEAD, because the tp_itemsize varies by instance, and the
actual data is not always immediately after the PyASCIIObject  header.



>> (2)  Why does PyASCIIObject have a wstr member, and why does
>> PyCompactUnicodeObject have wstr_length?  As best I can tell from the
>> PEP or header file, wstr is only meaningful when either:

> No. wstr is most of all relevant if someone calls
> PyUnicode_AsUnicode(AndSize); any unicode object might get the
> wstr pointer filled out at some point.

I am willing to believe that requests for a wchar_t (or utf-8 or
System Locale charset) representation are common enough to justify
caching the data after the first request.

But then why throw it away in the first place?  Wouldn't programs that
create unicode from wchar_t data also be the most likely to request
wchar_t data back?

> wstr_length is only relevant if wstr is not NULL. For a pure ASCII
> string (and also for Latin-1 and other BMP strings), the wstr length
> will always equal the canonical length (number of code points).

wstr_length != length exactly when:

2==sizeof(wchar_t) &&
PyUnicode_4BYTE_KIND == PyUnicode_KIND( str )

which can sometimes be eliminated at compile-time, and always by
string creation time.

In all other cases, (wstr_length == length), and wstr can be generated
by widening the data without having to inspect it.  Is it worth
eliminating wstr_length (or even wstr) in those cases, or is that too
much complexity?



>> (3)  I would feel much less nervous if the remaining 4 values of
>> PyUnicode_Kind were explicitly reserved, and the macros raised an
>> error when they showed up. ...

> If people use C, they can construct all kinds of "illegal" ...
> kind values: many places will either work incorrectly, or have
> an assertion in debug mode already if an unexpected kind is
> encountered.

What I'm asking is that
(1)  The other values be documented as reserved, rather than as illegal.
(2)  The macros produce an error rather than silently corrupting data.

This allows at least the possibility of a later change such that

(3)  The macros handle the new values correctly, if only by delegating
back to type-supplied functions.

-jJ
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Terry Reedy 

On 12/13/2011 10:54 AM, Vinay Sajip wrote:


I started writing a tool today, tentatively called '2to23', which aims to do
this. It's basically 2to3, but with a package of custom fixers in a package
'lib2to23.fixers' adapted from the corresponding fixers in lib2to3.


When, some year in the future, people want to drop Python 2 
compatibility from their Python23 code, they will need a 23to3 tool. You 
might keep this in mind when designing and documenting a bridge 
language. For each 2to23 fixer, is there a 23to3 fixer so that 
23to3(2to23(code)) == 2to3(code) or close enough. (23to3 can and should 
assume that its input is the output of 2to23, and only look to convert 
the ultimately temporary scaffolding inserted by 2to23.)


The point about documentation is to list the names that 2to23 introduces 
(with its special meanings) and that 23to3 will remove (assuming the 
special meanings). So these names should neither be in the 2 code before 
running 2to23 nor added to 23 code (with a different meaning) before 
running 23to3.


If 2to23 were paired with a 23to3, so people knew that its output is not 
a deadend cul-de-sac, but a stepping stone to the future, it would be 
even more attractive.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Michael Foord 

On 13/12/2011 21:10, Terry Reedy wrote:

On 12/13/2011 2:02 PM, PJ Eby wrote:

On Tue, Dec 13, 2011 at 11:24 AM, Antoine Pitrou mailto:[email protected]>> wrote:

On Tue, 13 Dec 2011 15:28:31 +0100
"Laurence Rowe" mailto:[email protected]>> wrote:
>
> The approach that most people seem to have settled on for porting
> libraries to Python 3 is to make a single codebase that is
compatible with
> both Python 2 and Python 3, perhaps making use of the six library.

Do you have evidence that "most" people have settled on that 
approach?

(besides the couple of library writers who have commented on this
thread)


I think there is clearly enough 'some' people to justify official 
support of a 2to23 (or more obscurely, 2to6, but I just got the point 
that 6=2*3).


More specifically "six" [1] is the name of Benjamin Peterson's support 
package to help write code that works on both 2 and 3. So the idea is 
that the conversion isn't just a straight syntax conversion - but that 
it [could] generate code using this library.


All the best,

Michael

[1] http://packages.python.org/six/

Beyond that, we don't know and don't need to know.


I've seen more projects doing it that way than maintaining dual code
bases.  In retrospect, it seems way more attractive than having to run a
converter all the time, especially if I could run a "2to6" tool *once*
and then simply write new code using six-isms

Among other things, it means that:

* There's only one codebase
* If the conversion isn't perfect, you only have to fix it once
* Line numbers are the same
* There's no conversion step slowing down development

So, I expect that if the approach is at all viable, it'll quickly become
the One Obvious Way to do it.  In effect, 2to3 is a "purity" solution,
but six is more like a "practicality" solution.


2to3 is the practical solution for someone converting private Python 2 
code to run on Python 3 *instead* of Python 2, without looking back. 
By the nature of things, such conversions will be private and 
scattered over the next decade or so. If 2to3 works well, we will 
never hear about them, except for the rare praise. Ditto for public 
code whose author wishes to abandon Py 2. But that seems to rare so far.


So we are really talking about upgrading public libraries and apps to 
work with Python 3 *as well as* 'recent' Python 2 versions. For that, 
a 'Python23' bridge seems to work for some.


Looking ahead, there will in the future be a need for a 23to3 
converter to remove the then extraneous bridge code. But that will 
need a semi-standard 'Python23' to work from.





--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Nick Coghlan 
On Wed, Dec 14, 2011 at 8:17 AM, Michael Foord
 wrote:
> More specifically "six" [1] is the name of Benjamin Peterson's support
> package to help write code that works on both 2 and 3. So the idea is that
> the conversion isn't just a straight syntax conversion - but that it [could]
> generate code using this library.

The thing is, the code you want to generate varies depending on
whether you want to target 2.6+, or include 2.5 and earlier.

For 2.6+, you can just use the print_function and unicode_literal
__future__ imports to minimise overhead.

But if 2.5 and earlier is in the mix, you need to lean more heavily on
six (for u(), b() and print_())

String translation is also an open question. For some codebases, you
want both u"" and "" to translate to a Unicode "" (either in Py3k or
via the future import), but if a code base deals with WSGI-style
native strings (by means of u"" for text, "" for native, b"" for
binary), then the more appropriate translation is to use the future
import and map them to "", str("") and b"" respectively.

So, rather than an overall "2to6", it may be better to focus on
*specific* fixers that can be tweaked or added to help with:

2.4+ -> 2.4+, 3.2+
2.4+ -> 2.6+, 3.2+
2.6+ -> 2.6+, 3.2+
2.6+, 3.2+ -> 3.2+

(with handling of string literals being the most significant, and
likely most complicated)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Barry Warsaw 
On Dec 14, 2011, at 08:38 AM, Nick Coghlan wrote:

>String translation is also an open question. For some codebases, you
>want both u"" and "" to translate to a Unicode "" (either in Py3k or
>via the future import)

I have a fixer for this:

http://bazaar.launchpad.net/~barry/flufl.i18n/devel/view/head:/myfixers/fix_ugettext.py

(or maybe by "translation" you don't mean "gettext").

Cheers,
-Barry
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Serhiy Storchaka 

14.12.11 00:38, Nick Coghlan написав(ла):

String translation is also an open question. For some codebases, you
want both u"" and "" to translate to a Unicode "" (either in Py3k or
via the future import), but if a code base deals with WSGI-style
native strings (by means of u"" for text, "" for native, b"" for
binary), then the more appropriate translation is to use the future
import and map them to "", str("") and b"" respectively.


There are other place for native strings -- sys.argv.

if sys.argv[1] == str('-'):
f = sys.stdin
else:
f = open(sys.argv[1], 'r')

Yet another pitfall -- sys.stdin is bytes stream in 2.x and text stream 
in 3.x. For reading binary data:


if sys.argv[1] == str('-'):
if sys.version_info[0] >= 3:
f = sys.stdin.buffer.raw
else:
f = sys.stdin
else:
f = open(sys.argv[1], 'r')

Reading text data is even more complicated in Python 2.x.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread exarkun 

On 09:37 pm, [email protected] wrote:

On 12/13/2011 10:54 AM, Vinay Sajip wrote:
I started writing a tool today, tentatively called '2to23', which aims 
to do
this. It's basically 2to3, but with a package of custom fixers in a 
package

'lib2to23.fixers' adapted from the corresponding fixers in lib2to3.


When, some year in the future, people want to drop Python 2 
compatibility from their Python23 code, they will need a 23to3 tool.


No, they will not.  They only need a 2to3 or 2to6 tool because Python 2 
and Python 3 are not compatible with each other, but they want one 
program to be valid in Python 2 and Python 3 simultaneously.


When they decide they no longer care about Python 2, they can just stop 
taking care to keep their program valid as Python 2 and only take care 
to keep it a valid Python 3 program.  There's no specific change to 
make, just a different approach to take with future maintenance.


You might say that they will *want* to immediately discard all of their 
legacy Python 2 support code.  I suspect many of them will not want 
this; but either way it's a want, not a need.


Jean-Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions

2011-12-13 Thread Martin v. Löwis 
> Any chance of adding the rationale to the code?

I'm really short of time right now, so you need to find somebody
else to make such a change.

> I am willing to believe that requests for a wchar_t (or utf-8 or
> System Locale charset) representation are common enough to justify
> caching the data after the first request.

That's not the issue; the real issue is memory management.

> But then why throw it away in the first place?  Wouldn't programs that
> create unicode from wchar_t data also be the most likely to request
> wchar_t data back?

Perhaps. But are they likely to access the string they just created
again at all? They know what's in it, so why look at it again?

> In all other cases, (wstr_length == length), and wstr can be generated
> by widening the data without having to inspect it.  Is it worth
> eliminating wstr_length (or even wstr) in those cases, or is that too
> much complexity?

It's too little saving.

> What I'm asking is that
> (1)  The other values be documented as reserved, rather than as illegal.

How is that different?

> (2)  The macros produce an error rather than silently corrupting data.

In debug mode, or release mode? -1 on release mode.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Antoine Pitrou 
On Tue, 13 Dec 2011 14:02:45 -0500
PJ Eby  wrote:
> 
> Among other things, it means that:
> 
> * There's only one codebase
> * If the conversion isn't perfect, you only have to fix it once
> * Line numbers are the same
> * There's no conversion step slowing down development
> 
> So, I expect that if the approach is at all viable, it'll quickly become
> the One Obvious Way to do it.

Well, with all due respect, this is hand-waving. Sure, if it's
viable, then fine. The question is if it's "viable", precisely. That
depends on which project we're talking about.

> In effect, 2to3 is a "purity" solution, but
> six is more like a "practicality" solution.

This sounds like your personal interpretation. I see nothing "pure" in
2to3.

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread PJ Eby 
On Tue, Dec 13, 2011 at 7:30 PM, Antoine Pitrou  wrote:

> On Tue, 13 Dec 2011 14:02:45 -0500
> PJ Eby  wrote:
> >
> > Among other things, it means that:
> >
> > * There's only one codebase
> > * If the conversion isn't perfect, you only have to fix it once
> > * Line numbers are the same
> > * There's no conversion step slowing down development
> >
> > So, I expect that if the approach is at all viable, it'll quickly become
> > the One Obvious Way to do it.
>
> Well, with all due respect, this is hand-waving. Sure, if it's
> viable, then fine. The question is if it's "viable", precisely. That
> depends on which project we're talking about.
>

What I'm saying is that it has many characteristics that are desirable for
people who need to support Python 2 and 3 - which is likely the most common
use case for library developers.

> In effect, 2to3 is a "purity" solution, but
> > six is more like a "practicality" solution.
>
> This sounds like your personal interpretation. I see nothing "pure" in
> 2to3.
>

It's "pure" in being optimized for a world where you just stop using Python
2 one day, and start using 3 the next, without any crossover support.

As someone else pointed out, this is a more common case for application
developers than for library developers.  However, until the libraries are
ported, it's harder for the app developers to port their apps.

Anyway, if you're supporting both 2 and 3, a common code base offers many
attractions, so if it can be done, it will.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Terry Reedy 

On 12/13/2011 6:36 PM, [email protected] wrote:

On 09:37 pm, [email protected] wrote:

On 12/13/2011 10:54 AM, Vinay Sajip wrote:

I started writing a tool today, tentatively called '2to23', which
aims to do
this. It's basically 2to3, but with a package of custom fixers in a
package
'lib2to23.fixers' adapted from the corresponding fixers in lib2to3.


When, some year in the future, people want to drop Python 2
compatibility from their Python23 code, they will need a 23to3 tool.


No, they will not.


Yes they will, if you read my conditional statement properly.

Anyway, quibbling over the meaning of 'need' is quite useless. It has 
two shades of meaning: lack of something required, and lack of something 
desired. You could have made the valid part of your point without 
starting off as you did.


But I already implied that removal is less urgent when I wrote "When, 
some year in the future...".



They only need a 2to3 or 2to6 tool because Python 2
and Python 3 are not compatible with each other, but they want one
program to be valid in Python 2 and Python 3 simultaneously.


They *need* the extra stuff inserted. They do not *want* to insert by 
hand. So by your narrow meaning of 'need', one could say that having the 
insertion done by program is a want, not a need.



When they decide they no longer care about Python 2, they can just stop
taking care to keep their program valid as Python 2 and only take care
to keep it a valid Python 3 program. There's no specific change to make,
just a different approach to take with future maintenance.

You might say that they will *want* to immediately discard all of their
legacy Python 2 support code. I suspect many of them will not want this;
but either way it's a want, not a need.


If and when someone wants the extra stuff removed to eliminated both the 
extra run-time and mental overhead of having it around, and they do not 
want to remove it by hand, they will want and therefore need in the more 
general sense to have it done automatically. In both cases, addition and 
removal, the process is tedious and error-prone if done by hand.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyUnicodeObject / PyASCIIObject questions

2011-12-13 Thread Terry Reedy 

On 12/13/2011 7:01 PM, "Martin v. Löwis" wrote:


What I'm asking is that
(1)  The other values be documented as reserved, rather than as illegal.

How is that different?

(2)  The macros produce an error rather than silently corrupting data.

In debug mode, or release mode? -1 on release mode.


These two requests seem slight contradictory. Non-official __xxx__ names 
are reserved for future use but not illegal now for user-use, and 
user-generated examples do not raise an exception. They simply do not 
get any special attention unless and until given an official meaning. 
Then too bad if that breaks code.


So by analogy, reserved type value would be ignored, neither corrupting 
data or raising errors, until put in use. But I don't know how 
easy/practical that would be.


Or maybe more to the point, how expensive a check would be. Not checking 
names for reservedness is the easiest thing to do.


--
Terry Jan Reedy


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] readd u'' literal support in 3.3?

2011-12-13 Thread Lennart Regebro 
On Tue, Dec 13, 2011 at 23:38, Nick Coghlan  wrote:
> On Wed, Dec 14, 2011 at 8:17 AM, Michael Foord
>  wrote:
>> More specifically "six" [1] is the name of Benjamin Peterson's support
>> package to help write code that works on both 2 and 3. So the idea is that
>> the conversion isn't just a straight syntax conversion - but that it [could]
>> generate code using this library.
>
> The thing is, the code you want to generate varies depending on
> whether you want to target 2.6+, or include 2.5 and earlier.

Sure. This is different fixers, and then script to run it could have a
parameter for version.
I'd expect though that a 2to6 first targets 2.6+, and possibly never
end up supporting 2.5 at all. I do realize there still is 2.4 out in
the wild, but fewer and fewer people need to support it, and the
effort to support it is much higher.

> String translation is also an open question. For some codebases, you
> want both u"" and "" to translate to a Unicode "" (either in Py3k or
> via the future import), but if a code base deals with WSGI-style
> native strings (by means of u"" for text, "" for native, b"" for
> binary), then the more appropriate translation is to use the future
> import and map them to "", str("") and b"" respectively.

Yeah, that can't be done automatically. There is no generic way to
determine if a string should be binary, unicode or native.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com