Re: [Python-Dev] EuroPython Language Summit report

2011-06-28 Thread Michael Foord

On 24/06/2011 15:55, R. David Murray wrote:

On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinsondicki...@gmail.com  wrote:

EuroPython 2011 Language Summit
===

[...]

 Unicode character classes is a particular need).  [Subtopic:  what needs
 to be done to get the new regex module into Python?  Should it replace
 the existing module?  What about backwards compatibility issues?]

I'm pretty sure regex has backward compatibility as a goal for just this
reason (so it can replace the current module).


The new regex library has some great improvements:

http://bugs.python.org/issue2636

It also has users and committed maintainers, so I hope we can bring it 
into 3.3. It wasn't easy to tell from skimming the change notes that 
Unicode character classes are amongst the new features. Is that the case?


Michael



PEP 0380: Syntax for delegating to a subgenerator.

PEP 3150: Statement local namespaces.

PEP 3152: Cofunctions.

For all three of the above PEPs, there was some feeling that additional syntax
changes to the language are unnecessary and make it harder to learn; where
possible, we should prefer using 3rd partly libraries.

I disagree with this with respect to 380.  Haven't looked at the others.


Python 2.7
--

How long will this be maintained?
Original decision was 5 years.

As I recall, the actual decision was at *least* 5 years.  It's only
been one so far...was the discussion that five years was too short or
too long?

As the code bases continue to drift apart, and we fix the fixable bugs,
the number of patches going in to 2.7 will decrease over time, so I
don't think the burden of continuing to support it will be too heavy.
Currently about half of the non-feature-request issues (ie: bugs) in
the tracker are tagged 2.7.  I expect to see that percentage continue
to decrease over time.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-28 Thread Victor Stinner
Le mardi 28 juin 2011 à 14:41 +0100, Michael Foord a écrit :
 On 24/06/2011 15:55, R. David Murray wrote:
  On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinsondicki...@gmail.com  
  wrote:
  EuroPython 2011 Language Summit
  ===
  [...]
   Unicode character classes is a particular need).  [Subtopic:  what 
  needs
   to be done to get the new regex module into Python?  Should it replace
   the existing module?  What about backwards compatibility issues?]
  I'm pretty sure regex has backward compatibility as a goal for just this
  reason (so it can replace the current module).
 
 The new regex library has some great improvements:
 
  http://bugs.python.org/issue2636

This issue is open since April 2008, has also the longest list of
attached files, and has a very long history. What is the status of the
issue? I see that there is now a third party project on:

   http://code.google.com/p/mrab-regex-hg/

--

There is also the re2 library from Google and especially this project:

   http://pypi.python.org/pypi/re2/

pyre2 is a Python extension that wraps Google's RE2 regular expression
library.

This version of pyre2 is similar to the one you'd find at facebook's
github repository except that the stated goal of *this version is to be
a drop-in replacement for the re module*.)

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-28 Thread Bill Janssen
Michael Foord fuzzy...@voidspace.org.uk wrote:

 The new regex library has some great improvements:
 
 http://bugs.python.org/issue2636
 
 It also has users and committed maintainers, so I hope we can bring it
 into 3.3. It wasn't easy to tell from skimming the change notes that
 Unicode character classes are amongst the new features. Is that the
 case?

According to http://effbot.org/zone/unicode-objects.htm (from 2004), the
existing re module already supports Unicode character classes, so the
regex module will as well.  But the support has been updated, according
to the change notes.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-28 Thread Michael Foord

On 28/06/2011 18:08, Bill Janssen wrote:

Michael Foordfuzzy...@voidspace.org.uk  wrote:


The new regex library has some great improvements:

 http://bugs.python.org/issue2636

It also has users and committed maintainers, so I hope we can bring it
into 3.3. It wasn't easy to tell from skimming the change notes that
Unicode character classes are amongst the new features. Is that the
case?

According to http://effbot.org/zone/unicode-objects.htm (from 2004), the
existing re module already supports Unicode character classes, so the
regex module will as well.  But the support has been updated, according
to the change notes.
Thanks. Support for Unicode character classes was one of the 
improvements needed in the re module reported from the language summit - 
so I wonder if the changes in regex are sufficient.


Michael


Bill



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-28 Thread Bill Janssen
Michael Foord fuzzy...@voidspace.org.uk wrote:

 Thanks. Support for Unicode character classes was one of the
 improvements needed in the re module reported from the language summit
 - 
 so I wonder if the changes in regex are sufficient.

I guess it depends on what you're asking -- what does support for
Unicode character classes mean?  Apparently something different from
what it meant in 2004 :-).

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-28 Thread Georg Brandl
On 28.06.2011 16:06, Victor Stinner wrote:
 Le mardi 28 juin 2011 à 14:41 +0100, Michael Foord a écrit :
 On 24/06/2011 15:55, R. David Murray wrote:
  On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinsondicki...@gmail.com  
  wrote:
  EuroPython 2011 Language Summit
  ===
  [...]
   Unicode character classes is a particular need).  [Subtopic:  what 
  needs
   to be done to get the new regex module into Python?  Should it 
  replace
   the existing module?  What about backwards compatibility issues?]
  I'm pretty sure regex has backward compatibility as a goal for just this
  reason (so it can replace the current module).
 
 The new regex library has some great improvements:
 
  http://bugs.python.org/issue2636
 
 This issue is open since April 2008, has also the longest list of
 attached files, and has a very long history. What is the status of the
 issue? I see that there is now a third party project on:
 
http://code.google.com/p/mrab-regex-hg/

This should be the same module as in the issue (and thankfully, because
code management of such a big project does not belong exclusively in a
tracker issue).  mrab stands for Matthew Barnett, who is the author of
regex.

 --
 
 There is also the re2 library from Google and especially this project:
 
http://pypi.python.org/pypi/re2/
 
 pyre2 is a Python extension that wraps Google's RE2 regular expression
 library.
 
 This version of pyre2 is similar to the one you'd find at facebook's
 github repository except that the stated goal of *this version is to be
 a drop-in replacement for the re module*.)

Well, while it can be called drop-in, it is hardly a good replacement:


That being said, there are features of the re module that this module may never
have. For example, RE2 does not handle lookahead assertions ((?=...)).


It falls back to old re in these cases.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-26 Thread P.J. Eby

At 12:32 PM 6/25/2011 -0400, R. David Murray wrote:

So your proposed code would allow me, when writing a generator in
my code, do something that would allow me to yield up all the
values from an arbitrary generator I'm calling, over which I have
no control (ie: I can't modify its code)?


With a decorator on your own function, yes.  See:

  http://mail.python.org/pipermail/python-dev/2010-July/102320.html

for details.  Mostly, though, that proposal was a suggestion for how 
the optimized implementation would work - i.e., a suggestion that 
PEP 380 be implemented that way under the hood, by implicitly turning 
'yield from' into 'yield From()' and wrapping the generator itself 
with another From() instance.


(IOW, that was a proposal for how to avoid the extra overhead of 
recursive yielding in a series of nested yield-from's.)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-25 Thread Greg Ewing

Nick Coghlan wrote:


Indeed, PEP 380 is *really* hard to do properly without language
support. The language moratorium and lack of a Python 3 compatible
patch


Pardon? My original patch was for 3.1.2.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-25 Thread Nick Coghlan
On Sat, Jun 25, 2011 at 9:27 PM, Greg Ewing greg.ew...@canterbury.ac.nz wrote:
 Nick Coghlan wrote:

 Indeed, PEP 380 is *really* hard to do properly without language
 support. The language moratorium and lack of a Python 3 compatible
 patch

 Pardon? My original patch was for 3.1.2.

My mistake. We must have changed something else somewhere along the
line that broke your patch...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-25 Thread P.J. Eby

At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:

Indeed, PEP 380 is *really* hard to do properly without language
support.


No, it isn't.  You add a decorator, a 'from_' class, and a 'return_' 
function, and there you go.  (See my previous code sketches here in 
early PEP 380 discussions.)


Python frameworks have been doing variations of the same thing (with 
varying features and APIs) for at least 7 years now -- even on Python 
versions that lack decorators or the ability to return values from 
yield statements.


So the main benefit of a PEP for this functionality would be 
providing a common implementation/API - and that could be initially 
done in the stdlib, without any added syntax support.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-25 Thread R. David Murray
On Sat, 25 Jun 2011 11:31:42 -0400, P.J. Eby p...@telecommunity.com wrote:
 At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:
 Indeed, PEP 380 is *really* hard to do properly without language
 support.
 
 No, it isn't.  You add a decorator, a 'from_' class, and a 'return_' 
 function, and there you go.  (See my previous code sketches here in 
 early PEP 380 discussions.)
 
 Python frameworks have been doing variations of the same thing (with 
 varying features and APIs) for at least 7 years now -- even on Python 
 versions that lack decorators or the ability to return values from 
 yield statements.
 
 So the main benefit of a PEP for this functionality would be 
 providing a common implementation/API - and that could be initially 
 done in the stdlib, without any added syntax support.

So your proposed code would allow me, when writing a generator in
my code, do something that would allow me to yield up all the
values from an arbitrary generator I'm calling, over which I have
no control (ie: I can't modify its code)?

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-25 Thread Guido van Rossum
On Sat, Jun 25, 2011 at 9:32 AM, R. David Murray rdmur...@bitdance.com wrote:
 On Sat, 25 Jun 2011 11:31:42 -0400, P.J. Eby p...@telecommunity.com wrote:
 At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:
 Indeed, PEP 380 is *really* hard to do properly without language
 support.

 No, it isn't.  You add a decorator, a 'from_' class, and a 'return_'
 function, and there you go.  (See my previous code sketches here in
 early PEP 380 discussions.)

 Python frameworks have been doing variations of the same thing (with
 varying features and APIs) for at least 7 years now -- even on Python
 versions that lack decorators or the ability to return values from
 yield statements.

 So the main benefit of a PEP for this functionality would be
 providing a common implementation/API - and that could be initially
 done in the stdlib, without any added syntax support.

 So your proposed code would allow me, when writing a generator in
 my code, do something that would allow me to yield up all the
 values from an arbitrary generator I'm calling, over which I have
 no control (ie: I can't modify its code)?

Let me cut this short. PEP 380 is pretty much approved. I know there
are a few details worth quibbling over, but they are not going to
jeopardize acceptance of the PEP. We are waiting for an implementation
in Python 3.3. In fact, I wouldn't mind at this point if someone took
their best effort at an implementation and checked it into the 3.3
branch, and we can do the quibbling over the details while we have a
working implementation to experiment with.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-25 Thread Greg Ewing

P.J. Eby wrote:

At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:


Indeed, PEP 380 is *really* hard to do properly without language
support.


No, it isn't.  You add a decorator, a 'from_' class, and a 'return_' 
function, and there you go.  (See my previous code sketches here in 
early PEP 380 discussions.)


Will it handle *all* of the generator protocol correctly,
including send(), exception handling, and generator closing?

Also, how efficient would it be? A major benefit of a built-in
implementation is that it can be almost as fast as using the
sub-generator directly.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Victor Stinner
Le vendredi 24 juin 2011 à 10:52 +0200, Mark Dickinson a écrit :
   - [Armin Ronacher] Python 3's Unicode support still has some dark areas.

What? Unicode support is perfect in Python 3!

   One  example: when opening a text file for reading and writing, the default
 encoding used depends on the platform and on various environment 
 variables.

... oh, I agree. This choice is a big portability issue. Mac OS X, most
Linux distro, BSD systems use UTF-8 local encoding, whereas Windows use
legacy code pages like cp1252 (something like ISO-8859-1) or cp952
(shift jis). But sometimes, the locale is C (e.g. on our buildbots)
and programs start to fail with Unicode errors...

I see two options to improve the situation.


(1) hard way: change open() API to make encoding a mandatory argument.
Problem: it breaks compatibility with Python 3.0, 3.1 and 3.2 (ooops!);
the encoding argument is the 4th argument, you have to use a keyword or
choose a value for the buffering argument. I proposed to change open()
API in Python 3.1 to make all arguments -except the filename and the
mode- keyword-only arguments, but Guido rejected my idea:

Remember, for 3.0 we're trying to get a release out of the door, not
cram in new features, no matter how small.

http://bugs.python.org/issue4121


(2) soft way: add a warning if the encoding is implicit (locale
encoding). I don't know what is the best warning type, and if it should
be always displayed, only once, or not by default. Even if it is hidden
by default, a careful developer will be able to use -Werror to fix
bugs... I suspect that most tests fail if open() raises an exception if
the encoding is not specified (e.g. see distutils/packaging issues about
the file encoding).

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Nick Coghlan
On Fri, Jun 24, 2011 at 6:52 PM, Mark Dickinson dicki...@gmail.com wrote:
 PEP 3118: (Not open, but ...)  Revising the buffer protocol.  The buffer
    protocol implementation is still buggy and incomplete.  (Mostly okay for 1d
    buffers, not even close for higher-dimensional work.)

Issue 10181 is the place to start for anyone that wants to help with
this one! :)

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Vinay Sajip
Mark Dickinson dickinsm at gmail.com writes:

 virtualenv in Python 3.3?
 -
 
 Apparently there was some discussion at the last PyCon about the possibility
 of virtualenv going into Python 3.3.  As far as I know there's currently no
 open tracker item or PEP for this.  Larry Hastings knows more here.

  Update from Larry: I'm supportive of putting it in in 3.3, and I'm
  sitting with Raymond Hettinger right now and he supports it too.  So I
  think if we get a working implementation it'll go in.  It's under heavy
  discussion in c.l.p-d so I gather it's moving forward.  Last I knew it was
  Carl Meyer pushing it forward, but Vinay Sajip is the current standard-
  bearer.  I understand Windows support will be a bit tricky; I don't know if
  they have someone who's going to handle it for them.

Mark, thanks for the summary.

Re. a PEP for virtual environments in Python, Carl is working on the PEP. The
first draft by Carl with my initial comments is at [1].  There's still some
work to do before we can actually present it as a PEP we're happy with.

I'm pleased to report good progress with the proof of concept implementation.
There are some issues still with packaging which I'm working through with Éric
Araujo, but I've gone ahead and committed changes on my branch to get things
working. It's a good exercising of the packaging functionality :-)

What I have at the moment is pythonv [2]:

A modified Python which supports virtual environments via changes in
Modules/getpath.c, PC/getpathp.c, Lib/site.py, Lib/sysconfig.py,
Lib/sysconfig.cfg and Lib/distutils/sysconfig.py.

These changes basically detect if you're running in a virtual environment by
looking for an env.cfg, and if found, fixing it up so sys.path is set as it
should be. In addition, sys.site_prefix and sys.site_exec_prefix are created -
if not in a virtual env, these have the same values as sys.prefix and
sys.exec_prefix. With just these changes, the basics of a virtual environment
are provided (in terms of isolation from other environments and the system
Python). However, in order to be useful, the packaging tools have to work with
sys.site_prefix and sys.site_exec_prefix, so changes have been made to
sysconfig, distutils and packaging to do this.

I'm presently working on a demonstration application which integrates the
above work with Doug Hellmann's virtualenvwrapper and Guillermo López' port of
it to Powershell to get a nice basic tool that'll support virtual environments
with packaging as an easy-to-use CLI, as well as Distribute support while
people migrate over to packaging, on Windows as well as Linux.

I'm expecting to cover both the Linux and Windows aspects (though I won't say
no to help ;-) and working through packaging issues relating to improved
Windows support. The basic functionality is there now, both in Windows and
Linux - the focus of work is in the ease-of-use CLI stuff which is not
envisaged to be part of core Python, but nevertheless needs to be done to make
virtual envs as painless as possible.

BTW Of the 398 packages on PyPI having a Py3K trove classifier, I've tested
installing all of them into a pythonv virtual env with Distribute, and 310
install without errors. Of the remaining 88, some are due to missing
dependencies, others due to broken packages on PyPI.

BTW of the Python regression test suite passes all tests in a pythonv
virtualenv, other than those for which there already are tracker issues for
the default branch in  hg.python.org/cpython (test_lib2to3, test_packaging,
test_sysconfig). Full test results are at [3].

All in all, it's looking reasonably good, and we hope to report more progress
on the PEP and the proof of concept soon!

Regards,

Vinay Sajip

[1] http://goo.gl/6u0S0
[2] https://bitbucket.org/vinay.sajip/pythonv
[3] https://gist.github.com/1022705

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread R. David Murray
On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson dicki...@gmail.com wrote:
 EuroPython 2011 Language Summit
 ===
[...] 
 Unicode character classes is a particular need).  [Subtopic:  what needs
 to be done to get the new regex module into Python?  Should it replace
 the existing module?  What about backwards compatibility issues?]

I'm pretty sure regex has backward compatibility as a goal for just this
reason (so it can replace the current module).

 PEP 0380: Syntax for delegating to a subgenerator.
 
 PEP 3150: Statement local namespaces.
 
 PEP 3152: Cofunctions.
 
 For all three of the above PEPs, there was some feeling that additional syntax
 changes to the language are unnecessary and make it harder to learn; where
 possible, we should prefer using 3rd partly libraries.

I disagree with this with respect to 380.  Haven't looked at the others.

 Python 2.7
 --
 
 How long will this be maintained?
 Original decision was 5 years.

As I recall, the actual decision was at *least* 5 years.  It's only
been one so far...was the discussion that five years was too short or
too long?

As the code bases continue to drift apart, and we fix the fixable bugs,
the number of patches going in to 2.7 will decrease over time, so I
don't think the burden of continuing to support it will be too heavy.
Currently about half of the non-feature-request issues (ie: bugs) in
the tracker are tagged 2.7.  I expect to see that percentage continue
to decrease over time.

--
R. David Murray   http://www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Terry Reedy

On 6/24/2011 7:18 AM, Victor Stinner wrote:

Le vendredi 24 juin 2011 à 10:52 +0200, Mark Dickinson a écrit :

   One  example: when opening a text file for reading and writing, the default
 encoding used depends on the platform and on various environment variables.


... oh, I agree. This choice is a big portability issue. Mac OS X, most
Linux distro, BSD systems use UTF-8 local encoding, whereas Windows use
legacy code pages like cp1252 (something like ISO-8859-1) or cp952
(shift jis). But sometimes, the locale is C (e.g. on our buildbots)
and programs start to fail with Unicode errors...

I see two options to improve the situation.


The third is to make utf-8 the default. I believe this *is* the proper 
long term solution and both options are contrary to this.

I believe that this is what I want for myself even on Windows.
I believe utf-8 is the default or only storage by cross-platform
international programs (certainly the ones I use).


(1) hard way: change open() API to make encoding a mandatory argument.



(2) soft way: add a warning if the encoding is implicit (locale
encoding).


(3) In 3.3, if the default is used and it is not utf-8, add a warning 
that the default will become utf-8 always in 3.4. Actually, I would like 
a PendingDeprecationWarning in 3.2.1 if possible.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Glenn Linderman

On 6/24/2011 1:30 PM, Terry Reedy wrote:


The third is to make utf-8 the default. I believe this *is* the proper 
long term solution and both options are contrary to this. 


+1
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Victor Stinner
Le vendredi 24 juin 2011 à 16:30 -0400, Terry Reedy a écrit :
  I see two options to improve the situation.
 
 The third is to make utf-8 the default. I believe this *is* the proper 
 long term solution and both options are contrary to this.

Oh yes, I also prefer this option, but I suspect that some people prefer
to not break backward compatibility.

Or should we consider this bad design choice as a bug?

The UTF-8 encoder (of Python 3) raises an error if the text contains a
surrogate character. The surrogatepass error handler should be used to
encode surrogages.

... The surrogateescape can be used to encode back undecodable bytes
(e.g. filename decoded by Python using the surrogateescape), but it is
not a good idea to write illegal byte sequences (most programs will fail
to open the file).

 I believe that this is what I want for myself even on Windows.

Can you open a UTF-8 files in all Windows program (and the text is
displayed correctly)? I remember that notepad.exe writes an evil UTF-8
BOM, but I don't know if it requires this BOM to detect the UTF-8
encoding.

Or do some program expect text files encoded to the ANSI code page?

If you want to write files in the ANSI code page, you can use
encoding=mbcs (or use an explicit code page, like encoding=cp1252).

 (3) In 3.3, if the default is used and it is not utf-8, add a warning 
 that the default will become utf-8 always in 3.4. Actually, I would like 
 a PendingDeprecationWarning in 3.2.1 if possible.

I'm not sure that the and it is not utf-8 condition is a good idea. If
you develop on Linux but your users are on Windows, you will not get the
warning (even with -Werror) nor your users (users don't want to see
warnings)... Or maybe an user using Windows and Linux will notice the
bug without the warning :-)

It doesn't mean that it is not possible to check your program: you can
change your locale encoding (e.g. use LANG=C).

At least, it will be possible to check test_distutils and test_packaging
using LANG=C and -Werror :-)

--

A fourth option is to use ASCII by default! Your program will work and
be portable until you write the first non-ASCII character... Oh wait, it
remembers me the Unicode nightmare of Python 2!

Victor

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] EuroPython Language Summit report

2011-06-24 Thread Nick Coghlan
On Sat, Jun 25, 2011 at 12:55 AM, R. David Murray rdmur...@bitdance.com wrote:

[quoting VM summit notes]
 PEP 0380: Syntax for delegating to a subgenerator.

 PEP 3150: Statement local namespaces.

 PEP 3152: Cofunctions.

 For all three of the above PEPs, there was some feeling that additional 
 syntax
 changes to the language are unnecessary and make it harder to learn; where
 possible, we should prefer using 3rd partly libraries.

 I disagree with this with respect to 380.  Haven't looked at the others.

Indeed, PEP 380 is *really* hard to do properly without language
support. The language moratorium and lack of a Python 3 compatible
patch are the only reasons it didn't go into 3.2 (and there's a patch
porting the implementation to 3.3 up on bitbucket, although we've been
tinkering with the compiler so it is likely out of date at this
point).

I'm the author PEP 3150 and *I* think it's a highly questionable and
most likely terrible idea (hence the Deferred status). However, it's a
good place to gather the related discussions, since proposals in that
vein recur often on python-ideas.

PEP 3152 definitely fits into the let third parties experiment
bucket, though - once PEP 380 makes generators first class peers to
functions in their support for modularisation, then we need to let the
wider community play with the concept for a while before we embed
anything more into the core language or standard library.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com