Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
By the way, what are the ASCII characters that are not suppported by Shift-JIS? Not many I suppose? (if I read the Wikipedia entry correctly, it's only the backslash and the tilde). The problem with this encoding is that bytes below 128 appear as second bytes of a two-byte encoding: py

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Glenn Linderman
On approximately 5/6/2009 10:53 PM, came the following characters from the keyboard of Martin v. Löwis: The error handler designed with utf-8 in mind has no name in the encode direction and is called utf_8b_decoder_invalid_bytes in the decode direction. By your reasoning, *that* should be its

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
Wouldn't renaming the existing surrogates handler be an incompatible change, and thus inappropriate? No - it's new in Python 3.1. So what do you think about Antoine's proposal? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Glenn Linderman
On approximately 5/6/2009 11:16 PM, came the following characters from the keyboard of Martin v. Löwis: So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up

Re: [Python-Dev] Help on issue 5941

2009-05-07 Thread Tarek Ziadé
On Thu, May 7, 2009 at 11:50 AM, David Cournapeau courn...@gmail.com wrote: Then, in the customize_compiler function, set archiver to $AR + $ARFLAGS. IOW, just copying the logic used for e.g. ldshared, I can prepare a patch if you want, I am ok on Distutils side, but I wouldn't mind some help

Re: [Python-Dev] Help on issue 5941

2009-05-07 Thread Tarek Ziadé
On Thu, May 7, 2009 at 1:37 PM, David Cournapeau courn...@gmail.com wrote: On Thu, May 7, 2009 at 7:07 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: On Thu, May 7, 2009 at 11:50 AM, David Cournapeau courn...@gmail.com wrote: Then, in the customize_compiler function, set archiver to $AR +

Re: [Python-Dev] Help on issue 5941

2009-05-07 Thread David Cournapeau
On Thu, May 7, 2009 at 8:49 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: Notice that from the beginning, the unixcompiler class options are never used if the option has been customized in distutils.sysconfig and present in the Makefile, so we need to clean this behavior as well at some

Re: [Python-Dev] Help on issue 5941

2009-05-07 Thread Tarek Ziadé
On Thu, May 7, 2009 at 2:11 PM, David Cournapeau courn...@gmail.com wrote: But I don't know if that's easy to set up such as both python and numpy are built from sources. I don't know about the numpy part, but the PyBots project code could be a source of inspiration for the Python part

[Python-Dev] [RELEASED] Python 3.1 beta 1

2009-05-07 Thread Benjamin Peterson
On behalf of the Python development team, I'm thrilled to announce the first and only beta release of Python 3.1. Python 3.1 focuses on the stabilization and optimization of features and changes Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Walter Dörwald
M.-A. Lemburg wrote: Antoine Pitrou wrote: Martin v. Löwis martin at v.loewis.de writes: py b'\xed\xa0\x80'.decode(utf-8,surrogates) '\ud800' The point is, surrogates does not mean anything intuitive for an /error handler/. You seem to be the only one who finds this name explicit enough,

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread MRAB
Martin v. Löwis wrote: Wouldn't renaming the existing surrogates handler be an incompatible change, and thus inappropriate? No - it's new in Python 3.1. So what do you think about Antoine's proposal? +1 Although it looks like it would be without the '-' for consistency with existing error

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Michael Urman
On Thu, May 7, 2009 at 00:43, Martin v. Löwis mar...@v.loewis.de wrote: Michael Urman wrote: On Wed, May 6, 2009 at 15:42, Martin v. Löwis mar...@v.loewis.de wrote: Despite there being also an error handler called surrogates. Not that I have to be, but I'm not sold on the previous UTF-8 codec

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Michael Urman
On Thu, May 7, 2009 at 01:16, Martin v. Löwis mar...@v.loewis.de wrote: I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Walter Dörwald
Michael Urman wrote: [...] Well, there is a way to stack error handlers, although it's not pretty: [...] codecs.register_error(surrogates_then_replace, surrogates_then_replace) That mitigates my arguments significantly, although I'd rather see something like

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread MRAB
Walter Dörwald wrote: Michael Urman wrote: [...] Well, there is a way to stack error handlers, although it's not pretty: [...] codecs.register_error(surrogates_then_replace, surrogates_then_replace) That mitigates my arguments significantly, although I'd rather see

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
Well, there is a way to stack error handlers, although it's not pretty: [...] codecs.register_error(surrogates_then_replace, surrogates_then_replace) That mitigates my arguments significantly, although I'd rather see something like errors=('surrogates', 'replace')

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
I haven't come up with anything I like better than errors=lenient for the old utf8 behavior handler; would errors=nonvalidating be correct? I think either is fairly unspecific. For the utf8b error handler, I could see any of errors=roundtrip, errors=roundtripreplace, errors=tosurrogate,

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
The error handler for undoing this operation (ie. when converting a Unicode string to some other encoding) should probably use the same name based on symmetry and the fact that the escaping scheme is meant to be used for enabling round-trip safety. Could you please familiarize yourself with

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Stephen J. Turnbull
Walter Dörwald writes: surrogatepass (for the don't complain about lone half surrogates handler) and surrogatereplace sound OK to me. However the other ...replace handlers are destructive (i.e. when such a ...replace handler is used for encoding, decoding will not produce the original

[Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?

2009-05-07 Thread Neal Becker
format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? ___ Python-Dev mailing

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Terry Reedy
Martin v. Löwis wrote: So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs

Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?

2009-05-07 Thread Eric Smith
Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this:

Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?

2009-05-07 Thread Eric Smith
Eric Smith wrote: Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking

Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?

2009-05-07 Thread Terry Reedy
Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? Apparent typo reports should go to the tracker,

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Martin v. Löwis
Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Gregory P. Smith
On Thu, May 7, 2009 at 12:39 PM, Martin v. Löwis mar...@v.loewis.de wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine.  Thus, I considoer that and 'surrogates_excape' the best proposal the

[Python-Dev] py3k build broken

2009-05-07 Thread Eric Smith
Tarek: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: $ make ... gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -I./Modules/_io -c ./Modules/_io/textio.c

Re: [Python-Dev] py3k build broken

2009-05-07 Thread Tarek Ziadé
On Thu, May 7, 2009 at 11:36 PM, Eric Smith e...@trueblade.com wrote: Tarek: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: Sorry yes, I am on it now, the produced Makefile is broken, until then you can change it

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Terry Reedy
Martin v. Löwis wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the

Re: [Python-Dev] py3k build broken

2009-05-07 Thread Eric Smith
Tarek Ziadé wrote: On Thu, May 7, 2009 at 11:36 PM, Eric Smith e...@trueblade.com wrote: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: Sorry yes, I am on it now, the produced Makefile is broken, until then you can

Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?

2009-05-07 Thread Terry Reedy
Eric Smith wrote: Eric Smith wrote: Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That

Re: [Python-Dev] py3k build broken

2009-05-07 Thread Tarek Ziadé
On Thu, May 7, 2009 at 11:51 PM, Eric Smith e...@trueblade.com wrote: Tarek Ziadé wrote: On Thu, May 7, 2009 at 11:36 PM, Eric Smith e...@trueblade.com wrote: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: Sorry

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread MRAB
Terry Reedy wrote: Martin v. Löwis wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you

Re: [Python-Dev] py3k build broken

2009-05-07 Thread Eric Smith
Tarek Ziadé wrote: I have fixed configure by runing autoconf, everything should be fine now And indeed, it's working fine now, thanks. Sorry for the inconvenience. Not a problem. Anyone who volunteers for autoconf work gets a free pass from me. Eric.

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread M.-A. Lemburg
Martin v. Löwis wrote: The error handler for undoing this operation (ie. when converting a Unicode string to some other encoding) should probably use the same name based on symmetry and the fact that the escaping scheme is meant to be used for enabling round-trip safety. Could you please

Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?

2009-05-07 Thread Robert Kern
On 2009-05-07 18:29, Brett Cannon wrote: [my python-dev sabbatical is still in effect, so make sure I am at least cc'ed on any replies to this email] I cannot be the only person who has a need to run tests conditionally based on whether the file system is case-sensitive or not, so I feel like I

Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?

2009-05-07 Thread Antoine Pitrou
Robert Kern robert.kern at gmail.com writes: Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of the existence of code that does such a query,

[Python-Dev] Adding a sysconfig module in the stdlib

2009-05-07 Thread Tarek Ziadé
Hello, I am trying to refactor distutils.log in order to use logging but I have been bugged by the fact that site.py uses distutils.util.get_platform() in addbuilddir. The problem is the order of imports at initialization time : importing logging into distutils will make the initialization/build

Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?

2009-05-07 Thread Andrew Bennetts
Antoine Pitrou wrote: Robert Kern robert.kern at gmail.com writes: Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of the existence of

Re: [Python-Dev] PEP 383 update: utf8b is now the error handler

2009-05-07 Thread Glenn Linderman
On approximately 5/7/2009 3:27 PM, came the following characters from the keyboard of MRAB: Terry Reedy wrote: Martin v. Löwis wrote: So I'm happy to make it surrogatepass and surrogateescape as These seem adequate. It is not what I would choose or suggest, but it is adequate, and it is

Re: [Python-Dev] Adding a sysconfig module in the stdlib

2009-05-07 Thread Aahz
On Fri, May 08, 2009, Tarek Ziad? wrote: This module can be used by site.py, by distutils, and others, and will focus on this role. This should get kicked around on python-ideas; I don't think it will require a full-blown PEP unless there's disagreement about what it should contain. -- Aahz

Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?

2009-05-07 Thread John Arbash Meinel
Andrew Bennetts wrote: Antoine Pitrou wrote: Robert Kern robert.kern at gmail.com writes: Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of

Re: [Python-Dev] Adding a sysconfig module in the stdlib

2009-05-07 Thread David Cournapeau
On Fri, May 8, 2009 at 9:36 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: Hello, I am trying to refactor distutils.log in order to use logging but I have been bugged by the fact that site.py uses distutils.util.get_platform() in addbuilddir. The problem is the order of imports at