Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
By the way, what are the ASCII characters that are not suppported by Shift-JIS? Not many I suppose? (if I read the Wikipedia entry correctly, it's only the backslash and the tilde). The problem with this encoding is that bytes below 128 appear as second bytes of a two-byte encoding: py \x81@.decode(shift-jis) u'\u3000' py \x81A.decode(shift-jis) u'\u3001' So in on decoding, it may be the second byte (i.e. the ASCII byte) that causes a problem: py \x81/.decode(shift-jis) Traceback (most recent call last): File stdin, line 1, in module UnicodeDecodeError: 'shift_jis' codec can't decode bytes in position 0-1: illegal multibyte sequence For the shift-jis codec, that's actually not a problem, though: py b\x81/.decode(shift-jis,utf8b) '\udc81/' so the utf8b error handler will escape the first of the two bytes, and then pass the second byte to the codec again, which then decodes as ASCII. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it should have a different name. And one that is less cumbersome than utf_8b_encoder_invalid_codepoints I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It would be helpful if people would indicate support for Antoine's proposal. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On approximately 5/6/2009 10:53 PM, came the following characters from the keyboard of Martin v. Löwis: The error handler designed with utf-8 in mind has no name in the encode direction and is called utf_8b_decoder_invalid_bytes in the decode direction. By your reasoning, *that* should be its name in Python. The encoding error handler would then be named analogously utf_8b_encoder_invalid_codepoints. Even these, to me, would be better than confusing giving them the same name as the codec. So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it should have a different name. And one that is less cumbersome than utf_8b_encoder_invalid_codepoints -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Wouldn't renaming the existing surrogates handler be an incompatible change, and thus inappropriate? No - it's new in Python 3.1. So what do you think about Antoine's proposal? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On approximately 5/6/2009 11:16 PM, came the following characters from the keyboard of Martin v. Löwis: So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it should have a different name. And one that is less cumbersome than utf_8b_encoder_invalid_codepoints I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It would be helpful if people would indicate support for Antoine's proposal. Wouldn't renaming the existing surrogates handler be an incompatible change, and thus inappropriate? I assume that is the second handler you are referring to? bytes-as-lone-surrogates That would be very descriptive of the decode case for PEP 383, but very long. One problem with the word surrogates is that anything you add to it makes it too long. bytes-ls This is short, but a meaningless as is -- however, adding the understanding via documentation that ls means lone surrogates would make it meaningful, and mnemonic. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 11:50 AM, David Cournapeau courn...@gmail.com wrote: Then, in the customize_compiler function, set archiver to $AR + $ARFLAGS. IOW, just copying the logic used for e.g. ldshared, I can prepare a patch if you want, I am ok on Distutils side, but I wouldn't mind some help on the makefile/configure side Even if I could mimic what's in there, I am not confident enough yet. Please do so, by attaching your patch in the issue, Thanks Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 1:37 PM, David Cournapeau courn...@gmail.com wrote: On Thu, May 7, 2009 at 7:07 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: On Thu, May 7, 2009 at 11:50 AM, David Cournapeau courn...@gmail.com wrote: Then, in the customize_compiler function, set archiver to $AR + $ARFLAGS. IOW, just copying the logic used for e.g. ldshared, I can prepare a patch if you want, I am ok on Distutils side, but I wouldn't mind some help on the makefile/configure side Ok, I ended up making a patch for everything. I tested it on Linux, where it fixed the issue while keeping the customization (both AR and ARFLAGS can be customized through environment variables). numpy now builds under python 2.7, cheers, David ok thanks David, I'll complete your patch with the test I have written for this issue and commit it so it's included in 2.7/3.1. Notice that from the beginning, the unixcompiler class options are never used if the option has been customized in distutils.sysconfig and present in the Makefile, so we need to clean this behavior as well at some point, and document the customization features. By the way, do you happen to have a buildbot or something that builds numpy ? If not it'll be very interesting: I wouldn't mind having one numpy track running on the Python trunk and receiving mails if something is broken. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 8:49 PM, Tarek Ziadé ziade.ta...@gmail.com wrote: Notice that from the beginning, the unixcompiler class options are never used if the option has been customized in distutils.sysconfig and present in the Makefile, so we need to clean this behavior as well at some point, and document the customization features. Indeed, I have never bothered much with this part, though. Flags customization with distutils is too awkward to be useful in general for something like numpy IMHO, I just use scons instead when I need fine grained control. By the way, do you happen to have a buildbot or something that builds numpy ? We have a buildbot: http://buildbot.scipy.org/ But I don't know if that's easy to set up such as both python and numpy are built from sources. If not it'll be very interesting: I wouldn't mind having one numpy track running on the Python trunk and receiving mails if something is broken. Well, I would not mind either :) David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 2:11 PM, David Cournapeau courn...@gmail.com wrote: But I don't know if that's easy to set up such as both python and numpy are built from sources. I don't know about the numpy part, but the PyBots project code could be a source of inspiration for the Python part http://code.google.com/p/pybots/source/browse/trunk/master/community.cfg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASED] Python 3.1 beta 1
On behalf of the Python development team, I'm thrilled to announce the first and only beta release of Python 3.1. Python 3.1 focuses on the stabilization and optimization of features and changes Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. [1] Other features include an ordered dictionary implementation and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/dev/py3k/whatsnew/3.1.html or Misc/NEWS in the Python distribution. Please note that this is a beta release, and as such is not suitable for production environments. We continue to strive for a high degree of quality, but there are still some known problems and the feature sets have not been finalized. This beta is being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 3.1 might impact you. If you find things broken or incorrect, please submit a bug report at http://bugs.python.org For more information and downloadable distributions, see the Python 3.1 website: http://www.python.org/download/releases/3.1/ See PEP 375 for release schedule details: http://www.python.org/dev/peps/pep-0375/ Enjoy, -- Benjamin Benjamin Peterson benjamin at python.org Release Manager (on behalf of the entire python-dev team and 3.1's contributors) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
M.-A. Lemburg wrote: Antoine Pitrou wrote: Martin v. Löwis martin at v.loewis.de writes: py b'\xed\xa0\x80'.decode(utf-8,surrogates) '\ud800' The point is, surrogates does not mean anything intuitive for an /error handler/. You seem to be the only one who finds this name explicit enough, perhaps because you chose it. Most other handlers' names have verbs in them (ignore, replace, xmlcharrefreplace, etc.). Correct. The purpose of an error handler name is to indicate to the user what it does, hence the use of verbs. Walter started with xmlcharrefreplace, ie. no space names, so surrogatereplace would be the logically correct name for the replace with lone surrogates scheme invented by Markus Kuhn. surrogatepass (for the don't complain about lone half surrogates handler) and surrogatereplace sound OK to me. However the other ...replace handlers are destructive (i.e. when such a ...replace handler is used for encoding, decoding will not produce the original unicode string). The purpose of the PEP 383 error handler however is to be roundtrip safe, so maybe we should choose a slightly different name? How about surrogateescape? The error handler for undoing this operation (ie. when converting a Unicode string to some other encoding) should probably use the same name based on symmetry and the fact that the escaping scheme is meant to be used for enabling round-trip safety. We have only one error handler registry, but we *can* have one error handler for both directions (encoding and decoding) as the error handler can simply check whether it got passed a UnicodeEncodeError or UnicodeDecodeError object. BTW: It would also be appropriate to reference Markus Kuhn in the PEP as the inventor of the escaping scheme. Even if only to give the reader an idea of how that scheme works and why (the PEP on python.org currently doesn't explain this). It should also explain that the scheme is meant to assure round-trip safety and doesn't necessarily work when using transcoding, ie. reading using one encoding, writing using another. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: Wouldn't renaming the existing surrogates handler be an incompatible change, and thus inappropriate? No - it's new in Python 3.1. So what do you think about Antoine's proposal? +1 Although it looks like it would be without the '-' for consistency with existing error handlers. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On Thu, May 7, 2009 at 00:43, Martin v. Löwis mar...@v.loewis.de wrote:
Michael Urman wrote:
On Wed, May 6, 2009 at 15:42, Martin v. Löwis mar...@v.loewis.de wrote:
Despite there being also an error handler called surrogates.
Not that I have to be, but I'm not sold on the previous UTF-8 codec
behavior becoming an error handler of the name surrogates for two
reasons (I do respect the obvious PBP argument for the implementation,
and have no better name - lenient?).
PBP?
Practicality beats purity. From a purity standpoint, the legacy
invalid utf-8 seems more like an encoding than an error handler to me.
From a practicality standpoint, it's presumably much more convenient
to implement it on top of the new valid UTF-8 codec's behavior. And
then any error handler needs a name.
Well, there is a way to stack error handlers, although it's not pretty:
[...]
codecs.register_error(surrogates_then_replace,
surrogates_then_replace)
That mitigates my arguments significantly, although I'd rather see
something like errors=('surrogates', 'replace') chain the handlers
without additional registrations. But that's a different PEP or
arbitrary change. :)
The stacking argument also applies to the new utf8b behavior on encode
(only, as it handles all errors on decode). This may be a YAGNI
Indeed - in particular, as, in the primary application of this error
handler (i.e. file IO operations), there is no way of specifying
an addition error handler anyway.
Would it be useful to allow setting this somewhere? It'd be analogous
to setfsencoding, perhaps a setfsencodingerrors. It's not hard to
imagine an application working on Windows where all Unicode characters
are valid, and constructing backup filenames by adding some arbitrary
character, or receiving them from a user who doesn't understand
encodings. When this application is taken to a non-Unicode filesystem,
without the ability to say I really want a valid filename: so
replace, that could get messy. But it may still be a YAGNI, or a
don't do that.
--
Michael Urman
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On Thu, May 7, 2009 at 01:16, Martin v. Löwis mar...@v.loewis.de wrote: I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It would be helpful if people would indicate support for Antoine's proposal. Part of the problem is they both allow byte sequences to decode to invalid Unicode strings, and in particular they both affect the same byte subsequences, and that brought us to the crossroads where we wanted to name both of them surrogates. So I'll offer a few more colors, and try to get out of the way of choosing between them or the other proposed ones. :) I haven't come up with anything I like better than errors=lenient for the old utf8 behavior handler; would errors=nonvalidating be correct? It still seems to me that a new codec, perhaps utf8-lenient, reads better. For the utf8b error handler, I could see any of errors=roundtrip, errors=roundtripreplace, errors=tosurrogate, errors=surrogatereplace, errors=surrogateescape, errors=binaryreplace, errors=binaryescape. This includes Antoine's proposal (sans hyphen). -- Michael Urman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Michael Urman wrote:
[...]
Well, there is a way to stack error handlers, although it's not pretty:
[...]
codecs.register_error(surrogates_then_replace,
surrogates_then_replace)
That mitigates my arguments significantly, although I'd rather see
something like errors=('surrogates', 'replace') chain the handlers
without additional registrations. But that's a different PEP or
arbitrary change. :)
The first version of PEP 293 changed the errors argument to be a string
or callable. This would have simplified handler stacking somewhat
(because you don't have to register or lookup handlers) but it had the
disadvantage that many char * arguments in the C API would have had to
changed to PyObject *. Changing the errors argument to a list of
strings would have the same problem.
Servus,
Walter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Walter Dörwald wrote:
Michael Urman wrote:
[...]
Well, there is a way to stack error handlers, although it's not pretty:
[...]
codecs.register_error(surrogates_then_replace,
surrogates_then_replace)
That mitigates my arguments significantly, although I'd rather see
something like errors=('surrogates', 'replace') chain the handlers
without additional registrations. But that's a different PEP or
arbitrary change. :)
The first version of PEP 293 changed the errors argument to be a string
or callable. This would have simplified handler stacking somewhat
(because you don't have to register or lookup handlers) but it had the
disadvantage that many char * arguments in the C API would have had to
changed to PyObject *. Changing the errors argument to a list of
strings would have the same problem.
A comma-separated or space-separated string, eg 'surrogates replace' or
'surrogates,replace'? It could be treated as handler stacking
internally.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Well, there is a way to stack error handlers, although it's not pretty:
[...]
codecs.register_error(surrogates_then_replace,
surrogates_then_replace)
That mitigates my arguments significantly, although I'd rather see
something like errors=('surrogates', 'replace') chain the handlers
without additional registrations. But that's a different PEP or
arbitrary change. :)
I think you can provide something like
errors=combine_errors('surrogates', 'replace')
as a library function, and it doesn't have to be part of the
standard library.
The stacking argument also applies to the new utf8b behavior on encode
(only, as it handles all errors on decode). This may be a YAGNI
Indeed - in particular, as, in the primary application of this error
handler (i.e. file IO operations), there is no way of specifying
an addition error handler anyway.
Would it be useful to allow setting this somewhere?
I'm deliberately not proposing this as part of the PEP. First, it
has enough features already, and is approved as-is; plus YAGNI.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
I haven't come up with anything I like better than errors=lenient for the old utf8 behavior handler; would errors=nonvalidating be correct? I think either is fairly unspecific. For the utf8b error handler, I could see any of errors=roundtrip, errors=roundtripreplace, errors=tosurrogate, errors=surrogatereplace, errors=surrogateescape, errors=binaryreplace, errors=binaryescape. This includes Antoine's proposal (sans hyphen). Giving multiple choices does not exactly make this proposal readily implementable :-) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
The error handler for undoing this operation (ie. when converting a Unicode string to some other encoding) should probably use the same name based on symmetry and the fact that the escaping scheme is meant to be used for enabling round-trip safety. Could you please familiarize yourself with the implementation before commenting further? Thanks, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Walter Dörwald writes: surrogatepass (for the don't complain about lone half surrogates handler) and surrogatereplace sound OK to me. However the other ...replace handlers are destructive (i.e. when such a ...replace handler is used for encoding, decoding will not produce the original unicode string). That doesn't bother me in the slightest. Replace does not connote destructive or non-destructive to me; it connotes substitution. The fact that other error handlers happen to be destructive doesn't affect that at all for me. YMMV. The purpose of the PEP 383 error handler however is to be roundtrip safe, so maybe we should choose a slightly different name? How about surrogateescape? To me, escape has a strong connotation of a multicharacter representation of a single character, and that's not true here. How about surrogatetranslate? I still prefer surrogatereplace, as it's slightly easier for me to type. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: So are you proposing that I should rename the PEP 383 handler to utf_8b_encoder_invalid_codepoints? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it should have a different name. And one that is less cumbersome than utf_8b_encoder_invalid_codepoints Correct. Thank you Glenn. I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It would be helpful if people would indicate support for Antoine's proposal. Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this: format(10, '.3x') Traceback (most recent call last): File stdin, line 1, in module ValueError: Precision not allowed in integer format specifier ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Eric Smith wrote: Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this: format(10, '.3x') Traceback (most recent call last): File stdin, line 1, in module ValueError: Precision not allowed in integer format specifier So I guess it shouldn't say is ignored, it should be is not allowed. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Neal Becker wrote:
format_spec ::= [[fill]align][sign][#][0][width][.precision][type]
The precision is ignored for integer values.
In [36]: '%3x' % 10
Out[36]: ' a'
In [37]: '%.3x' % 10
Out[37]: '00a'
Apparently, precision is _not_ ignored?
Apparent typo reports should go to the tracker, along with version
information. In this case, the Format Specification Mini-Language is
for the new str.format() and format() facilities, not for % formatting,
which is described in Old String Formatting Operations. Ironically, you
report does point to a doc problem: precision is actually not allowed
for integer types.
3.0.1
format(10, '3x')
' a'
format(10, '.3x')
Traceback (most recent call last):
File pyshell#2, line 1, in module
format(10, '.3x')
ValueError: Precision not allowed in integer format specifier
'{0:3x}'.format(10)
' a'
'{0:.3x}'.format(10)
Traceback (most recent call last):
File pyshell#4, line 1, in module
'{0:.3x}'.format(10)
ValueError: Precision not allowed in integer format specifier
http://bugs.python.org/issue5963
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it surrogatepass and surrogateescape as proposed by Walter. I'm sure you didn't really mean the spelling of excape to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On Thu, May 7, 2009 at 12:39 PM, Martin v. Löwis mar...@v.loewis.de wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it surrogatepass and surrogateescape as proposed by Walter. I'm sure you didn't really mean the spelling of excape to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. singular is good. +1 on these names. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] py3k build broken
Tarek:
With you ARFLAGS change, I now get the following error on a 32 bit
Fedora 6 box. I've done make distclean and ./configure:
$ make
...
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -I./Modules/_io -c
./Modules/_io/textio.c -o Modules/textio.o
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -I./Modules/_io -c
./Modules/_io/stringio.c -o Modules/stringio.o
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/zipimport.c -o
Modules/zipimport.o
./Modules/zipimport.c: In function ‘get_module_code’:
./Modules/zipimport.c:1132: warning: format ‘%c’ expects type ‘int’, but
argument 3 has type ‘long int’
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/symtablemodule.c
-o Modules/symtablemodule.o
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/xxsubtype.c -o
Modules/xxsubtype.o
gcc -pthread -c -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -DSVNVERSION=\`LC_ALL=C
svnversion .`\ -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
rm -f libpython3.1.a
ar @ARFLAGS@ libpython3.1.a Modules/getbuildinfo.o
ar: illegal option -- @
Usage: ar [emulation options] [-]{dmpqrstx}[abcfilNoPsSuvV]
[member-name] [count] archive-file file...
ar -M [mri-script]
commands:
d- delete file(s) from the archive
m[ab]- move file(s) in the archive
p- print file(s) found in the archive
q[f] - quick append file(s) to the archive
r[ab][f][u] - replace existing or insert new file(s) into the archive
t- display contents of archive
x[o] - extract file(s) from the archive
command specific modifiers:
[a] - put file(s) after [member-name]
[b] - put file(s) before [member-name] (same as [i])
[N] - use instance [count] of name
[f] - truncate inserted file names
[P] - use full path names when matching
[o] - preserve original dates
[u] - only replace files that are newer than current archive
contents
generic modifiers:
[c] - do not warn if the library had to be created
[s] - create an archive index (cf. ranlib)
[S] - do not build a symbol table
[v] - be verbose
[V] - display the version number
@file - read options from file
emulation options:
No emulation specific options
ar: supported targets: elf32-i386 a.out-i386-linux efi-app-ia32
elf32-little elf32-big srec symbolsrec tekhex binary ihex trad-core
make: *** [libpython3.1.a] Error 1
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
On Thu, May 7, 2009 at 11:36 PM, Eric Smith e...@trueblade.com wrote: Tarek: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: Sorry yes, I am on it now, the produced Makefile is broken, until then you can change it line 71 arflags=...@arflags@ ARFLAGS= cr Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it surrogatepass and surrogateescape as proposed by Walter. I'm sure you didn't really mean the spelling of excape to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. Those minor tweaks for consistency with existing names are what I meant by 'improve' (with good arguments) and I approve of them also. +1 on stopping here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
Tarek Ziadé wrote: On Thu, May 7, 2009 at 11:36 PM, Eric Smith e...@trueblade.com wrote: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: Sorry yes, I am on it now, the produced Makefile is broken, until then you can change it ... No problem. I'll wait. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Eric Smith wrote: Eric Smith wrote: Neal Becker wrote: format_spec ::= [[fill]align][sign][#][0][width][.precision][type] The precision is ignored for integer values. In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this: format(10, '.3x') Traceback (most recent call last): File stdin, line 1, in module ValueError: Precision not allowed in integer format specifier So I guess it shouldn't say is ignored, it should be is not allowed. My exact suggestion in http://bugs.python.org/issue5963 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
On Thu, May 7, 2009 at 11:51 PM, Eric Smith e...@trueblade.com wrote: Tarek Ziadé wrote: On Thu, May 7, 2009 at 11:36 PM, Eric Smith e...@trueblade.com wrote: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done make distclean and ./configure: Sorry yes, I am on it now, the produced Makefile is broken, until then you can change it ... No problem. I'll wait. I have fixed configure by runing autoconf, everything should be fine now Sorry for the inconvenience. Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Terry Reedy wrote: Martin v. Löwis wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it surrogatepass and surrogateescape as proposed by Walter. I'm sure you didn't really mean the spelling of excape to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. Those minor tweaks for consistency with existing names are what I meant by 'improve' (with good arguments) and I approve of them also. +1 on stopping here. We argue because we care. :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
Tarek Ziadé wrote: I have fixed configure by runing autoconf, everything should be fine now And indeed, it's working fine now, thanks. Sorry for the inconvenience. Not a problem. Anyone who volunteers for autoconf work gets a free pass from me. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: The error handler for undoing this operation (ie. when converting a Unicode string to some other encoding) should probably use the same name based on symmetry and the fact that the escaping scheme is meant to be used for enabling round-trip safety. Could you please familiarize yourself with the implementation before commenting further? I did and it already uses the same (wrong) name for both encoding and decoding handlers which is good. The reason for my above comment was that the thread mentions two different names for the handler depending on the direction, e.g. surrogatereplace and surrogatepass. I guess that surrogatepass was just an attempt to find a new name for the surrogates error handler (which also doesn't match the naming scheme) and that got me confused. I'd use allowlonesurrogates as name for the surrogates error handler and lonesurrogatereplace for the utf8b one. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 08 2009) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2009-06-29: EuroPython 2009, Birmingham, UK51 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
On 2009-05-07 18:29, Brett Cannon wrote: [my python-dev sabbatical is still in effect, so make sure I am at least cc'ed on any replies to this email] I cannot be the only person who has a need to run tests conditionally based on whether the file system is case-sensitive or not, so I feel like I am re-inventing the wheel for issue 5442 to handle OS X with a case-sensitive filesystem. Is there a boolean somewhere that I can simply check or get to know whether the filesystem is case-sensitive? Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of the existence of code that does such a query, though. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
Robert Kern robert.kern at gmail.com writes: Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of the existence of code that does such a query, though. Or you can just be practical and test for it. Create a file foobar and see if you can open FOOBAR in read mode... Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a sysconfig module in the stdlib
Hello, I am trying to refactor distutils.log in order to use logging but I have been bugged by the fact that site.py uses distutils.util.get_platform() in addbuilddir. The problem is the order of imports at initialization time : importing logging into distutils will make the initialization/build fail because site.py wil break when trying to import logging, then time. Anyways, So why site.py looks into distutils ? because distutils has a few functions to get some info about the platform and about the Makefile and some other header files like pyconfig.h etc. But I don't think it's the best place for this, and I have a proposal : let's create a dedicated sysconfig module in the standard library that will provide all the (refactored) functions located in distutils.sysconfig (but not customize_compiler) and disutils.util.get_platform. This module can be used by site.py, by distutils, and others, and will focus on this role. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
Antoine Pitrou wrote: Robert Kern robert.kern at gmail.com writes: Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of the existence of code that does such a query, though. Or you can just be practical and test for it. Create a file foobar and see if you can open FOOBAR in read mode... Agreed. That is how Bazaar's test suite detects this, and it works well. -Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On approximately 5/7/2009 3:27 PM, came the following characters from the keyboard of MRAB: Terry Reedy wrote: Martin v. Löwis wrote: So I'm happy to make it surrogatepass and surrogateescape as These seem adequate. It is not what I would choose or suggest, but it is adequate, and it is unlikely you can delight everyone with your choice of names, or even someone else's choice of names. These at least have a logical justification for their meaning, and can be documented reasonably. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a sysconfig module in the stdlib
On Fri, May 08, 2009, Tarek Ziad? wrote: This module can be used by site.py, by distutils, and others, and will focus on this role. This should get kicked around on python-ideas; I don't think it will require a full-blown PEP unless there's disagreement about what it should contain. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ It is easier to optimize correct code than to correct optimized code. --Bill Harlan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
Andrew Bennetts wrote:
Antoine Pitrou wrote:
Robert Kern robert.kern at gmail.com writes:
Since one may have more than one filesystem side-by-side, this can't be just
be
a system-wide boolean somewhere. One would have to query the target
directory
for this information. I am not aware of the existence of code that does such
a
query, though.
Or you can just be practical and test for it. Create a file foobar and see
if
you can open FOOBAR in read mode...
Agreed. That is how Bazaar's test suite detects this, and it works well.
-Andrew.
Actually, I believe we do:
open('format', 'wb').close()
try:
os.lstat('FoRmAt')
except IOError, e:
if e.errno == errno.ENOENT:
...
I don't know that it really matters, just wanted to indicate we use
'lstat' rather than 'open()' to check. I could be wrong about the test
suite, but I know that is what we do for 'live' files. (We always create
a format file, so we know it is there to 'stat' it via a different name.)
John
=:-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a sysconfig module in the stdlib
On Fri, May 8, 2009 at 9:36 AM, Tarek Ziadé ziade.ta...@gmail.com wrote: Hello, I am trying to refactor distutils.log in order to use logging but I have been bugged by the fact that site.py uses distutils.util.get_platform() in addbuilddir. The problem is the order of imports at initialization time : importing logging into distutils will make the initialization/build fail because site.py wil break when trying to import logging, then time. Anyways, So why site.py looks into distutils ? because distutils has a few functions to get some info about the platform and about the Makefile and some other header files like pyconfig.h etc. But I don't think it's the best place for this, and I have a proposal : let's create a dedicated sysconfig module in the standard library that will provide all the (refactored) functions located in distutils.sysconfig (but not customize_compiler) and disutils.util.get_platform. If we are talking about putting this into the stdlib proper, I would suggest thinking about putting information for every platform in sysconfig, instead of just Unix. I understand it is not an easy problem (because windows builds are totally different than every other platform), but it would really help for interoperability with other build tools. If sysconfig is to become independent of distutils, it should be cross platform and not unix specific. cheers, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
