Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
M.-A. Lemburg writes: > I'd use "allowlonesurrogates" as name for the "surrogates" error > handler and "lonesurrogatereplace" for the "utf8b" one. +1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a "sysconfig" module in the stdlib
On Fri, May 8, 2009 at 9:36 AM, Tarek Ziadé wrote: > Hello, > > I am trying to refactor distutils.log in order to use logging but I > have been bugged by the fact that site.py uses > distutils.util.get_platform() in "addbuilddir". > The problem is the order of imports at initialization time : importing > "logging" into distutils will make the initialization/build fail > because site.py wil break when > trying to import "logging", then "time". > > Anyways, > So why site.py looks into distutils ? because distutils has a few > functions to get some info about the platform and about the Makefile > and some > other header files like pyconfig.h etc. > > But I don't think it's the best place for this, and I have a proposal : > > let's create a dedicated "sysconfig" module in the standard library > that will provide all the (refactored) functions located in > distutils.sysconfig (but not customize_compiler) > and disutils.util.get_platform. If we are talking about putting this into the stdlib proper, I would suggest thinking about putting information for every platform in sysconfig, instead of just Unix. I understand it is not an easy problem (because windows builds are totally different than every other platform), but it would really help for interoperability with other build tools. If sysconfig is to become independent of distutils, it should be cross platform and not unix specific. cheers, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
Andrew Bennetts wrote:
> Antoine Pitrou wrote:
>> Robert Kern gmail.com> writes:
>>> Since one may have more than one filesystem side-by-side, this can't be just
>> be
>>> a system-wide boolean somewhere. One would have to query the target
>>> directory
>>> for this information. I am not aware of the existence of code that does such
>> a
>>> query, though.
>> Or you can just be practical and test for it. Create a file "foobar" and see
>> if
>> you can open "FOOBAR" in read mode...
>
> Agreed. That is how Bazaar's test suite detects this, and it works well.
>
> -Andrew.
Actually, I believe we do:
open('format', 'wb').close()
try:
os.lstat('FoRmAt')
except IOError, e:
if e.errno == errno.ENOENT:
...
I don't know that it really matters, just wanted to indicate we use
'lstat' rather than 'open()' to check. I could be wrong about the test
suite, but I know that is what we do for 'live' files. (We always create
a format file, so we know it is there to 'stat' it via a different name.)
John
=:->
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding a "sysconfig" module in the stdlib
On Fri, May 08, 2009, Tarek Ziad? wrote: > > This module can be used by site.py, by distutils, and others, and will > focus on this role. This should get kicked around on python-ideas; I don't think it will require a full-blown PEP unless there's disagreement about what it should contain. -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On approximately 5/7/2009 3:27 PM, came the following characters from the keyboard of MRAB: Terry Reedy wrote: Martin v. Löwis wrote: So I'm happy to make it "surrogatepass" and "surrogateescape" as These seem adequate. It is not what I would choose or suggest, but it is adequate, and it is unlikely you can delight everyone with your choice of names, or even someone else's choice of names. These at least have a logical justification for their meaning, and can be documented reasonably. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
Antoine Pitrou wrote: > Robert Kern gmail.com> writes: > > > > Since one may have more than one filesystem side-by-side, this can't be just > be > > a system-wide boolean somewhere. One would have to query the target > > directory > > for this information. I am not aware of the existence of code that does such > a > > query, though. > > Or you can just be practical and test for it. Create a file "foobar" and see > if > you can open "FOOBAR" in read mode... Agreed. That is how Bazaar's test suite detects this, and it works well. -Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a "sysconfig" module in the stdlib
Hello, I am trying to refactor distutils.log in order to use logging but I have been bugged by the fact that site.py uses distutils.util.get_platform() in "addbuilddir". The problem is the order of imports at initialization time : importing "logging" into distutils will make the initialization/build fail because site.py wil break when trying to import "logging", then "time". Anyways, So why site.py looks into distutils ? because distutils has a few functions to get some info about the platform and about the Makefile and some other header files like pyconfig.h etc. But I don't think it's the best place for this, and I have a proposal : let's create a dedicated "sysconfig" module in the standard library that will provide all the (refactored) functions located in distutils.sysconfig (but not customize_compiler) and disutils.util.get_platform. This module can be used by site.py, by distutils, and others, and will focus on this role. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
Robert Kern gmail.com> writes: > > Since one may have more than one filesystem side-by-side, this can't be just be > a system-wide boolean somewhere. One would have to query the target directory > for this information. I am not aware of the existence of code that does such a > query, though. Or you can just be practical and test for it. Create a file "foobar" and see if you can open "FOOBAR" in read mode... Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Easy way to detect filesystem case-sensitivity?
On 2009-05-07 18:29, Brett Cannon wrote: [my python-dev sabbatical is still in effect, so make sure I am at least cc'ed on any replies to this email] I cannot be the only person who has a need to run tests conditionally based on whether the file system is case-sensitive or not, so I feel like I am re-inventing the wheel for issue 5442 to handle OS X with a case-sensitive filesystem. Is there a boolean somewhere that I can simply check or get to know whether the filesystem is case-sensitive? Since one may have more than one filesystem side-by-side, this can't be just be a system-wide boolean somewhere. One would have to query the target directory for this information. I am not aware of the existence of code that does such a query, though. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Easy way to detect filesystem case-sensitivity?
[my python-dev sabbatical is still in effect, so make sure I am at least cc'ed on any replies to this email] I cannot be the only person who has a need to run tests conditionally based on whether the file system is case-sensitive or not, so I feel like I am re-inventing the wheel for issue 5442 to handle OS X with a case-sensitive filesystem. Is there a boolean somewhere that I can simply check or get to know whether the filesystem is case-sensitive? -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: >> The error handler for undoing this operation (ie. when converting >> a Unicode string to some other encoding) should probably use the >> same name based on symmetry and the fact that the escaping >> scheme is meant to be used for enabling round-trip safety. > > Could you please familiarize yourself with the implementation > before commenting further? I did and it already uses the same (wrong) name for both encoding and decoding handlers which is good. The reason for my above comment was that the thread mentions two different names for the handler depending on the direction, e.g. "surrogatereplace" and "surrogatepass". I guess that "surrogatepass" was just an attempt to find a new name for the "surrogates" error handler (which also doesn't match the naming scheme) and that got me confused. I'd use "allowlonesurrogates" as name for the "surrogates" error handler and "lonesurrogatereplace" for the "utf8b" one. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 08 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2009-06-29: EuroPython 2009, Birmingham, UK51 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
Tarek Ziadé wrote: I have fixed configure by runing autoconf, everything should be fine now And indeed, it's working fine now, thanks. Sorry for the inconvenience. Not a problem. Anyone who volunteers for autoconf work gets a free pass from me. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Terry Reedy wrote: Martin v. Löwis wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it "surrogatepass" and "surrogateescape" as proposed by Walter. I'm sure you didn't really mean the spelling of "excape" to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. Those minor tweaks for consistency with existing names are what I meant by 'improve' (with good arguments) and I approve of them also. +1 on stopping here. We argue because we care. :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
On Thu, May 7, 2009 at 11:51 PM, Eric Smith wrote: > Tarek Ziadé wrote: >> >> On Thu, May 7, 2009 at 11:36 PM, Eric Smith wrote: >>> >>> With you ARFLAGS change, I now get the following error on a 32 bit Fedora >>> 6 >>> box. I've done "make distclean" and "./configure": >> >> Sorry yes, I am on it now, the produced Makefile is broken, until then >> you can change it > > ... > > No problem. I'll wait. I have fixed configure by runing autoconf, everything should be fine now Sorry for the inconvenience. Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Eric Smith wrote: Eric Smith wrote: Neal Becker wrote: "format_spec ::= [[fill]align][sign][#][0][width][.precision][type]" "The precision is ignored for integer values." In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this: >>> format(10, '.3x') Traceback (most recent call last): File "", line 1, in ValueError: Precision not allowed in integer format specifier So I guess it shouldn't say "is ignored", it should be "is not allowed". My exact suggestion in http://bugs.python.org/issue5963 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
Tarek Ziadé wrote: On Thu, May 7, 2009 at 11:36 PM, Eric Smith wrote: With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 box. I've done "make distclean" and "./configure": Sorry yes, I am on it now, the produced Makefile is broken, until then you can change it ... No problem. I'll wait. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it "surrogatepass" and "surrogateescape" as proposed by Walter. I'm sure you didn't really mean the spelling of "excape" to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. Those minor tweaks for consistency with existing names are what I meant by 'improve' (with good arguments) and I approve of them also. +1 on stopping here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k build broken
On Thu, May 7, 2009 at 11:36 PM, Eric Smith wrote: > Tarek: > > With you ARFLAGS change, I now get the following error on a 32 bit Fedora 6 > box. I've done "make distclean" and "./configure": Sorry yes, I am on it now, the produced Makefile is broken, until then you can change it <<< line 71 arflags=...@arflags@ <<< ARFLAGS= cr <<< Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] py3k build broken
Tarek:
With you ARFLAGS change, I now get the following error on a 32 bit
Fedora 6 box. I've done "make distclean" and "./configure":
$ make
...
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -I./Modules/_io -c
./Modules/_io/textio.c -o Modules/textio.o
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -I./Modules/_io -c
./Modules/_io/stringio.c -o Modules/stringio.o
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/zipimport.c -o
Modules/zipimport.o
./Modules/zipimport.c: In function ‘get_module_code’:
./Modules/zipimport.c:1132: warning: format ‘%c’ expects type ‘int’, but
argument 3 has type ‘long int’
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/symtablemodule.c
-o Modules/symtablemodule.o
gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/xxsubtype.c -o
Modules/xxsubtype.o
gcc -pthread -c -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I.
-IInclude -I./Include -DPy_BUILD_CORE -DSVNVERSION=\"`LC_ALL=C
svnversion .`\" -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
rm -f libpython3.1.a
ar @ARFLAGS@ libpython3.1.a Modules/getbuildinfo.o
ar: illegal option -- @
Usage: ar [emulation options] [-]{dmpqrstx}[abcfilNoPsSuvV]
[member-name] [count] archive-file file...
ar -M [ [u] - only replace files that are newer than current archive
contents
generic modifiers:
[c] - do not warn if the library had to be created
[s] - create an archive index (cf. ranlib)
[S] - do not build a symbol table
[v] - be verbose
[V] - display the version number
@ - read options from
emulation options:
No emulation specific options
ar: supported targets: elf32-i386 a.out-i386-linux efi-app-ia32
elf32-little elf32-big srec symbolsrec tekhex binary ihex trad-core
make: *** [libpython3.1.a] Error 1
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On Thu, May 7, 2009 at 12:39 PM, "Martin v. Löwis" wrote: >> Given your explanation of what the new 'surrogates' handler does (pass >> rather than reject erroneous surrogates), I think 'surrogates_pass' is >> fine. Thus, I considoer that and 'surrogates_excape' the best proposal >> the best so far and suggest that you make this pair the current status >> quo to be argued against and improved ... or not. > > That's exactly what I want to avoid: more bike-shedding. If this is now > changed, it cannot be possibly be argued against and improved - it would > be final, end of discussion (please!!!). > > So I'm happy to make it "surrogatepass" and "surrogateescape" as > proposed by Walter. I'm sure you didn't really mean the spelling of > "excape" to be taken literally - whether or not you meant the plural > and the underscore literally, I cannot tell. Stephen Turnbull approved > singular, so that's good enough for me. singular is good. +1 on these names. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
> Given your explanation of what the new 'surrogates' handler does (pass > rather than reject erroneous surrogates), I think 'surrogates_pass' is > fine. Thus, I considoer that and 'surrogates_excape' the best proposal > the best so far and suggest that you make this pair the current status > quo to be argued against and improved ... or not. That's exactly what I want to avoid: more bike-shedding. If this is now changed, it cannot be possibly be argued against and improved - it would be final, end of discussion (please!!!). So I'm happy to make it "surrogatepass" and "surrogateescape" as proposed by Walter. I'm sure you didn't really mean the spelling of "excape" to be taken literally - whether or not you meant the plural and the underscore literally, I cannot tell. Stephen Turnbull approved singular, so that's good enough for me. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Neal Becker wrote:
"format_spec ::= [[fill]align][sign][#][0][width][.precision][type]"
"The precision is ignored for integer values."
In [36]: '%3x' % 10
Out[36]: ' a'
In [37]: '%.3x' % 10
Out[37]: '00a'
Apparently, precision is _not_ ignored?
Apparent typo reports should go to the tracker, along with version
information. In this case, the Format Specification Mini-Language is
for the new str.format() and format() facilities, not for % formatting,
which is described in Old String Formatting Operations. Ironically, you
report does point to a doc problem: precision is actually not allowed
for integer types.
3.0.1
>> format(10, '3x')
' a'
>>> format(10, '.3x')
Traceback (most recent call last):
File "", line 1, in
format(10, '.3x')
ValueError: Precision not allowed in integer format specifier
>>> '{0:3x}'.format(10)
' a'
>>> '{0:.3x}'.format(10)
Traceback (most recent call last):
File "", line 1, in
'{0:.3x}'.format(10)
ValueError: Precision not allowed in integer format specifier
http://bugs.python.org/issue5963
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Eric Smith wrote: Neal Becker wrote: "format_spec ::= [[fill]align][sign][#][0][width][.precision][type]" "The precision is ignored for integer values." In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this: >>> format(10, '.3x') Traceback (most recent call last): File "", line 1, in ValueError: Precision not allowed in integer format specifier So I guess it shouldn't say "is ignored", it should be "is not allowed". ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
Neal Becker wrote: "format_spec ::= [[fill]align][sign][#][0][width][.precision][type]" "The precision is ignored for integer values." In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? That section is talking about this: >>> format(10, '.3x') Traceback (most recent call last): File "", line 1, in ValueError: Precision not allowed in integer format specifier ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: So are you proposing that I should rename the PEP 383 handler to "utf_8b_encoder_invalid_codepoints"? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it should have a different name. And one that is less cumbersome than "utf_8b_encoder_invalid_codepoints" Correct. Thank you Glenn. I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It would be helpful if people would indicate support for Antoine's proposal. Given your explanation of what the new 'surrogates' handler does (pass rather than reject erroneous surrogates), I think 'surrogates_pass' is fine. Thus, I considoer that and 'surrogates_excape' the best proposal the best so far and suggest that you make this pair the current status quo to be argued against and improved ... or not. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] typo in 8.1.3.1. Format Specification Mini-Language?
"format_spec ::= [[fill]align][sign][#][0][width][.precision][type]" "The precision is ignored for integer values." In [36]: '%3x' % 10 Out[36]: ' a' In [37]: '%.3x' % 10 Out[37]: '00a' Apparently, precision is _not_ ignored? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Walter Dörwald writes: > "surrogatepass" (for the "don't complain about lone half surrogates" > handler) and "surrogatereplace" sound OK to me. However the other > "...replace" handlers are destructive (i.e. when such a "...replace" > handler is used for encoding, decoding will not produce the original > unicode string). That doesn't bother me in the slightest. "Replace" does not connote "destructive" or "non-destructive" to me; it connotes "substitution". The fact that other error handlers happen to be destructive doesn't affect that at all for me. YMMV. > The purpose of the PEP 383 error handler however is to be roundtrip > safe, so maybe we should choose a slightly different name? How > about "surrogateescape"? To me, "escape" has a strong connotation of a multicharacter representation of a single character, and that's not true here. How about "surrogatetranslate"? I still prefer "surrogatereplace", as it's slightly easier for me to type. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
> The error handler for undoing this operation (ie. when converting > a Unicode string to some other encoding) should probably use the > same name based on symmetry and the fact that the escaping > scheme is meant to be used for enabling round-trip safety. Could you please familiarize yourself with the implementation before commenting further? Thanks, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
> I haven't come up with anything I like better than errors="lenient" > for the old utf8 behavior handler; would errors="nonvalidating" be > correct? I think either is fairly unspecific. > For the utf8b error handler, I could see any of errors="roundtrip", > errors="roundtripreplace", errors="tosurrogate", > errors="surrogatereplace", errors="surrogateescape", > errors="binaryreplace", errors="binaryescape". This includes Antoine's > proposal (sans hyphen). Giving multiple choices does not exactly make this proposal readily implementable :-) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
>> Well, there is a way to stack error handlers, although it's not pretty:
>> [...]
>> codecs.register_error("surrogates_then_replace",
>> surrogates_then_replace)
>
> That mitigates my arguments significantly, although I'd rather see
> something like errors=('surrogates', 'replace') chain the handlers
> without additional registrations. But that's a different PEP or
> arbitrary change. :)
I think you can provide something like
errors=combine_errors('surrogates', 'replace')
as a library function, and it doesn't have to be part of the
standard library.
>>> The stacking argument also applies to the new utf8b behavior on encode
>>> (only, as it handles all errors on decode). This may be a YAGNI
>> Indeed - in particular, as, in the primary application of this error
>> handler (i.e. file IO operations), there is no way of specifying
>> an addition error handler anyway.
>
> Would it be useful to allow setting this somewhere?
I'm deliberately not proposing this as part of the PEP. First, it
has enough features already, and is approved as-is; plus YAGNI.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Walter Dörwald wrote:
Michael Urman wrote:
[...]
Well, there is a way to stack error handlers, although it's not pretty:
[...]
codecs.register_error("surrogates_then_replace",
surrogates_then_replace)
That mitigates my arguments significantly, although I'd rather see
something like errors=('surrogates', 'replace') chain the handlers
without additional registrations. But that's a different PEP or
arbitrary change. :)
The first version of PEP 293 changed the errors argument to be a string
or callable. This would have simplified handler stacking somewhat
(because you don't have to register or lookup handlers) but it had the
disadvantage that many "char *" arguments in the C API would have had to
changed to "PyObject *". Changing the errors argument to a list of
strings would have the same problem.
A comma-separated or space-separated string, eg 'surrogates replace' or
'surrogates,replace'? It could be treated as handler stacking
internally.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Michael Urman wrote:
> [...]
>> Well, there is a way to stack error handlers, although it's not pretty:
>> [...]
>> codecs.register_error("surrogates_then_replace",
>> surrogates_then_replace)
>
> That mitigates my arguments significantly, although I'd rather see
> something like errors=('surrogates', 'replace') chain the handlers
> without additional registrations. But that's a different PEP or
> arbitrary change. :)
The first version of PEP 293 changed the errors argument to be a string
or callable. This would have simplified handler stacking somewhat
(because you don't have to register or lookup handlers) but it had the
disadvantage that many "char *" arguments in the C API would have had to
changed to "PyObject *". Changing the errors argument to a list of
strings would have the same problem.
Servus,
Walter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On Thu, May 7, 2009 at 01:16, "Martin v. Löwis" wrote: > I'm still at a loss what name to give it, though. I understand that > I have to rename both error handlers, but I'm uncertain what I should > rename them to. So proposals that rename only one of them aren't > that helpful. It would be helpful if people would indicate support > for Antoine's proposal. Part of the problem is they both allow byte sequences to decode to invalid Unicode strings, and in particular they both affect the same byte subsequences, and that brought us to the crossroads where we wanted to name both of them "surrogates". So I'll offer a few more colors, and try to get out of the way of choosing between them or the other proposed ones. :) I haven't come up with anything I like better than errors="lenient" for the old utf8 behavior handler; would errors="nonvalidating" be correct? It still seems to me that a new codec, perhaps "utf8-lenient", reads better. For the utf8b error handler, I could see any of errors="roundtrip", errors="roundtripreplace", errors="tosurrogate", errors="surrogatereplace", errors="surrogateescape", errors="binaryreplace", errors="binaryescape". This includes Antoine's proposal (sans hyphen). -- Michael Urman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On Thu, May 7, 2009 at 00:43, "Martin v. Löwis" wrote:
> Michael Urman wrote:
>> On Wed, May 6, 2009 at 15:42, "Martin v. Löwis" wrote:
>>> Despite there being also an error handler called "surrogates".
>>
>> Not that I have to be, but I'm not sold on the previous UTF-8 codec
>> behavior becoming an error handler of the name "surrogates" for two
>> reasons (I do respect the obvious PBP argument for the implementation,
>> and have no better name - "lenient"?).
>
> PBP?
Practicality beats purity. From a purity standpoint, the legacy
invalid utf-8 seems more like an encoding than an error handler to me.
From a practicality standpoint, it's presumably much more convenient
to implement it on top of the new valid UTF-8 codec's behavior. And
then any error handler needs a name.
> Well, there is a way to stack error handlers, although it's not pretty:
> [...]
> codecs.register_error("surrogates_then_replace",
> surrogates_then_replace)
That mitigates my arguments significantly, although I'd rather see
something like errors=('surrogates', 'replace') chain the handlers
without additional registrations. But that's a different PEP or
arbitrary change. :)
>> The stacking argument also applies to the new utf8b behavior on encode
>> (only, as it handles all errors on decode). This may be a YAGNI
>
> Indeed - in particular, as, in the primary application of this error
> handler (i.e. file IO operations), there is no way of specifying
> an addition error handler anyway.
Would it be useful to allow setting this somewhere? It'd be analogous
to setfsencoding, perhaps a setfsencodingerrors. It's not hard to
imagine an application working on Windows where all Unicode characters
are valid, and constructing backup filenames by adding some arbitrary
character, or receiving them from a user who doesn't understand
encodings. When this application is taken to a non-Unicode filesystem,
without the ability to say "I really want a valid filename: so
replace", that could get messy. But it may still be a YAGNI, or a
"don't do that."
--
Michael Urman
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Martin v. Löwis wrote: Wouldn't renaming the existing "surrogates" handler be an incompatible change, and thus inappropriate? No - it's new in Python 3.1. So what do you think about Antoine's proposal? +1 Although it looks like it would be without the '-' for consistency with existing error handlers. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
M.-A. Lemburg wrote:
> Antoine Pitrou wrote:
>> Martin v. Löwis v.loewis.de> writes:
>>> py> b'\xed\xa0\x80'.decode("utf-8","surrogates")
>>> '\ud800'
>> The point is, "surrogates" does not mean anything intuitive for an /error
>> handler/. You seem to be the only one who finds this name explicit enough,
>> perhaps because you chose it.
>> Most other handlers' names have verbs in them ("ignore", "replace",
>> "xmlcharrefreplace", etc.).
>
> Correct.
>
> The purpose of an error handler name is to indicate to the user
> what it does, hence the use of verbs.
>
> Walter started with "xmlcharrefreplace", ie. no space names, so
> "surrogatereplace" would be the logically correct name for the
> "replace with lone surrogates" scheme invented by Markus Kuhn.
"surrogatepass" (for the "don't complain about lone half surrogates"
handler) and "surrogatereplace" sound OK to me. However the other
"...replace" handlers are destructive (i.e. when such a "...replace"
handler is used for encoding, decoding will not produce the original
unicode string). The purpose of the PEP 383 error handler however is to
be roundtrip safe, so maybe we should choose a slightly different name?
How about "surrogateescape"?
> The error handler for undoing this operation (ie. when converting
> a Unicode string to some other encoding) should probably use the
> same name based on symmetry and the fact that the escaping
> scheme is meant to be used for enabling round-trip safety.
We have only one error handler registry, but we *can* have one error
handler for both directions (encoding and decoding) as the error handler
can simply check whether it got passed a UnicodeEncodeError or
UnicodeDecodeError object.
> BTW: It would also be appropriate to reference Markus Kuhn in the PEP
> as the inventor of the escaping scheme.
>
> Even if only to give the reader an idea of how that scheme works and
> why (the PEP on python.org currently doesn't explain this).
>
> It should also explain that the scheme is meant to assure round-trip
> safety and doesn't necessarily work when using transcoding, ie.
> reading using one encoding, writing using another.
Servus,
Walter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASED] Python 3.1 beta 1
On behalf of the Python development team, I'm thrilled to announce the first and only beta release of Python 3.1. Python 3.1 focuses on the stabilization and optimization of features and changes Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. [1] Other features include an ordered dictionary implementation and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/dev/py3k/whatsnew/3.1.html or Misc/NEWS in the Python distribution. Please note that this is a beta release, and as such is not suitable for production environments. We continue to strive for a high degree of quality, but there are still some known problems and the feature sets have not been finalized. This beta is being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 3.1 might impact you. If you find things broken or incorrect, please submit a bug report at http://bugs.python.org For more information and downloadable distributions, see the Python 3.1 website: http://www.python.org/download/releases/3.1/ See PEP 375 for release schedule details: http://www.python.org/dev/peps/pep-0375/ Enjoy, -- Benjamin Benjamin Peterson benjamin at python.org Release Manager (on behalf of the entire python-dev team and 3.1's contributors) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 2:11 PM, David Cournapeau wrote: > But I don't know if that's easy to set up such as both python and > numpy are built from sources. I don't know about the numpy part, but the PyBots project code could be a source of inspiration for the Python part http://code.google.com/p/pybots/source/browse/trunk/master/community.cfg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 8:49 PM, Tarek Ziadé wrote: > > Notice that from the beginning, the unixcompiler class options are > never used if the option has been customized > in distutils.sysconfig and present in the Makefile, so we need to > clean this behavior as well at some point, and document > the customization features. Indeed, I have never bothered much with this part, though. Flags customization with distutils is too awkward to be useful in general for something like numpy IMHO, I just use scons instead when I need fine grained control. > By the way, do you happen to have a buildbot or something that builds numpy ? We have a buildbot: http://buildbot.scipy.org/ But I don't know if that's easy to set up such as both python and numpy are built from sources. > If not it'll be very interesting: I wouldn't mind having one numpy > track running on the Python trunk and receiving > mails if something is broken. Well, I would not mind either :) David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 1:37 PM, David Cournapeau wrote: > On Thu, May 7, 2009 at 7:07 PM, Tarek Ziadé wrote: >> On Thu, May 7, 2009 at 11:50 AM, David Cournapeau wrote: >>> Then, in the customize_compiler function, set archiver to $AR + >>> $ARFLAGS. IOW, just copying the logic used for e.g. ldshared, >>> >>> I can prepare a patch if you want, >> >> I am ok on Distutils side, but I wouldn't mind some help on the >> makefile/configure side > > Ok, I ended up making a patch for everything. I tested it on Linux, > where it fixed the issue while keeping the customization (both AR and > ARFLAGS can be customized through environment variables). > > numpy now builds under python 2.7, > > cheers, > > David > ok thanks David, I'll complete your patch with the test I have written for this issue and commit it so it's included in 2.7/3.1. Notice that from the beginning, the unixcompiler class options are never used if the option has been customized in distutils.sysconfig and present in the Makefile, so we need to clean this behavior as well at some point, and document the customization features. By the way, do you happen to have a buildbot or something that builds numpy ? If not it'll be very interesting: I wouldn't mind having one numpy track running on the Python trunk and receiving mails if something is broken. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Thu, May 7, 2009 at 11:50 AM, David Cournapeau wrote: > Then, in the customize_compiler function, set archiver to $AR + > $ARFLAGS. IOW, just copying the logic used for e.g. ldshared, > > I can prepare a patch if you want, I am ok on Distutils side, but I wouldn't mind some help on the makefile/configure side Even if I could mimic what's in there, I am not confident enough yet. Please do so, by attaching your patch in the issue, Thanks Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help on issue 5941
On Wed, May 6, 2009 at 6:01 PM, Tarek Ziadé wrote: > Hello, > > I need some help on http://bugs.python.org/issue5941 > > The bug is quite simple: the Distutils unixcompiler used to set the > archiver command to "ar -rc". > > For quite a while now, this behavior has changed in order to be able > to customize the compiler behavior from > the environment. That introduced a regression because the mechanism in > Distutils that looks for the > AR variable in the environment also looks into the Makefile of Python. > (in the Makefile then is os.environ) > > And as a matter of fact, AR is set to "ar" in there, so the -cr option > is not set anymore. > > So my question is : should I make a change into the Makefile by adding > for example a variable called AR_OPTIONS > then build the ar command with AR + AR_OPTIONS I think for consistency, it could be named ARFLAGS (this is the name usually taken for configure scripts), and both should be overridable as the other variable in distutils.sysconfig.customize_compiler. Those flags should be used in Makefile.pre as well, instead of the harcoded cr as currently used. Here is what I would try: - check for AR (already done in the configure script AFAICT) - if ARFLAGS is defined in the environment, use those, otherwise set ARFLAGS to cr - use ARFLAGS in the makefile Then, in the customize_compiler function, set archiver to $AR + $ARFLAGS. IOW, just copying the logic used for e.g. ldshared, I can prepare a patch if you want, cheers, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
Antoine Pitrou wrote:
> Martin v. Löwis v.loewis.de> writes:
>> py> b'\xed\xa0\x80'.decode("utf-8","surrogates")
>> '\ud800'
>
> The point is, "surrogates" does not mean anything intuitive for an /error
> handler/. You seem to be the only one who finds this name explicit enough,
> perhaps because you chose it.
> Most other handlers' names have verbs in them ("ignore", "replace",
> "xmlcharrefreplace", etc.).
Correct.
The purpose of an error handler name is to indicate to the user
what it does, hence the use of verbs.
Walter started with "xmlcharrefreplace", ie. no space names, so
"surrogatereplace" would be the logically correct name for the
"replace with lone surrogates" scheme invented by Markus Kuhn.
The error handler for undoing this operation (ie. when converting
a Unicode string to some other encoding) should probably use the
same name based on symmetry and the fact that the escaping
scheme is meant to be used for enabling round-trip safety.
BTW: It would also be appropriate to reference Markus Kuhn in the PEP
as the inventor of the escaping scheme.
Even if only to give the reader an idea of how that scheme works and
why (the PEP on python.org currently doesn't explain this).
It should also explain that the scheme is meant to assure round-trip
safety and doesn't necessarily work when using transcoding, ie.
reading using one encoding, writing using another.
Thanks,
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, May 07 2009)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
2009-06-29: EuroPython 2009, Birmingham, UK52 days to go
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
On approximately 5/6/2009 11:16 PM, came the following characters from the keyboard of Martin v. Löwis: So are you proposing that I should rename the PEP 383 handler to "utf_8b_encoder_invalid_codepoints"? No, he's saying that your algorithm for choosing the PEP 383 handler should have come up with that name, rather than utf8b. But since PEP 383 applies to other codecs besides UTF-8, it should have a different name. And one that is less cumbersome than "utf_8b_encoder_invalid_codepoints" I'm still at a loss what name to give it, though. I understand that I have to rename both error handlers, but I'm uncertain what I should rename them to. So proposals that rename only one of them aren't that helpful. It would be helpful if people would indicate support for Antoine's proposal. Wouldn't renaming the existing "surrogates" handler be an incompatible change, and thus inappropriate? I assume that is the second handler you are referring to? "bytes-as-lone-surrogates" That would be very descriptive of the decode case for PEP 383, but very long. One problem with the word "surrogates" is that anything you add to it makes it too long. "bytes-ls" This is short, but a meaningless as is -- however, adding the understanding via documentation that "ls" means "lone surrogates" would make it meaningful, and mnemonic. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
