[Python-Dev] Decoder functions accept str in py3k
Hello,
I've just noticed that in py3k, the decoding functions in the codecs module
accept str objects as well as bytes:
# import codecs
# c = codecs.getdecoder('utf8')
# c('aa')
('aa', 2)
# c('éé')
('éé', 4)
# c = codecs.getdecoder('latin1')
# c('aa')
('aa', 2)
# c('éé')
('éé', 4)
Is it a bug?
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a few strdup() questions...
On Wed, Jan 7, 2009 at 5:30 AM, Ulrich Eckhardt dooms...@knuut.de wrote: MS Windows CE doesn't provide strdup(), so where should I put it? I guess I should just compile in Python/strdup.c, right? I'm not an expert on Windows CE, but I believe it calls the function _strdup(): http://msdn.microsoft.com/en-us/library/ms861162.aspx -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
Sounds like yet another remnant of the old philosophy, which indeed
supported encode and decode operations on both string types. :-(
On Wed, Jan 7, 2009 at 5:39 AM, Antoine Pitrou solip...@pitrou.net wrote:
Hello,
I've just noticed that in py3k, the decoding functions in the codecs module
accept str objects as well as bytes:
# import codecs
# c = codecs.getdecoder('utf8')
# c('aa')
('aa', 2)
# c('éé')
('éé', 4)
# c = codecs.getdecoder('latin1')
# c('aa')
('aa', 2)
# c('éé')
('Ã(c)Ã(c)', 4)
Is it a bug?
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
Guido van Rossum guido at python.org writes: Sounds like yet another remnant of the old philosophy, which indeed supported encode and decode operations on both string types. How do we go for fixing it? Is it ok to raise a TypeError in 3.0.1? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
That depends a bit on how much code we find that breaks as a result. If you find you have to do a big cleanup in the stdlib after that change, it's likely that 3rd party code could have the same problem, and I'd be reluctant. I'd be okay with adding a warning in that case. OTOH if there's no cleanup to be done I'm fine with just deleting it. A -3 warning should be added to 2.6 about this too IMO. On Wed, Jan 7, 2009 at 7:39 AM, Antoine Pitrou solip...@pitrou.net wrote: Guido van Rossum guido at python.org writes: Sounds like yet another remnant of the old philosophy, which indeed supported encode and decode operations on both string types. How do we go for fixing it? Is it ok to raise a TypeError in 3.0.1? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pybots link obsolete?
Hello, In http://www.python.org/dev/buildbot/, there's a link suggesting to visit the pybots Web site for more information. However, http://www.pybots.org/ just says Nothing here #. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
On Wed, Jan 7, 2009 at 9:46 AM, Guido van Rossum gu...@python.org wrote: A -3 warning should be added to 2.6 about this too IMO. A Py3k warning when attempting to decode a unicode string? Wouldn't that open the door to adding warnings to everywhere a unicode string is used where a byte string is? I thought that unicode and str's compatibility was quite intentionally not being touched until 3.0. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
On 2009-01-07 16:34, Guido van Rossum wrote:
Sounds like yet another remnant of the old philosophy, which indeed
supported encode and decode operations on both string types. :-(
No, that's something I explicitly readded to Python 3k, since the
codecs interface is independent of the input and output types (the
codecs decide which combinations to support).
The bytes and Unicode *methods* do guarantee that you get either
Unicode or bytes as output.
On Wed, Jan 7, 2009 at 5:39 AM, Antoine Pitrou solip...@pitrou.net wrote:
Hello,
I've just noticed that in py3k, the decoding functions in the codecs module
accept str objects as well as bytes:
# import codecs
# c = codecs.getdecoder('utf8')
# c('aa')
('aa', 2)
# c('éé')
('éé', 4)
# c = codecs.getdecoder('latin1')
# c('aa')
('aa', 2)
# c('éé')
('Ã(c)Ã(c)', 4)
Is it a bug?
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jan 07 2009)
Python/Zope Consulting and Support ...http://www.egenix.com/
mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
On Wed, Jan 07, 2009, Antoine Pitrou wrote: Guido van Rossum guido at python.org writes: Sounds like yet another remnant of the old philosophy, which indeed supported encode and decode operations on both string types. How do we go for fixing it? Is it ok to raise a TypeError in 3.0.1? This definitely cannot be changed for 3.0.1 -- there's plenty of time to discuss this for 3.1. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. --Brian W. Kernighan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
M.-A. Lemburg mal at egenix.com writes: No, that's something I explicitly readded to Python 3k, since the codecs interface is independent of the input and output types (the codecs decide which combinations to support). But why would the utf8 decoder accept unicode as input? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
On 2009-01-07 19:32, Antoine Pitrou wrote: M.-A. Lemburg mal at egenix.com writes: No, that's something I explicitly readded to Python 3k, since the codecs interface is independent of the input and output types (the codecs decide which combinations to support). But why would the utf8 decoder accept unicode as input? It shouldn't. Looks like the codecs module codec interfaces were not updated to only accept bytes on decode for the Unicode codecs. BTW: The _codecsmodule.c file is a 4 spaces indent file as well (just like all Unicode support source files). Someone apparently has added tabs when adding support for Py_buffers. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 07 2009) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] a few strdup() questions...
MS Windows CE doesn't provide strdup(), so where should I put it? I guess I should just compile in Python/strdup.c, right? Right. However, where should I declare it? I recommend pyport.h. Also, there is HAVE_STRDUP. I would actually expect that #undef HAVE_STRDUP would do the trick to at least declare this, but it doesn't. I guess that most modern OS have this so this will probably just be bitrot ... right? Wrong, I think. The macro is a side effect of AC_REPLACE_FUNCS, which will a) add strdup.c to the list of files to compile, or b) define HAVE_STRDUP. BTW: there is another implementation (called my_strdup) in Modules/_ctypes/_ctypes_test.c, why not use the one in Python/strdup.c there? I guess that's historical, from the times when ctypes was still a separate package. First difference is that I wouldn't accept NULL as valid input, e.g. the glibc implementation doesn't either and GCC even warns you if you call strdup(NULL). Secondly, I would have used memcpy(), since the length is already known and then potentially quicker. Should I write a patch? Is that really worth it? It works as-is, doesn't it? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
OK, ignore my previous comment. Sounds like the inidividual codecs
need to tighten their type checking though -- perhaps *that* can be
fixed in 3.0.1? I really don't see why any codec used to convert
between text and bytes should support its output type as input.
--Guido
On Wed, Jan 7, 2009 at 10:26 AM, M.-A. Lemburg m...@egenix.com wrote:
On 2009-01-07 16:34, Guido van Rossum wrote:
Sounds like yet another remnant of the old philosophy, which indeed
supported encode and decode operations on both string types. :-(
No, that's something I explicitly readded to Python 3k, since the
codecs interface is independent of the input and output types (the
codecs decide which combinations to support).
The bytes and Unicode *methods* do guarantee that you get either
Unicode or bytes as output.
On Wed, Jan 7, 2009 at 5:39 AM, Antoine Pitrou solip...@pitrou.net wrote:
Hello,
I've just noticed that in py3k, the decoding functions in the codecs module
accept str objects as well as bytes:
# import codecs
# c = codecs.getdecoder('utf8')
# c('aa')
('aa', 2)
# c('éé')
('éé', 4)
# c = codecs.getdecoder('latin1')
# c('aa')
('aa', 2)
# c('éé')
('Ã(c)Ã(c)', 4)
Is it a bug?
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jan 07 2009)
Python/Zope Consulting and Support ...http://www.egenix.com/
mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free !
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Mathematica
Does anyone here have access to Mathematica?
I would like to know what it returns for:
In[1]:= Permutations({a, b, c}, {5})
Knowing this will help resolve a feature request
for itertools.permutations() and friends.
Thanks,
Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mathematica
On Wed, Jan 7, 2009 at 9:48 PM, Raymond Hettinger pyt...@rcn.com wrote:
Does anyone here have access to Mathematica?
I would like to know what it returns for:
In[1]:= Permutations({a, b, c}, {5})
Knowing this will help resolve a feature request
for itertools.permutations() and friends.
I assume you mean with square brackets:
Mathematica 6.0 for Linux x86 (32-bit)
Copyright 1988-2008 Wolfram Research, Inc.
In[1]:= Permutations[{a, b, c}, {5}]
Out[1]= {}
Fredrik
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] error in doc for fcntl module
In the doc page for the fcntl module, the example below is given. This seems like an error, or at least very misleading, as the normal usage is to get the flags (F_GETFL), set or unset the bits you want to change, then set the flags (F_SETFL). A reader might think that the example below merely sets O_NDELAY, but it also stomps all of the other bits to zero. If someone can confirm my thinking, this ought to be changed. import struct, fcntl, os f = open(...) rv = fcntl.fcntl(f, fcntl.F_SETFL, os.O_NDELAY) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
On Wed, Jan 7, 2009 at 10:57, M.-A. Lemburg m...@egenix.com wrote: [SNIP] BTW: The _codecsmodule.c file is a 4 spaces indent file as well (just like all Unicode support source files). Someone apparently has added tabs when adding support for Py_buffers. It looks like this formatting mix-up is just going to get worse for the next few years while the 2.x series is still being worked on. Should we just bite the bullet and start adding modelines for Vim and Emacs to .c/.h files that are written in the old 2.x style? For Vim I can then update the vimrc in Misc/Vim to then have 4-space indent be the default for C files. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
Guido van Rossum wrote: OK, ignore my previous comment. Sounds like the inidividual codecs need to tighten their type checking though -- perhaps *that* can be fixed in 3.0.1? I really don't see why any codec used to convert between text and bytes should support its output type as input. --Guido On Wed, Jan 7, 2009 at 10:26 AM, M.-A. Lemburg m...@egenix.com wrote: On 2009-01-07 16:34, Guido van Rossum wrote: Sounds like yet another remnant of the old philosophy, which indeed supported encode and decode operations on both string types. :-( No, that's something I explicitly readded to Python 3k, since the codecs interface is independent of the input and output types (the codecs decide which combinations to support). My memory is that making decode = bytes - str and encode = str- bytes was considered until it was noticed that there are sensible same-type transforms that fit the encode/decode model and then decided that reusing that model would be better than adding a transcode module/model. The bug of Unicode de/encoders allowing wrong inputs and giving weird outputs confuses people and has come up on c.l.p, so I think fixing it soon would be good. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decoder functions accept str in py3k
On Wed, Jan 7, 2009 at 2:35 PM, Brett Cannon br...@python.org wrote: On Wed, Jan 7, 2009 at 10:57, M.-A. Lemburg m...@egenix.com wrote: [SNIP] BTW: The _codecsmodule.c file is a 4 spaces indent file as well (just like all Unicode support source files). Someone apparently has added tabs when adding support for Py_buffers. It looks like this formatting mix-up is just going to get worse for the next few years while the 2.x series is still being worked on. Should we just bite the bullet and start adding modelines for Vim and Emacs to .c/.h files that are written in the old 2.x style? For Vim I can then update the vimrc in Misc/Vim to then have 4-space indent be the default for C files. Or better yet, really bite the bullet and just reindent everything to spaces. Not every one uses vim or emacs, nor do all tools understand their modelines. FYI, there are options to svn blame and git to skip whitespace-only changes. Just-spent-an-hour-fixing-screwed-up-indents-in-changes-to-Python/*.c-ly, Collin Winter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] error in doc for fcntl module
Well my Linux man page says that the only flags supported are O_APPEND, O_ASYNC, O_DIRECT, O_NOATIME, and O_NONBLOCK; and all of those are typically off -- so I'm not sure that it's a mistake or need correcting. These APIs should only be used by people who know what they're doing anyways; the examples are meant to briefly show the call format. On Wed, Jan 7, 2009 at 1:31 PM, Mike Coleman tutu...@gmail.com wrote: In the doc page for the fcntl module, the example below is given. This seems like an error, or at least very misleading, as the normal usage is to get the flags (F_GETFL), set or unset the bits you want to change, then set the flags (F_SETFL). A reader might think that the example below merely sets O_NDELAY, but it also stomps all of the other bits to zero. If someone can confirm my thinking, this ought to be changed. import struct, fcntl, os f = open(...) rv = fcntl.fcntl(f, fcntl.F_SETFL, os.O_NDELAY) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] What's New in Python 2.6: no string exceptions
After reading What's New in Python 2.6 and then upgrading, I quickly noticed an omission: string exceptions are no longer supported and raise a TypeError. It seems like this should be mentioned in the Porting to Python 2.6 section at minimum, or perhaps more prominently since this change will break code in many small projects (e.g., code from Python 2.5's tutorial). Were any other previously-deprecated features removed for 2.6? Also, it might be nice if whatever tool tests the code in the tutorial would treat Deprecation warnings as hard errors, so new users don't learn features slated for possible removal. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Can someone explain the fast_block_end manipulation?
I don't see anyplace obvious where a value resembling a jump offset or jump target was pushed onto the stack. Duh. Found it about one minute after sending... CONTINUE_LOOP. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Py3k] curses module and libncursesw library
-On [20090108 02:23], Victor Stinner (victor.stin...@haypocalc.com) wrote: It looks like libncursesw is available on Linux, *BSD, Mac OS X. On FreeBSD I know it is for 7.x, but I am not sure about 6.x. -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Earth to earth, ashes to ashes, dust to dust... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
