Re: [Python-Dev] accept string in a2b and base64?
It seems to me that part of the point of the byte/string split (and the lack of automatic coercion) is to make the programmer be explicit about converting between unicode and bytes. Having these functions, which convert between binary formats (ASCII-only representations of binary data and back) accept unicode strings is reintroducing automatic coercions, and I think it will lead to the same kind of bugs that automatic string coercions yielded in Python2: a program works fine until the input turns out to have non-ASCII data in it, and then it blows up with an unexpected UnicodeError. I agree with the change in principle, but I also agree in the choice of error with you: py binascii.a2b_hex(MURRAY) Traceback (most recent call last): File stdin, line 1, in module binascii.Error: Non-hexadecimal digit found py binascii.a2b_hex(VLÖWIS) Traceback (most recent call last): File stdin, line 1, in module ValueError: string argument should contain only ASCII characters I think it should give binascii.Error in both cases: Ö is as much a non-hexadecimal digit as M. With that changed, I'd have no issues with the patch: these functions are already fairly strict in their input, whether it's bytes or Unicode. So the chances that non-ASCII characters get it to fall over in a way that never causes problems in pure-ASCII communities are very low. If most people agree with Antoine I won't fight it, but it seems to me that accepting unicode in the binascii and base64 APIs is a bad idea. No - it's only the choice of error that is a bad idea. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hash randomization in 3.3
Brett Cannon writes: I think that's inviting trouble if you can provide the seed. It leads to a false sense of security I thought the point of providing the seed was for reproducability of tests and the like? As for false sense, can't we document this and chalk up hubristic behavior to consenting adults? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows build - fixing compile warnings before VS2010
On Tue, 21 Feb 2012 21:32:23 -0600 Brian Curtin br...@python.org wrote: While some effort has gone on to get the 32-bit build to compile without warnings (thanks for that!), 64-bit still has numerous warnings. Before I push forward on more of the VS2010 port, I'd like to have a clean 2008 build all around so we can more easily track what may have changed. +1. Of course, it doesn't help that Microsoft implements POSIX APIs incorrectly (for example, Microsoft's read() take the length as an int, not a size_t). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Optimize str%arg for number formats: %i, %d, %u, %x, %p
On Wed, 22 Feb 2012 13:58:45 +0100 victor.stinner python-check...@python.org wrote: +/* Copy a ASCII or latin1 char* string into a Python Unicode string. + Return the length of the input string. + + WARNING: Don't copy the terminating null character and don't check the + maximum character (may write a latin1 character in an ASCII string). */ If this is a description of what the function does, it should say doesn't, not don't. Right now this comment is ambiguous (is it an order given to the reader?). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows build - fixing compile warnings before VS2010
On Tue, Feb 21, 2012 at 23:45, mar...@v.loewis.de wrote: Zitat von Brian Curtin br...@python.org: While some effort has gone on to get the 32-bit build to compile without warnings (thanks for that!), 64-bit still has numerous warnings. Before I push forward on more of the VS2010 port, I'd like to have a clean 2008 build all around so we can more easily track what may have changed. Does that *really* have to be a prerequisite for porting to VS 2010? If yes, then my hopes that we can move to VS 2010 before 3.3 are falling... Is it a prerequisite? No. I guess with this question all I'm asking is Can I fix a lot of these warnings without someone wanting to undo them for the sake of cleaner merges or neat hg history? I'd prefer not to take 315 warnings into a compiler change, come out with 550, and not know what potentially went wrong. In a previous company, we changed from 2008 to 2010 by upping the warning level, fixing all warnings, then enabling warnings-as-errors (I'll address this later) - the port to 2010 went nicely and we experienced a very smooth transition. Much more smoothly than 2005 to 2008. I just cut out around 100 warnings last night in 45 minutes, so I don't plan on having this take several months or anything. If I get stuck, I'll just give it up. While I have your attention, I'd like to throw two other things out there to follow up the above effort: 1. Is anyone opposed to moving up to Level 4 warnings? Not sure what this means. What kind of warnings would this get us? MS says This option should be used only to provide lint level warnings and is not recommended as your usual warning level setting. Usually, following MS recommendations is a good thing to do on Windows. But then, the documentation goes on saying For a new project, it may be best to use /W4 in all compilations. This will ensure the fewest possible hard-to-find code defects. The last sentence (but applied to old projects) says it all. Like I mentioned above, my last company jacked everything up to the highest levels and stuck with it, and I think we wrote nicer code. That's really all I can say. No metrics, no strong support, no debate. You could just say no and I'll probably accept it. ...take a deep breath... 2. Is anyone opposed to enabling warnings as errors? The immediate consequence would be that the Windows buildbots break when somebody makes a checkin on Unix, and they cannot easily figure out how to rewrite the code to make the compiler happy. So I guess I'm -1. I didn't think about that, so yeah, I'm probably -1 here as well. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows build - fixing compile warnings before VS2010
On 22/02/2012 3:32am, Brian Curtin wrote:
1. Is anyone opposed to moving up to Level 4 warnings?
At that level I think it complains about common things like the do
{...} while (0) idiom, and the unreferenced self parameter of builtin
functions.
Presumably you would have to disable those specific warnings and any
other overly annoying ones?
sbt
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows build - fixing compile warnings before VS2010
On Wed, Feb 22, 2012 at 10:04, shibturn shibt...@gmail.com wrote:
On 22/02/2012 3:32am, Brian Curtin wrote:
1. Is anyone opposed to moving up to Level 4 warnings?
At that level I think it complains about common things like the do {...}
while (0) idiom, and the unreferenced self parameter of builtin functions.
Presumably you would have to disable those specific warnings and any other
overly annoying ones?
What we did was fix what was reasonable, then disable warnings which
were unreasonable. If that's reasonable, that's how I would do it.
(just to say it one more time: reasonable)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
On Wed, Feb 22, 2012 at 04:24:38AM +0200, Eli Bendersky wrote: Andrew, could you elaborate on your use case? Are you using cElementTree to do the parsing, or ElementTree (the Python implementation). Can you show a short code sample? I'm mostly using ElementTree because several classes/functions that I need are not in cElementTree or behave differently. Specifically, the program loads TreeBuilder, XMLParser, iterparse from ElementTree; the only class from cElementTree that works is Element. A shortened version of the program is available here: http://aml.cs.byu.edu/~amcnabb/gutenberg-short.py By the way, this code is just one example of how one might rely on the documented extensibility of ElementTree. There are probably many other examples out there that look nothing like mine. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] The ultimate question of life, the universe, and everything
What is the hash of ePjNTUhitHkL? Regards, Martin P.S. It took me roughly 86h to compute 150 strings colliding for the 64-bit hash function. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hash randomization in 3.3
On Feb 22, 2012, at 09:04 PM, Stephen J. Turnbull wrote: Brett Cannon writes: I think that's inviting trouble if you can provide the seed. It leads to a false sense of security I thought the point of providing the seed was for reproducability of tests and the like? As for false sense, can't we document this and chalk up hubristic behavior to consenting adults? +1 -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issue 13703 is closed for the Python 2.6 branch
Two more small details to address, and then I think we're ready to start
creating release candidates.
- sys.flags.hash_randomization
In the tracker issue, I had previously stated a preference that this flag
only reflect the state of the -R command line option, not the
$PYTHONHASHSEED environment variable. Well, that's not the way other
options/envars such as -O/$PYTHONOPTIMIZE work. sys.flags.optimize gets
set if either of those two things set it, so sys.flags.hash_randomization
needs to follow that convention. Thus no change is necessary here.
- sys.hash_seed
In the same tracker issue, I expressed my opinion that the hash seed should
be exposed in sys.hash_seed for reproducibility. There's a complication
that Victor first mentioned in IRC, but I didn't quite understand the
implications of at first. When PYTHONHASHSEED=random is set, there *is no*
hash seed. We pull random data straight out of urandom and use that
directly as the secret, so there's nothing to expose in sys.hash_seed.
In that case, sys.hash_seed is pretty much redundant, since Python code could
just check getenv('PYTHONHASHSEED') and be done with it. I don't think
there's anything useful to expose to Python or communicated between Python
executables when truly random hash data is used.
Thus, unless there are objections, I consider the current state of the Python
2.6 branch to be finished wrt issue 13703.
Cheers,
-Barry
signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hash randomization in 3.3
On Wed, 22 Feb 2012 12:59:33 -0500 Barry Warsaw ba...@python.org wrote: On Feb 22, 2012, at 09:04 PM, Stephen J. Turnbull wrote: Brett Cannon writes: I think that's inviting trouble if you can provide the seed. It leads to a false sense of security I thought the point of providing the seed was for reproducability of tests and the like? As for false sense, can't we document this and chalk up hubristic behavior to consenting adults? +1 How is it a false sense of security at all? It's the same as setting a private secret for e.g. session cookies in Web applications. As long as you don't leak the seed, it's (should be) secure. (the only hypothetical issue being with Victor's choice of an LCG pseudo-random generator to generate the secret from the seed) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The ultimate question of life, the universe, and everything
On Wed, Feb 22, 2012 at 9:59 AM, mar...@v.loewis.de wrote: What is the hash of ePjNTUhitHkL? Regards, Martin P.S. It took me roughly 86h to compute 150 strings colliding for the 64-bit hash function. You should have used pypy, should have been faster. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The ultimate question of life, the universe, and everything
Am 22.02.2012 19:46, schrieb Maciej Fijalkowski: On Wed, Feb 22, 2012 at 9:59 AM, mar...@v.loewis.de wrote: What is the hash of ePjNTUhitHkL? Regards, Martin P.S. It took me roughly 86h to compute 150 strings colliding for the 64-bit hash function. You should have used pypy, should have been faster. It was actually a C program - I doubt PyPy would have been faster. Perhaps my algorithm wasn't good enough if you think this is slow. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The ultimate question of life, the universe, and everything
On Wed, Feb 22, 2012 at 11:55 AM, Martin v. Löwis mar...@v.loewis.de wrote: Am 22.02.2012 19:46, schrieb Maciej Fijalkowski: On Wed, Feb 22, 2012 at 9:59 AM, mar...@v.loewis.de wrote: What is the hash of ePjNTUhitHkL? Regards, Martin P.S. It took me roughly 86h to compute 150 strings colliding for the 64-bit hash function. You should have used pypy, should have been faster. It was actually a C program - I doubt PyPy would have been faster. Perhaps my algorithm wasn't good enough if you think this is slow. Regards, Martin That was entirely a joke, sorry if a bad one :) I seriously doubt it'll be faster. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows build - fixing compile warnings before VS2010
I just cut out around 100 warnings last night in 45 minutes, so I don't plan on having this take several months or anything. If I get stuck, I'll just give it up. Would you mind posting a batch of these to the tracker? I'd like to review them, just to be sure we have the same understanding. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hash randomization in 3.3
On 2/22/2012 1:57 AM, Nick Coghlan wrote: In the tracker, someone proposed that the option is necessary to synchronize the seed across processes in a cluster. I'm sure people will use it for that if they can. Yeah, that use case sounds reasonable, too. Another example is that, even within a machine, if two processes are using shared memory rather than serialised IPC, synchronising the hashes may be necessary. The key point is that there *are* valid use cases for forcing a particular seed, so we shouldn't take that ability away. When we document the option to set the seed, we could mention that synchronization of processes that share data is the main intended use. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 13703 is closed for the Python 2.6 branch
On Wed, Feb 22, 2012 at 10:14 AM, Barry Warsaw ba...@python.org wrote:
Two more small details to address, and then I think we're ready to start
creating release candidates.
- sys.flags.hash_randomization
In the tracker issue, I had previously stated a preference that this flag
only reflect the state of the -R command line option, not the
$PYTHONHASHSEED environment variable. Well, that's not the way other
options/envars such as -O/$PYTHONOPTIMIZE work. sys.flags.optimize gets
set if either of those two things set it, so sys.flags.hash_randomization
needs to follow that convention. Thus no change is necessary here.
- sys.hash_seed
In the same tracker issue, I expressed my opinion that the hash seed should
be exposed in sys.hash_seed for reproducibility. There's a complication
that Victor first mentioned in IRC, but I didn't quite understand the
implications of at first. When PYTHONHASHSEED=random is set, there *is no*
hash seed. We pull random data straight out of urandom and use that
directly as the secret, so there's nothing to expose in sys.hash_seed.
In that case, sys.hash_seed is pretty much redundant, since Python code could
just check getenv('PYTHONHASHSEED') and be done with it. I don't think
there's anything useful to expose to Python or communicated between Python
executables when truly random hash data is used.
Thus, unless there are objections, I consider the current state of the Python
2.6 branch to be finished wrt issue 13703.
+10
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
Le 11/02/2012 12:00, Eli Bendersky a écrit : Well, I think the situation is pretty good now. If one goes to python.org and is interested in contributing, clicking on the Core Development link is a sensible step, right? Maybe, depending on your knowledge of jargon. How about rewording that link to “Contributing”? Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] requirements for moving __import__ over to importlib?
Hi Brett, I think this message went unanswered, so here’s a late reply: Le 07/02/2012 23:21, Brett Cannon a écrit : On Tue, Feb 7, 2012 at 15:28, Dirkjan Ochtman dirk...@ochtman.nl wrote: [...] Anyway, I think there was enough of a python3 port for Mercurial (from various GSoC students) that you can probably run some of the very simple commands (like hg parents or hg id), which should be enough for your purposes, right? Possibly. Where is the code? # get Mercurial from a repo or tarball hg clone http://selenic.com/repo/hg/ cd hg # convert files in place (don’t commit after this :) python3.2 contrib/setup3k.py # the makefile is not py3k-aware, need to run manually # the current stable head fails with a TypeError for me PYTHONPATH=. python3.2 build/scripts-3.2 Cheers ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
On Wed, Feb 22, 2012 at 18:21, Éric Araujo mer...@netwok.org wrote: Le 11/02/2012 12:00, Eli Bendersky a écrit : Well, I think the situation is pretty good now. If one goes to python.org and is interested in contributing, clicking on the Core Development link is a sensible step, right? Maybe, depending on your knowledge of jargon. How about rewording that link to “Contributing”? If you want to contribute to development, I think you'll know that a link about development is relevant. If you want to contribute money, a contribute link about development means you have to try again to give away your money. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hash randomization in 3.3
Antoine Pitrou writes: How is it a false sense of security at all? It's the same as setting a private secret for e.g. session cookies in Web applications. As long as you don't leak the seed, it's (should be) secure. That's true. The problem is, the precondition that you won't leak the seed is all too often false. If a user takes advantage of the ability to set the seed, she can leak it, or a coworker (or a virus) can steal it from her source or keystroke logging, etc. And it's not the same, at least not for a highly secure application. In high-quality security, session keys are generated for each session (and changed frequently); the user doesn't know them (of course, he can always find out if he really wants to know, and sometimes that's necessary -- Hello, Debian OpenSSH maintainer!), and so can't leak them. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
Brian Curtin writes: If you want to contribute to development, I think you'll know that a link about development is relevant. For values of you in experienced programmers, yes. But translators and tech writers don't consider what they do to be development. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
On Thu, Feb 23, 2012 at 01:15, Stephen J. Turnbull step...@xemacs.org wrote: Brian Curtin writes: If you want to contribute to development, I think you'll know that a link about development is relevant. For values of you in experienced programmers, yes. But translators and tech writers don't consider what they do to be development. I don't know what this is saying, but I'll guess it's some suggestion that we should still name the link Contributing. Keep in mind that the current Core Development link on the front page goes directly to http://docs.python.org/devguide/ -- getting this page in people's hands earlier is a Good Thing. However, this is not a correct link from something named Contributing. It would have to say Contributing Code, but then it leaves out docs and translations and our resident spelling bee contestants. Paint the bike shed any way you want except the plain Contributing color, please. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
