[Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Steve Holden
I hope this is an appropriate dev topic. It seems to me that the unicode discussions of recent days are well highlighted by difficulties I am having using the mailbox module (hardly surprising given the difficulties of handling email generally) even though it passes its tests. I can't find

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Miki Tebeka
Hello Steve, Can someone who is set up to do easily just do a timing of test_mailbox under 2.6 and 3.2, to verify they see the same disparity as me? The test takes about twice as long under 3.1 here On Ubuntu timing was: Python 2.6.5: 23.8sec Python 2.7rc2: 32.7sec Python 3.1.2: 32.3sec

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Senthil Kumaran
On Tue, Jun 29, 2010 at 09:56:11AM -0400, Steve Holden wrote: Can someone who is set up to do easily just do a timing of test_mailbox under 2.6 and 3.2, to verify they see the same disparity as me? The test Actually, No. Python 2.7b2+ (trunk:81685M, Jun 4 2010, 21:52:06) Ran 274 tests in

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Nick Coghlan
Command line: ./python -m test.regrtest -v test_mailbox trunk: Ran 274 tests in 25.239s py3k: Ran 268 tests in 26.263s So I don't see any substantial difference on a Kubuntu 10.04 box (both builds are recent'ish, but not completely up to date). However, the underlying IO access is significantly

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Steve Holden
Nick Coghlan wrote: Command line: ./python -m test.regrtest -v test_mailbox trunk: Ran 274 tests in 25.239s py3k: Ran 268 tests in 26.263s So I don't see any substantial difference on a Kubuntu 10.04 box (both builds are recent'ish, but not completely up to date). However, the

Re: [Python-Dev] what environment variable should contain compiler warning suppression flags?

2010-06-29 Thread Barry Warsaw
On Jun 28, 2010, at 05:28 PM, M.-A. Lemburg wrote: How many Python users will compile Python in debug mode ? How many Python users compile Python at all? :) The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Tim Golden
On 29/06/2010 15:26, Steve Holden wrote: Nick Coghlan wrote: Command line: ./python -m test.regrtest -v test_mailbox trunk: Ran 274 tests in 25.239s py3k: Ran 268 tests in 26.263s So I don't see any substantial difference on a Kubuntu 10.04 box (both builds are recent'ish, but not completely

Re: [Python-Dev] what environment variable should contain compiler warning suppression flags?

2010-06-29 Thread Barry Warsaw
On Jun 28, 2010, at 06:03 PM, M.-A. Lemburg wrote: OPT already uses -O0 if --with-pydebug is used and the compiler supports -g. Since OPT gets added after CFLAGS, the override already happens... So nobody's proposing to drop that? Good! Ignore my last message then. :) -Barry signature.asc

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Tim Golden
On 29/06/2010 15:51, Tim Golden wrote: On 29/06/2010 15:26, Steve Holden wrote: Nick Coghlan wrote: Command line: ./python -m test.regrtest -v test_mailbox trunk: Ran 274 tests in 25.239s py3k: Ran 268 tests in 26.263s So I don't see any substantial difference on a Kubuntu 10.04 box (both

Re: [Python-Dev] Pickle security and remote logging

2010-06-29 Thread Vinay Sajip
anatoly techtonik techtonik at gmail.com writes: insecure. SocketHandler and DatagramHandler docs should at least contain a warning about danger of exposing unpickling interfaces to insecure networks. I've updated the documentation of SocketHandler.makePickle to mention security concerns, and

Re: [Python-Dev] what environment variable should contain compiler warning suppression flags?

2010-06-29 Thread Steve Holden
Barry Warsaw wrote: On Jun 28, 2010, at 05:28 PM, M.-A. Lemburg wrote: How many Python users will compile Python in debug mode ? How many Python users compile Python at all? :) The point is that the default build of Python should use the correct production settings for the C compiler

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Antoine Pitrou
On Tue, 29 Jun 2010 11:40:50 -0400 Steve Holden st...@holdenweb.com wrote: Sure. I attach the outputs of both files, as well as the program and the data. With profiling (python -m cProfile test3.py) the run took less than a third of a second under 2.5, and 168 seconds under 3.1. I'd say that

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread A.M. Kuchling
On Tue, Jun 29, 2010 at 07:56:22AM -0700, Guido van Rossum wrote: Since you have such a great reproducible test case, could you point the profiler at it? (Perhaps on a reduced dataset... The profiler multiples your run time by some number between 2 and 10 IIRC.) Let me underline Guido's

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread A.M. Kuchling
On Tue, Jun 29, 2010 at 11:40:50AM -0400, Steve Holden wrote: I will leave the profiler output to speak for itself, since I can find nothing much to say about it except that there's a hell of a lot of decoding going on inside mailbox.iterkeys(). The problem is actually in _generate_toc(),

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread R. David Murray
On Tue, 29 Jun 2010 18:34:22 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 29 Jun 2010 11:40:50 -0400 Steve Holden st...@holdenweb.com wrote: Sure. I attach the outputs of both files, as well as the program and the data. With profiling (python -m cProfile test3.py) the run took

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Antoine Pitrou
On Tue, 29 Jun 2010 12:52:28 -0400 A.M. Kuchling a...@amk.ca wrote: But should mailboxes really be opened in a UTF-8 encoding, or should they be treated as 7-bit text? I'll have to think about this. I don't see how you can assume UTF-8 for mailbox files, given that each message will have its

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Steve Holden
A.M. Kuchling wrote: On Tue, Jun 29, 2010 at 11:40:50AM -0400, Steve Holden wrote: I will leave the profiler output to speak for itself, since I can find nothing much to say about it except that there's a hell of a lot of decoding going on inside mailbox.iterkeys(). The problem is actually

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Guido van Rossum
It should probably be opened in binary mode. Binary files do have a .readline() method (returning a bytes object), and bytes objects have a .startswith() method. The tell positions computed this way are even compatible with those used by the text file. So you could do it this way: - open binary

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Steve Holden
Guido van Rossum wrote: It should probably be opened in binary mode. Binary files do have a .readline() method (returning a bytes object), and bytes objects have a .startswith() method. The tell positions computed this way are even compatible with those used by the text file. So you could do

Re: [Python-Dev] Pickle security and remote logging

2010-06-29 Thread anatoly techtonik
On Tue, Jun 29, 2010 at 6:15 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: I've updated the documentation of SocketHandler.makePickle to mention security concerns, and that the method can be overridden to use a more secure implementation (e.g. HMAC-signed pickles). Thanks. But I doubt HMAC

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread R. David Murray
On Tue, 29 Jun 2010 13:54:09 -0400, Steve Holden st...@holdenweb.com wrote: A.M. Kuchling wrote: But should mailboxes really be opened in a UTF-8 encoding, or should they be treated as 7-bit text? I'll have to think about this. Neither! You can't open them as 7-bit text, because

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread R. David Murray
On Tue, 29 Jun 2010 17:02:14 -0400, Steve Holden st...@holdenweb.com wrote: Guido van Rossum wrote: - wrap the binary stream in a text stream wrap how? The ultimate destiny of the text is twofold: I would imagine Guido is talking about an io.TextIOWrapper...in other words, take the binary

Re: [Python-Dev] Mailbox module - timings and functionality changes

2010-06-29 Thread Steve Holden
R. David Murray wrote: On Tue, 29 Jun 2010 13:54:09 -0400, Steve Holden st...@holdenweb.com wrote: A.M. Kuchling wrote: But should mailboxes really be opened in a UTF-8 encoding, or should they be treated as 7-bit text? I'll have to think about this. Neither! You can't open them as 7-bit

[Python-Dev] OS X buildbots: why am I skipping these tests?

2010-06-29 Thread Bill Janssen
My Leopard and Tiger PPC buildbots are momentarily green! But I'm looking into why I'm skipping some tests. My buildbots are up-to-date OS-wise and very vanilla, with the latest applicable Xcode. 4 skips unexpected on darwin: test_gdb test_ioctl test_readline test_ttk_guionly Three of

Re: [Python-Dev] what environment variable should contain compiler warning suppression flags?

2010-06-29 Thread Stephen J. Turnbull
Steve Holden writes: I agree - trying to step through -O2 optimized code isn't going to help debug your code, it's going to help you debug the optimizer. That's a very rare use case. Not really. I don't have a lot of practice in debugging at that level, so take it with a grain of salt,

Re: [Python-Dev] OS X buildbots: why am I skipping these tests?

2010-06-29 Thread Guido van Rossum
On Tue, Jun 29, 2010 at 7:55 PM, Bill Janssen jans...@parc.com wrote: My Leopard and Tiger PPC buildbots are momentarily green!  But I'm looking into why I'm skipping some tests.  My buildbots are up-to-date OS-wise and very vanilla, with the latest applicable Xcode. 4 skips unexpected on

Re: [Python-Dev] OS X buildbots: why am I skipping these tests?

2010-06-29 Thread Martin v. Löwis
Seems to work fine. So this I don't understand. Any ideas, anyone? Didn't we discuss this before? The buildbot slave has no controlling terminal anymore, hence it cannot open /dev/tty. If you are curious, just patch your checkout to output the exact errno (e.g. to stdout), and trigger a build

[Python-Dev] Taking over the Mercurial Migration

2010-06-29 Thread Martin v. Löwis
It seems that both Dirkjan and Brett are very caught up with real life for the coming months. So I suggest that some other committer who favors the Mercurial transition steps forward and takes over this project. If nobody volunteers, I propose that we release 3.2 from Subversion, and reconsider