Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Antoine Pitrou
On Thu, 16 May 2013 11:42:30 -0400 Barry Warsaw ba...@python.org wrote: On May 16, 2013, at 08:33 AM, Nick Coghlan wrote: Personally, I would be suspicious of developmental web services doing auto-reloading while an installer is recompiling the world. I don't have enough context to be sure

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Armin Rigo
Hi all, How about using the shared-or-exclusive advisory file locks (with flock() or fcntl())? It may only work on Posix though. A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

[Python-Dev] HAVE_FSTAT?

2013-05-17 Thread Antoine Pitrou
Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks unconditional? It would avoid having to maintain

Re: [Python-Dev] HAVE_FSTAT?

2013-05-17 Thread Benjamin Peterson
2013/5/17 Antoine Pitrou solip...@pitrou.net: Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks

Re: [Python-Dev] HAVE_FSTAT?

2013-05-17 Thread Skip Montanaro
Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif Are there other guards for similarly common libc functions? If so, perhaps each one should be removed in a series of change sets, one per guard. Skip ___ Python-Dev mailing

Re: [Python-Dev] HAVE_FSTAT?

2013-05-17 Thread Antoine Pitrou
On Fri, 17 May 2013 09:15:29 -0500 Skip Montanaro s...@pobox.com wrote: Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif Are there other guards for similarly common libc functions? I don't think so. Someone should take a look though :-) Regards Antoine.

[Python-Dev] Summary of Python tracker Issues

2013-05-17 Thread Python tracker
ACTIVITY SUMMARY (2013-05-10 - 2013-05-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open3966 ( +3) closed 25805 (+47) total 29771 (+50) Open issues

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
On May 16, 2013, at 02:19 PM, Guido van Rossum wrote: Now consider the following scenario. It involves *three* processes. - Two unrelated processes both start and want to import the same module. - They both see the .pyc file is missing/corrupt and decide to write it. - The first process

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On May 16, 2013, at 11:48 PM, Tres Seaver wrote: I can confirm at least that I have seen this problem within the last two weeks on Ubuntu boxes unrelated to the thw Debian / Ubuntu build infrastruction. Hi Tres. If you see this happen, *please*

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
On May 17, 2013, at 12:10 AM, Thomas Wouters wrote: The 'unknown type codes' can also be explained if the two processes writing to the .pyc files are *different Python versions*. As you may recall, at Google we used to use modified Python interpreters that used '.pyc-2.2', '.pyc-2.4', etc, for

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
On May 16, 2013, at 05:30 PM, Brett Cannon wrote: Just so people know, this is how we used to do it. In importlib we write the entire file to a temp file and then to an atomic rename. Yep, and I suspect that our fix, even if we don't completely identify the root cause, will be to change

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that asking for reports on other systems is part of the reason you posted, but I don't remember seeing any others yet. Right. :) It's harder to dig out

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread David Malcolm
On Fri, 2013-05-17 at 12:42 -0400, Barry Warsaw wrote: On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that asking for reports on other systems is part of the reason you posted, but I don't remember

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
On May 17, 2013, at 01:19 PM, David Malcolm wrote: Fedora/RH pregenerate the .pyc files during rpm creation, and they exist as part of the rpm payload. Good to know, thanks. Do you use `$PYTHON -m py_compile` to generate the pyc files at build time? -Barry signature.asc Description: PGP

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/17/2013 12:26 PM, Barry Warsaw wrote: On May 16, 2013, at 11:48 PM, Tres Seaver wrote: I can confirm at least that I have seen this problem within the last two weeks on Ubuntu boxes unrelated to the thw Debian / Ubuntu build

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Terry Jan Reedy
On 5/17/2013 12:42 PM, Barry Warsaw wrote: On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: Do failures only occur during compileall process? (or whatever substitute you use). No, they are all post-installation failures in unrelated packages that try to import pure-Python modules.

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Barry Warsaw
On May 17, 2013, at 03:02 PM, Terry Jan Reedy wrote: What I mean is, is the corruption (not the detection of corruption) only happening during mass compilation of the stdlib? When user imports a single non-stdlib file he has written the first time, does that ever get corrupted. It's not limited

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread David Malcolm
On Fri, 2013-05-17 at 14:23 -0400, Barry Warsaw wrote: On May 17, 2013, at 01:19 PM, David Malcolm wrote: Fedora/RH pregenerate the .pyc files during rpm creation, and they exist as part of the rpm payload. Good to know, thanks. Do you use `$PYTHON -m py_compile` to generate the pyc

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Matthias Klose
Am 15.05.2013 22:58, schrieb Barry Warsaw: I am looking into a particularly vexing Python problem on Ubuntu that manifests in several different ways. I think the problem is the same one described in http://bugs.python.org/issue13146 and I sent a message on the subject to the ubuntu-devel

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-17 Thread Antoine Pitrou
On Thu, 16 May 2013 13:24:36 +0200 Charles-François Natali cf.nat...@gmail.com wrote: 2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2.

Re: [Python-Dev] Mysterious Python pyc file corruption problems

2013-05-17 Thread Nick Coghlan
On Sat, May 18, 2013 at 3:19 AM, David Malcolm dmalc...@redhat.com wrote: On Fri, 2013-05-17 at 12:42 -0400, Barry Warsaw wrote: On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote: If the corruption only happens on Ubuntu, that would constitute 'rhyme' ;-). I realize that asking for reports

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-17 Thread Nick Coghlan
On Sat, May 18, 2013 at 7:59 AM, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 16 May 2013 13:24:36 +0200 Charles-François Natali cf.nat...@gmail.com wrote: 2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-17 Thread Georg Brandl
Am 18.05.2013 05:21, schrieb Nick Coghlan: On Sat, May 18, 2013 at 7:59 AM, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 16 May 2013 13:24:36 +0200 Charles-François Natali cf.nat...@gmail.com wrote: 2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl