On Sat, May 18, 2013 at 3:19 AM, David Malcolm wrote:
> On Fri, 2013-05-17 at 12:42 -0400, Barry Warsaw wrote:
>> On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote:
>>
>> >If the corruption only happens on Ubuntu, that would constitute 'rhyme'
>> >;-). I realize that asking for reports on other
Am 15.05.2013 22:58, schrieb Barry Warsaw:
> I am looking into a particularly vexing Python problem on Ubuntu that
> manifests in several different ways. I think the problem is the same one
> described in http://bugs.python.org/issue13146 and I sent a message on the
> subject to the ubuntu-deve
On Fri, 2013-05-17 at 14:23 -0400, Barry Warsaw wrote:
> On May 17, 2013, at 01:19 PM, David Malcolm wrote:
>
> >Fedora/RH pregenerate the .pyc files during rpm creation, and they exist
> >as part of the rpm payload.
>
> Good to know, thanks. Do you use `$PYTHON -m py_compile` to generate the py
On May 17, 2013, at 03:02 PM, Terry Jan Reedy wrote:
>What I mean is, is the corruption (not the detection of corruption) only
>happening during mass compilation of the stdlib? When user imports a single
>non-stdlib file he has written the first time, does that ever get corrupted.
It's not limite
On 5/17/2013 12:42 PM, Barry Warsaw wrote:
On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote:
Do failures only occur during compileall process? (or whatever substitute you
use).
No, they are all post-installation failures in unrelated packages that try to
import pure-Python modules.
What
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/17/2013 12:26 PM, Barry Warsaw wrote:
> On May 16, 2013, at 11:48 PM, Tres Seaver wrote:
>
>> I can confirm at least that I have seen this problem within the last
>> two weeks on Ubuntu boxes unrelated to the thw Debian / Ubuntu
>> build infrast
On May 17, 2013, at 01:19 PM, David Malcolm wrote:
>Fedora/RH pregenerate the .pyc files during rpm creation, and they exist
>as part of the rpm payload.
Good to know, thanks. Do you use `$PYTHON -m py_compile` to generate the pyc
files at build time?
-Barry
signature.asc
Description: PGP sig
On Fri, 2013-05-17 at 12:42 -0400, Barry Warsaw wrote:
> On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote:
>
> >If the corruption only happens on Ubuntu, that would constitute 'rhyme'
> >;-). I realize that asking for reports on other systems is part of the reason
> >you posted, but I don't rem
On May 16, 2013, at 04:52 PM, Terry Jan Reedy wrote:
>If the corruption only happens on Ubuntu, that would constitute 'rhyme'
>;-). I realize that asking for reports on other systems is part of the reason
>you posted, but I don't remember seeing any others yet.
Right. :) It's harder to dig out s
On May 16, 2013, at 05:30 PM, Brett Cannon wrote:
>Just so people know, this is how we used to do it. In importlib we
>write the entire file to a temp file and then to an atomic rename.
Yep, and I suspect that our fix, even if we don't completely identify the root
cause, will be to change py_comp
On May 17, 2013, at 12:10 AM, Thomas Wouters wrote:
>The 'unknown type codes' can also be explained if the two processes writing
>to the .pyc files are *different Python versions*. As you may recall, at
>Google we used to use modified Python interpreters that used '.pyc-2.2',
>'.pyc-2.4', etc, for
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On May 16, 2013, at 11:48 PM, Tres Seaver wrote:
>I can confirm at least that I have seen this problem within the last two
>weeks on Ubuntu boxes unrelated to the thw Debian / Ubuntu build
>infrastruction.
Hi Tres. If you see this happen, *please*
On May 16, 2013, at 02:19 PM, Guido van Rossum wrote:
>Now consider the following scenario. It involves *three* processes.
>
>- Two unrelated processes both start and want to import the same module.
>- They both see the .pyc file is missing/corrupt and decide to write it.
>- The first process fini
Hi all,
How about using the shared-or-exclusive advisory file locks (with
flock() or fcntl())? It may only work on Posix though.
A bientôt,
Armin.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Un
On Thu, 16 May 2013 11:42:30 -0400
Barry Warsaw wrote:
> On May 16, 2013, at 08:33 AM, Nick Coghlan wrote:
>
> >Personally, I would be suspicious of developmental web services doing
> >auto-reloading while an installer is recompiling the world. I don't have
> >enough context to be sure how plaus
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/16/2013 06:59 PM, Nick Coghlan wrote:
> 3.2 uses __pycache__, so it should only potentially conflict within
> the same version.
>
> I haven't heard any rumblings about anything like this in Fedora or
> RHEL, so my suspicions still lean toward
On 17 May 2013 08:37, "Guido van Rossum" wrote:
>
> On Thu, May 16, 2013 at 3:27 PM, Greg Ewing
wrote:
> > Guido van Rossum wrote:
> >>
> >> This reminds me of the following bug, which can happen when two
> >> processes are both writing the .pyc file and a third is reading it.
> >> ... I think a
On Thu, May 16, 2013 at 3:27 PM, Greg Ewing wrote:
> Guido van Rossum wrote:
>>
>> This reminds me of the following bug, which can happen when two
>> processes are both writing the .pyc file and a third is reading it.
>> ... I think all the errors are
>>
>> actually explainable from this scenario
Guido van Rossum wrote:
This reminds me of the following bug, which can happen when two
processes are both writing the .pyc file and a third is reading it.
... I think all the errors are
actually explainable from this scenario.
The second writer will still carry on to write a valid
.pyc file,
On Thu, May 16, 2013 at 11:19 PM, Guido van Rossum wrote:
> This reminds me of the following bug, which can happen when two
> processes are both writing the .pyc file and a third is reading it.
> First some background.
>
> When writing a .pyc file, we use the following strategy:
> - open the file
On 5/16/2013 5:30 PM, Brett Cannon wrote:
On Thu, May 16, 2013 at 5:19 PM, Guido van Rossum wrote:
This reminds me of the following bug, which can happen when two
processes are both writing the .pyc file and a third is reading it.
First some background.
When writing a .pyc file, we use the fol
On Thu, May 16, 2013 at 5:40 PM, Guido van Rossum wrote:
> I still suspect this might explain most of what Barry saw, if not all.
Quite possible, especially since he is seeing more issues on 3.2 than
3.3. Just wanted to fill people in on how 3.3 onwards does things is
all.
-Brett
> —
> Sent fro
I still suspect this might explain most of what Barry saw, if not all.
—
Sent from Mailbox
On Thu, May 16, 2013 at 2:36 PM, Brett Cannon wrote:
> On Thu, May 16, 2013 at 5:19 PM, Guido van Rossum wrote:
>> This reminds me of the following bug, which can happen when two
>> processes are both wr
On Thu, May 16, 2013 at 5:19 PM, Guido van Rossum wrote:
> This reminds me of the following bug, which can happen when two
> processes are both writing the .pyc file and a third is reading it.
> First some background.
>
> When writing a .pyc file, we use the following strategy:
> - open the file
This reminds me of the following bug, which can happen when two
processes are both writing the .pyc file and a third is reading it.
First some background.
When writing a .pyc file, we use the following strategy:
- open the file for writing
- write a dummy header (four null bytes)
- write the .py f
On 5/16/2013 2:04 PM, Barry Warsaw wrote:
No, it's all different kinds of machines, at different times, on different
files. So far, there's no rhyme or reason to the corruptions that I can
tell.
If the corruption only happens on Ubuntu, that would constitute 'rhyme'
;-). I realize that askin
On Thu, May 16, 2013 at 11:04 AM, Barry Warsaw wrote:
> On May 16, 2013, at 09:44 AM, Ethan Furman wrote:
>
> >Is it happening on the same machines? If so, perhaps a daemon to monitor
> >those files and then scream and shout when one changes. Might help track
> >down what's going on at the time
On May 16, 2013, at 09:44 AM, Ethan Furman wrote:
>Is it happening on the same machines? If so, perhaps a daemon to monitor
>those files and then scream and shout when one changes. Might help track
>down what's going on at the time. (Yeah, that does sound like saying
>'inotify' but with more wo
On 05/16/2013 09:38 AM, Barry Warsaw wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On May 16, 2013, at 06:22 PM, Christian Heimes wrote:
Are you able to reproduce the issue? Perhaps you could use inotify to
track down file activity. It shouldn't affect timing much and you can
track if
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On May 16, 2013, at 06:22 PM, Christian Heimes wrote:
>Are you able to reproduce the issue? Perhaps you could use inotify to
>track down file activity. It shouldn't affect timing much and you can
>track if more than one process it writing to the sam
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Am 16.05.2013 17:40, schrieb Barry Warsaw:
> We've since found a few cases where Python 3.3 pyc files are
> probably corrupted, so that shoots down my theory about a race
> condition on reading/writing pyc files, since 3.3 implements
> atomic-rename
On May 16, 2013, at 08:33 AM, Nick Coghlan wrote:
>Personally, I would be suspicious of developmental web services doing
>auto-reloading while an installer is recompiling the world. I don't have
>enough context to be sure how plausible that is as a possible explanation,
>though.
It's possible tha
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On May 15, 2013, at 06:06 PM, Tres Seaver wrote:
>On 05/15/2013 04:58 PM, Barry Warsaw wrote:
>> This leads me to hypothesize that the bug is due to an as yet
>> unidentified race condition during installation of Python source code
>> on Ubuntu, whi
On 16 May 2013 08:11, "Tres Seaver" wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 05/15/2013 04:58 PM, Barry Warsaw wrote:
> > This leads me to hypothesize that the bug is due to an as yet
> > unidentified race condition during installation of Python source code
> > on Ubuntu,
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 05/15/2013 04:58 PM, Barry Warsaw wrote:
> This leads me to hypothesize that the bug is due to an as yet
> unidentified race condition during installation of Python source code
> on Ubuntu, which is normally when we automatically byte compile the
>
On Wed, May 15, 2013 at 4:58 PM, Barry Warsaw wrote:
> I am looking into a particularly vexing Python problem on Ubuntu that
> manifests in several different ways. I think the problem is the same one
> described in http://bugs.python.org/issue13146 and I sent a message on the
> subject to the ubu
I am looking into a particularly vexing Python problem on Ubuntu that
manifests in several different ways. I think the problem is the same one
described in http://bugs.python.org/issue13146 and I sent a message on the
subject to the ubuntu-devel list:
https://lists.ubuntu.com/archives/ubuntu-devel
37 matches
Mail list logo
