[issue25083] Python can sometimes create incorrect .pyc files

Change by STINNER Victor : -- components: +Interpreter Core ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25083] Python can sometimes create incorrect .pyc files

Change by STINNER Victor : -- resolution: -> fixed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25083] Python can sometimes create incorrect .pyc files

Change by Petr Viktorin : -- stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___ Python-bugs-list

[issue25083] Python can sometimes create incorrect .pyc files

Petr Viktorin added the comment: New changeset f64c813de84011a84ca21d75a294861a9cc2dfdc by Petr Viktorin (tzickel) in branch '2.7': bpo-25083: Python can sometimes create incorrect .pyc files (GH-8449) https://github.com/python/cpython/commit/f64c813de84011a84ca21d75a294861a9cc2dfdc

[issue25083] Python can sometimes create incorrect .pyc files

Change by Brett Cannon : -- nosy: -brett.cannon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25083] Python can sometimes create incorrect .pyc files

Change by tzickel : -- pull_requests: +7971 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue25083] Python can sometimes create incorrect .pyc files

Change by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

Petr Viktorin added the comment: tzickel, could you sign the CLA? It will formally allow PSF to distribute your patch with Python. (You retain the copyright.) See the dev guide for instructions: https://docs.python.org/devguide/pullrequest.html#cla I can't do that for you,

[issue25083] Python can sometimes create incorrect .pyc files

Nick Coghlan added the comment: Adding a couple of Red Hat folks to the nosy list, as even though this was originally reported for Windows, the reproducers show that it's a cross-platform issue. tzickel would you mind signing the Python CLA and turning your patch into a

[issue25083] Python can sometimes create incorrect .pyc files

Gregory P. Smith added the comment: I'm un-cc'ing myself as I don't use Python 2 in an environment where we allow it to write .pyc files at application runtime. -- ___ Python tracker

[issue25083] Python can sometimes create incorrect .pyc files

Change by Gregory P. Smith : -- nosy: -gregory.p.smith ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: Added a script to check if the bug exists (provided you have an updated strace 4.15 or above). Without patch: # ./import_io_check.sh strace: Requested path 'tmp.py' resolved into '/root/tmp.py' read(3, 0x55fc3a71cc50, 4096) = -1 ENOSYS

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: Ignore the hash append / link at the start of each shell command (it's the output from docker, and not related to python commits). BTW, forgot to mention, of course when doing the fault injection on the .py files, the error is bad as well, it

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: OK, This issue has been biting me a few more times in production, so for now I've added the environment variable PYTHONDONTWRITEBYTECODE which resolves it (but it's a hack). I'm sure I am not the only one with it (recall that this is happening

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: Sorry Brett of course I meant the upcoming 2.7.12 -- ___ Python tracker ___

[issue25083] Python can sometimes create incorrect .pyc files

Changes by Brett Cannon : -- nosy: -brett.cannon ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

Brett Cannon added the comment: Python 2.6 is no longer supported. -- ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: any chance for 2.6.12 ? 4 line patch. -- ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

Changes by tzickel : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

Brett Cannon added the comment: As I said previously, tzickel, this is an edge case in Python 2.7 which means it has a low priority for most core developers. Realize that we are also working towards a Python 3.5.1 release and a 3.4.4 release as well (which will be the first and last bugfix

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: Meador Inge any other questions regarding the issue ? I can't believe 2.7.11 is coming out soon, and nobody is taking this issue seriously enough... -- ___ Python tracker

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: 1. You are correct the issue I am talking about is in parsing source files (Altough because python caches them as .pyc it's a worse situation). 2. The example you give is EINTR handling (which is mostly handling interrupted I/O operations by signals and retrying

[issue25083] Python can sometimes create incorrect .pyc files

Brett Cannon added the comment: Nope, I think you probably have everyone who might be interested in looking at fixing this on the issue, tzickel. Just a matter of someone taking the interest and initiative to fix this in Python 2.7. -- ___ Python

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: TL:DR Python 2 forgot to do I/O error checking when reading .py files from disk. On some rare situations this can bite Python in the ass and cause it to bork .pyc files. Checked python 3, it checks the I/O in a different / better way. Next python 2.7 is out in 1.5

[issue25083] Python can sometimes create incorrect .pyc files

Meador Inge added the comment: I started poking at the patch a little and have a few comments. My understanding of the issue comments is that the read error actually happens when reading in the *source* file and *not* the bytecode file. This happens because 'ferror' is not checked after

[issue25083] Python can sometimes create incorrect .pyc files

Brett Cannon added the comment: I have not looked at your patch, tzickel, but I just wanted to let you know a lack of reply just means lack of time on the part of the core developers. We are all volunteers and essentially don't get paid to spend our time to work on Python. Add in the fact

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: Not sure why nobody has responded yet, but I have gone up and made a patch for the problem for 2.7 HEAD. Would be great if someone with more understanding of python's source could say if this is the optimal place to do the ferror test. I am able to see that this

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: As for the "example" .pyc just create an empty 0 byte .py file and compile it, that is the same .pyc that is created in my system (instead in my case the .py is not empty). Just so people don't have to trace the code like I did, here is the traceback of the primary

[issue25083] Python can sometimes create incorrect .pyc files

tzickel added the comment: You are not looking at the correct code, the function you are pointing to, check_compiled_module is run to check the existing .pyc (it is a good question, why the .pyc is overriden, but that is a secondary issue, which I cannot reproduce as I've said by demand). I

[issue25083] Python can sometimes create incorrect .pyc files

Changes by tzickel : -- nosy: +brett.cannon, meador.inge ___ Python tracker ___ ___

[issue25083] Python can sometimes create incorrect .pyc files

Gregory P. Smith added the comment: we'll need a test case .pyc where this happens. -- ___ Python tracker ___

[issue25083] Python can sometimes create incorrect .pyc files

Gregory P. Smith added the comment: Inspecting the 2.7 code: https://hg.python.org/cpython/file/2.7/Python/import.c#l761 following that down to https://hg.python.org/cpython/file/2.7/Python/marshal.c#l1126 it looks like it does the right thing on EOF error (from either getc or from fread)