[issue34722] Non-deterministic bytecode generation
Change by Peter Ebden : -- keywords: +patch pull_requests: +8885 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue34722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34722] Non-deterministic bytecode generation
Peter Ebden added the comment: Thanks for the pointer, I'll have a bit more of a dig into it (although Serhiy makes a good point too...). -- ___ Python tracker <https://bugs.python.org/issue34722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34722] Non-deterministic bytecode generation
New submission from Peter Ebden :
We've found that the following code produces non-deterministic bytecode,
even post PEP-552:
def test(x):
if x in {'ONE', 'TWO', 'THREE'}:
pass
It's not too hard to test it:
$ python3.7 -m compileall --invalidation-mode=unchecked-hash test.py
Compiling 'test.py'...
$ sha1sum __pycache__/test.cpython-37.pyc
61e5682ca95e8707b4ef2a79f6454dafd800 __pycache__/test.cpython-37.pyc
$ rm __pycache__/test.cpython-37.pyc
$ python3.7 -m compileall --invalidation-mode=unchecked-hash test.py
Compiling 'test.py'...
$ sha1sum __pycache__/test.cpython-37.pyc
222a06621b491879e5317b34e9dd715bacd89b7d __pycache__/test.cpython-37.pyc
It looks like the peephole optimiser is converting the LOAD_CONST instructions
for the set into a single LOAD_CONST for a frozenset which then serialises in
nondeterministic order. One can hence work around it by setting PYTHONHASHSEED
to a known value.
I'm happy to help out with this if needed, although I don't have a lot of
familiarity with the relevant code.
--
components: Interpreter Core
messages: 325644
nosy: Peter Ebden
priority: normal
severity: normal
status: open
title: Non-deterministic bytecode generation
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue34722>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29094] Regression in zipfile writing in 2.7.13
New submission from Peter Ebden:
In Python 2.7.13, using zipfile.ZipFile to write into a file with some initial
preamble produces a zip file that cannot be read again by some zip
implementations.
Our use case is using pex (https://github.com/pantsbuild/pex) which writes a
zip that begins with a shebang, and later attempting to manipulate that using
Go's standard archive/zip package. In 2.7.12 that works OK, but in 2.7.13 the
.pex file is rejected on reading. Linux's command-line unzip tool will read the
archive, but issues a warning ("4 extra bytes at beginning or within zipfile")
which wasn't present previously.
zipfile.ZipFile does read the files OK.
I assume this is related to https://bugs.python.org/issue26293 since that's the
most obvious zipfile change in 2.7.13. It's pretty easy to reproduce using the
example in that issue:
from zipfile import ZipFile
with open('a.zip', 'wb') as base:
base.write(b'old\n')
with ZipFile(base, 'a') as myzip:
myzip.write('eggs.txt')
unzip -t a.zip
Archive: a.zip
warning [a.zip]: 4 extra bytes at beginning or within zipfile
(attempting to process anyway)
...
--
components: Library (Lib)
messages: 284172
nosy: Peter Ebden
priority: normal
severity: normal
status: open
title: Regression in zipfile writing in 2.7.13
versions: Python 2.7
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29094>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
