bug#22533: Python bytecode reproducibility

Ricardo Wurmus skribis: > Ricardo Wurmus writes: > >> Now that we’re using Python 3.7 and this version supports hash-based pyc >> files, is this still an issue? Do we need to do anything to enable >> hash-based pyc compilation? >> >> See: >>

bug#22533: Python bytecode reproducibility

Ricardo Wurmus writes: > Now that we’re using Python 3.7 and this version supports hash-based pyc > files, is this still an issue? Do we need to do anything to enable > hash-based pyc compilation? > > See: > https://docs.python.org/3/whatsnew/3.7.html#pep-552-hash-based-pyc-files >

bug#22533: Python bytecode reproducibility

Now that we’re using Python 3.7 and this version supports hash-based pyc files, is this still an issue? Do we need to do anything to enable hash-based pyc compilation? See: https://docs.python.org/3/whatsnew/3.7.html#pep-552-hash-based-pyc-files https://www.python.org/dev/peps/pep-0552/

bug#22533: Python bytecode reproducibility

2018-03-04 20:18 GMT+01:00 Ricardo Wurmus : > I have applied this patch locally: > > > diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm > index 5f701701a..0d1ecc3c6 100644 > --- a/gnu/packages/python.scm > +++ b/gnu/packages/python.scm > @@ -359,8 +359,42 @@ data

bug#22533: Python bytecode reproducibility

2018-03-06 15:43 GMT+01:00 Ricardo Wurmus : > > Ricardo Wurmus writes: > > > Marius Bakke writes: > > > >> I suppose we'll have to set PYTHONHASHSEED somewhere in > >> python-build-system as well. Did you check if that makes a

bug#22533: Python bytecode reproducibility

Ricardo Wurmus writes: > Marius Bakke writes: > >> I suppose we'll have to set PYTHONHASHSEED somewhere in >> python-build-system as well. Did you check if that makes a difference >> for numpy? Perhaps it's enough to set it if we add an

bug#22533: Python bytecode reproducibility

Marius Bakke writes: > The only remark I have is: is introducing a new variable necessary? > SOURCE_DATE_EPOCH implies that the user wants a deterministic build; > the upstream patch doesn't actually honor it outside of making the > hashing method deterministic. So, I

bug#22533: Python bytecode reproducibility

Ricardo Wurmus writes: > I have applied this patch locally: > > diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm > index 5f701701a..0d1ecc3c6 100644 > --- a/gnu/packages/python.scm > +++ b/gnu/packages/python.scm > @@ -359,8 +359,42 @@ data types.") >

bug#22533: Python bytecode reproducibility

Ricardo Wurmus writes: > Ricardo Wurmus writes: > >> Unfortunately, this doesn’t fix all reproducibility problems with numpy: >> >> --8<---cut here---start->8--- >> Binary files >>

bug#22533: Python bytecode reproducibility

Gábor Boskovits writes: > 2018-03-05 16:36 GMT+01:00 Gábor Boskovits : > >> 2018-03-05 1:05 GMT+01:00 Ricardo Wurmus : >> >>> >>> Ricardo Wurmus writes: >>> >>> > Unfortunately, this doesn’t fix all

bug#22533: Python bytecode reproducibility

2018-03-05 16:36 GMT+01:00 Gábor Boskovits : > 2018-03-05 1:05 GMT+01:00 Ricardo Wurmus : > >> >> Ricardo Wurmus writes: >> >> > Unfortunately, this doesn’t fix all reproducibility problems with numpy: >> > >> > --8<---cut

bug#22533: Python bytecode reproducibility

2018-03-05 1:05 GMT+01:00 Ricardo Wurmus : > > Ricardo Wurmus writes: > > > Unfortunately, this doesn’t fix all reproducibility problems with numpy: > > > > --8<---cut here---start->8--- > > Binary files

bug#22533: Python bytecode reproducibility

Hello! Ricardo Wurmus skribis: > Is it a bad idea to override the timestamps in the generated binaries? > I think that we could avoid the recency check then, which was an > obstacle to resetting the timestamps of the source files. I think it’s good if we can fix Python

bug#22533: Python bytecode reproducibility

Ricardo Wurmus writes: > Unfortunately, this doesn’t fix all reproducibility problems with numpy: > > --8<---cut here---start->8--- > Binary files >

bug#22533: Python bytecode reproducibility

I have applied this patch locally: diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 5f701701a..0d1ecc3c6 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -359,8 +359,42 @@ data types.") "Lib/ctypes/test/test_win32.py" ; fails

bug#22533: Python bytecode reproducibility

2018-03-04 13:46 GMT+01:00 Ricardo Wurmus : > > Hi Gábor, > > > Nix had this issue, it seems they have a python 3.5 solution, which > > should be easy to adopt: https://github.com/NixOS/nixpkgs/issues/22570. > > WDYT? > > Here’s the patch for Nix: > >

bug#22533: Python bytecode reproducibility

Hi Gábor, > Nix had this issue, it seems they have a python 3.5 solution, which > should be easy to adopt: https://github.com/NixOS/nixpkgs/issues/22570. > WDYT? Here’s the patch for Nix: https://patch-diff.githubusercontent.com/raw/NixOS/nixpkgs/pull/22585.diff Here are the relevant

bug#22533: Python bytecode reproducibility

2018-03-03 23:37 GMT+01:00 Ricardo Wurmus : > Hi Guix, > > Marius Bakke writes: > > > It would be great to revive this longstanding bug! > > Indeed. > > Here’s another attempt. As far as I understand, the timestamp in the > pyc files only affects the

bug#22533: Python bytecode reproducibility

Hi Guix, Marius Bakke writes: > It would be great to revive this longstanding bug! Indeed. Here’s another attempt. As far as I understand, the timestamp in the pyc files only affects the header. Up until Python 3.6 (incl) the header looks like this: magic | timestamp

bug#22533: Python bytecode reproducibility

Hello! I stumbled across this bug after re-discovering that Python bytecode is not reproducible (through "glib"). Just sharing some notes.. Nix recently made an effort to fix this. AFAICT the ".pyc" files are still a problem, but at least they got the interpreters building reproducibly: