On 31/12/15 03:26, Ludovic Courtès wrote:
Ben Woodcroft <[email protected]> skribis:
On 29/12/15 15:46, Ben Woodcroft wrote:
Unfortunately none of these builds are reproducible because rubygems
in Guix generally aren't. For one, this is because .gem files are
archives whose contents are timestamped.
I should clarify. What I meant was the cache .gem files
/gnu/store/ib83mg5zsyr5x2w0m3i1f84gdvdbp5x9-ruby-ascii85-1.0.2/lib/ruby/gems/2.2.0/cache$
tar tvf Ascii85-1.0.2.gem |head
-r--r--r-- wheel/wheel 703 2015-12-27 22:44 metadata.gz
-r--r--r-- wheel/wheel 7436 2015-12-27 22:44 data.tar.gz
-r--r--r-- wheel/wheel 268 2015-12-27 22:44 checksums.yaml.gz
We should arrange so that gems are created with a fixed timestamp and
UID/GID, and a well-defined file ordering, as with:
--mtime=@0 --sort=name --owner=root:0 --group=root:0
We also need to make sure gzip is always run with -n/--no-name. That
way, the gz files above will not include an additional timestamp.
From what I can see in
<git://git.debian.org/git/reproducible/notes.git>, this is not addressed
yet in other distros.
Ludo are you suggesting we should abandon the deletion approach?
On 30/12/15 18:26, Ricardo Wurmus wrote:
Ben Woodcroft <[email protected]> writes:
The .gem file stored in GEM_HOME after install is both redundant and an
archive that stores timestamped files which makes builds non-deterministic. So
delete it after 'gem install'.
Good idea! I don’t know if the existence of the cached gem is checked
for by any Ruby tools (bundler or the like). Is there some
documentation about this cache?
I wondered that too, but I built all of the ruby packages again without
issue and many of them use bundler. It also doesn't seem like a good
idea for bundler to use cached gems since I would guess that gems that
are downloaded but fail to install are kept in the cache. I also wasn't
able to see any mention of the cache in the rubygems API.
- (zero? (apply system* "gem" "install" (first-matching-file "\\.gem$")
- "--local" "--ignore-dependencies"
- ;; Executables should go into /bin, not /lib/ruby/gems.
- "--bindir" (string-append out "/bin")
- gem-flags))))
+ (apply system* "gem" "install" gem-name
+ "--local" "--ignore-dependencies"
+ ;; Executables should go into /bin, not /lib/ruby/gems.
+ "--bindir" (string-append out "/bin")
+ gem-flags)
+ ;; Remove the cached gem file as this is unnecessary and contains
+ ;; timestamped files rendering builds not reproducible.
+ (delete-file (string-append gem-home "/cache/" gem-name))
+ #t))
I’d prefer to keep ‘(zero? ...)’ and only delete the file when the
‘system*’ call above succeeded. It would be nice if we could propagate
any bad return value from ‘system*’ to the end of the procedure.
Maybe something like this:
(and (zero? (apply system* ...))
(begin (delete-file ...) #t))
It’s a bit clunky but the return value would still be #f if ‘system*’
fails. What do you think?
I think you are right as usual. Better in attached?
Thanks,
ben
>From c00a032644c02474212be97c185c0953967de4e5 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft <[email protected]>
Date: Wed, 30 Dec 2015 10:27:33 +1000
Subject: [PATCH] build: ruby: Remove cached gem after install.
The .gem file stored in GEM_HOME after install is both redundant and an
archive that stores timestamped files which makes builds non-deterministic. So
delete it after 'gem install'.
* guix/build/ruby-build-system.scm (install): Remove cached gem after install.
---
guix/build/ruby-build-system.scm | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/guix/build/ruby-build-system.scm b/guix/build/ruby-build-system.scm
index 2685da1..6439bf6 100644
--- a/guix/build/ruby-build-system.scm
+++ b/guix/build/ruby-build-system.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 David Thompson <[email protected]>
;;; Copyright © 2015 Pjotr Prins <[email protected]>
+;;; Copyright © 2015 Ben Woodcroft <[email protected]>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -115,15 +116,19 @@ GEM-FLAGS are passed to the 'gem' invokation, if present."
(assoc-ref inputs "ruby"))
1))
(out (assoc-ref outputs "out"))
- (gem-home (string-append out "/lib/ruby/gems/" ruby-version ".0")))
-
+ (gem-home (string-append out "/lib/ruby/gems/" ruby-version ".0"))
+ (gem-name (first-matching-file "\\.gem$")))
(setenv "GEM_HOME" gem-home)
(mkdir-p gem-home)
- (zero? (apply system* "gem" "install" (first-matching-file "\\.gem$")
- "--local" "--ignore-dependencies"
- ;; Executables should go into /bin, not /lib/ruby/gems.
- "--bindir" (string-append out "/bin")
- gem-flags))))
+ (and (apply system* "gem" "install" gem-name
+ "--local" "--ignore-dependencies"
+ ;; Executables should go into /bin, not /lib/ruby/gems.
+ "--bindir" (string-append out "/bin")
+ gem-flags)
+ ;; Remove the cached gem file as this is unnecessary and contains
+ ;; timestamped files rendering builds not reproducible.
+ (begin (delete-file (string-append gem-home "/cache/" gem-name))
+ #t))))
(define %standard-phases
(modify-phases gnu:%standard-phases
--
2.6.3
