Re: Bug#862112: #862112: r-base-dev: Generate reproducible output independently of the build path
Sorry, was out traveling / running a relay. Now back. On 15 May 2017 at 17:12, Ximin Luo wrote: | Holger Levsen: | > control: found -1 3.4.0-1 | > control: notfound -1 3.4.0-1.0~reproducible2 | > | > Hi Dirk, | > | > I've asked Ximin to file this bug so that we have something to track and to | > refer in discussions… | > | > you wrote: | > | >> At work now with limited time, but I think I already told you twice that | >> there were two of the three parts of the patch you proposed to upstream that | >> I would not take, were I upstream (which I am not). | > | > Dirk, could you please point out (here in this bug report) which of the two parts | > you consider unsuitable? | > | | We talked about this over private email, this refers to the patch hunks in: | | - src/library/tools/R/admin.R | - src/library/tools/R/parseRd.R | | The suggestion was to guard them behind a command-line option using getOption, so presumably Debian could set this whilst upstream / other distros do not. Corret. As posted, the patch changes existing behaviour _unconditionally_ so I don't not think I can say with a straight face to upstream that they should take this. And also: you can turn them on for your builds, I won't at first, but advertise so that more users can test it. "Eventually" I may turn them on as well. The R "universe" is _at least_ the (currently around) 10600+ packages (yes, really, 10 thousand) on CRAN. The fact that we successfully rebuilt our 400+ is important aspect but nowhere near comprehensive enough. | However, since I'm not familiar with the full R codebase I was waiting to see if upstream had any further comments, because even with the getOption() change there might be other consequences that I didn't foresee. In this case it would be beneficial for Debian's behaviour to exactly match upstream, so we get the same bugs (if any). | | > Ximin, looking at https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds | > I believe it would be best to merge those two (top most) commits into one? | > | >> A decent longer-term strategy may well to stress-test the patch by including | >> it in our builds for a while. We can work on that. | > | > actually we've been using Ximin's patches on tests.reproducible-builds.org | > since the 2nd of May on amd64 and since the 3rd on arm64, armhf and i386. | > | > According to https://tests.reproducible-builds.org/debian/index_performance.html | > (top row labeled "oldest build result) in the first table on the right) this | > means we've almost done a full rebuild with the patch on amd64+arm64 (probably | > 85% done now) and pretty close to that on i386… | > | > According to this, this patch seems to work well… | > | | The patch works well for getting stuff built, but I haven't tested all of | the *behaviour* of these packages. (Some probably have unit tests, but these | don't cover everything at any rate.) Exactly. | That is why I wanted to wait for upstreams' comments first, before adding the getOption guards - which are relatively minor IMO, compared to what the full patch does. Correct if I am wrong but you have not heard back from upstream, have you? [ That is not uncommon for posts on r-devel. Sometimes one needs to be persistent, and/or ally with an R Core maintainer. Which is pretty much what I suggested early one. ] | I also haven't tested other potential tools that might process R rdb files, who might for some reason expect absolute build paths. Correct as well. We are moving in the right direction, but we should not rush this. Upstream *is* sympathetic, they have taken an earlier 'repro build' patch. Dirk | X | | -- | GPG: ed25519/56034877E1F87C35 | GPG: rsa4096/1318EFAC5FBBDBCE | https://github.com/infinity0/pubkeys.git -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: #862112: r-base-dev: Generate reproducible output independently of the build path
Holger Levsen: > control: found -1 3.4.0-1 > control: notfound -1 3.4.0-1.0~reproducible2 > > Hi Dirk, > > I've asked Ximin to file this bug so that we have something to track and to > refer in discussions… > > you wrote: > >> At work now with limited time, but I think I already told you twice that >> there were two of the three parts of the patch you proposed to upstream that >> I would not take, were I upstream (which I am not). > > Dirk, could you please point out (here in this bug report) which of the two > parts > you consider unsuitable? > We talked about this over private email, this refers to the patch hunks in: - src/library/tools/R/admin.R - src/library/tools/R/parseRd.R The suggestion was to guard them behind a command-line option using getOption, so presumably Debian could set this whilst upstream / other distros do not. However, since I'm not familiar with the full R codebase I was waiting to see if upstream had any further comments, because even with the getOption() change there might be other consequences that I didn't foresee. In this case it would be beneficial for Debian's behaviour to exactly match upstream, so we get the same bugs (if any). > Ximin, looking at > https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds > I believe it would be best to merge those two (top most) commits into one? > >> A decent longer-term strategy may well to stress-test the patch by including >> it in our builds for a while. We can work on that. > > actually we've been using Ximin's patches on tests.reproducible-builds.org > since the 2nd of May on amd64 and since the 3rd on arm64, armhf and i386. > > According to > https://tests.reproducible-builds.org/debian/index_performance.html > (top row labeled "oldest build result) in the first table on the right) this > means we've almost done a full rebuild with the patch on amd64+arm64 (probably > 85% done now) and pretty close to that on i386… > > According to this, this patch seems to work well… > The patch works well for getting stuff built, but I haven't tested all of the *behaviour* of these packages. (Some probably have unit tests, but these don't cover everything at any rate.) That is why I wanted to wait for upstreams' comments first, before adding the getOption guards - which are relatively minor IMO, compared to what the full patch does. I also haven't tested other potential tools that might process R rdb files, who might for some reason expect absolute build paths. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: #862112: r-base-dev: Generate reproducible output independently of the build path
On Sun, May 14, 2017 at 01:19:44PM +0200, Holger Levsen wrote: > According to > https://tests.reproducible-builds.org/debian/index_performance.html > (top row labeled "oldest build result) in the first table on the right) this > means we've almost done a full rebuild with the patch on amd64+arm64 (probably > 85% done now) and pretty close to that on i386… > > According to this, this patch seems to work well… actually, we've done a full rebuild of all r-* packages on amd64, not just 85%: Ximin scheduled them all manually on amd64 before filing this bug… :) -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
re: #862112: r-base-dev: Generate reproducible output independently of the build path
control: found -1 3.4.0-1 control: notfound -1 3.4.0-1.0~reproducible2 Hi Dirk, I've asked Ximin to file this bug so that we have something to track and to refer in discussions… you wrote: > At work now with limited time, but I think I already told you twice that > there were two of the three parts of the patch you proposed to upstream that > I would not take, were I upstream (which I am not). Dirk, could you please point out (here in this bug report) which of the two parts you consider unsuitable? Ximin, looking at https://anonscm.debian.org/cgit/reproducible/r-base.git/log/?h=pu/reproducible_builds I believe it would be best to merge those two (top most) commits into one? > A decent longer-term strategy may well to stress-test the patch by including > it in our builds for a while. We can work on that. actually we've been using Ximin's patches on tests.reproducible-builds.org since the 2nd of May on amd64 and since the 3rd on arm64, armhf and i386. According to https://tests.reproducible-builds.org/debian/index_performance.html (top row labeled "oldest build result) in the first table on the right) this means we've almost done a full rebuild with the patch on amd64+arm64 (probably 85% done now) and pretty close to that on i386… According to this, this patch seems to work well… -- cheers, Holger signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
