Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
Hi Niels, On Sonntag, 7. Juni 2015, Niels Thykier wrote: I see no partial issue in setting this in either dh xor dh_auto_build, provided it is one ENV for all tools (and not one for each tool). Is the variable still actual? I vaguely remember seeing a mention of a SOURCEDATE_UTC or something similar not too long ago in #d-reproducible. dkg wrote in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=787444#65 on June 5th 2015: On Fri 2015-06-05 10:55:34 -0400, Brendan O'Dea wrote: Any of UTC_SOURCE_DATE, SOURCE_DATE_UTC My vote is for SOURCE_DATE_UTC, and i agree with Brendan that we should take the opportunity to define this as strictly and narrowly as possible (i.e. end in a 'Z', none of the other offsets), so that people relying on it know they're getting a fixed thing, and don't have to implement any fancy parsing/offsetting code if they're not already using an ISO8601-compliant date-parsing library. to which I agreed in #787444 and Lunar on irc (SOURCE_DATE_UTC sounds like a good proposal) and noone objected, so I think SOURCE_DATE_UTC is it. cheers, Holger signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On 2015-06-04 17:24, Holger Levsen wrote: Hi, On Donnerstag, 4. Juni 2015, Ximin Luo wrote: https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-2 0150330/001317.html for details. TL;DR is to have SOURCEDATE as an environment variable in ISO8601 format. It would be awesome for help2man to support this. At some point, debhelper can even set this environment variable automatically. sounds good to me! Niels, what do you think? cheers, Holger Hi, I see no partial issue in setting this in either dh xor dh_auto_build, provided it is one ENV for all tools (and not one for each tool). Is the variable still actual? I vaguely remember seeing a mention of a SOURCEDATE_UTC or something similar not too long ago in #d-reproducible. Thanks, ~Niels ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On 05/06/15 04:19, Daniel Kahn Gillmor wrote: On Thu 2015-06-04 21:51:37 -0400, Brendan O'Dea wrote: Local times, and daylight savings are just too much of a PITA. Just use UTC and if builds on the first of the month are possibly different to the changelog, so be it. I agree with Brendan here. If there are no objections to the idea that we're sticking with ISO-8601 in UTC, preferably with the Z suffix (e.g. 2015-06-05T01:08:20Z) then we just need to settle on what the right name is. If we're going to mandate that it ends with Z, might I suggest that we add UTC or _UTC to the variable name? It leaves the option open in the future that we might allow TZ offsets. Note that the TZ offsets mentioned in ISO8601 and the other RFC standards are not time zones or local times that are subject to DST. They are *fixed offsets*, and don't require extra context (such as the TZ database) to parse correctly. So it would be less of a PITA than you might think. Lunar^, you'd mentioned that there had been discussions about a preferred variable name that you might like better than SOURCEDATE. Any preference? Where should we document this? Holger had previously suggested to me to put it on a subpage in the reproducible builds section on the Debian Wiki, and I agree that would be a good place. X -- GPG: 4096R/1318EFAC5FBBDBCE git://github.com/infinity0/pubkeys.git signature.asc Description: OpenPGP digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On Thu 2015-06-04 21:51:37 -0400, Brendan O'Dea wrote: Local times, and daylight savings are just too much of a PITA. Just use UTC and if builds on the first of the month are possibly different to the changelog, so be it. I agree with Brendan here. If there are no objections to the idea that we're sticking with ISO-8601 in UTC, preferably with the Z suffix (e.g. 2015-06-05T01:08:20Z) then we just need to settle on what the right name is. Lunar^, you'd mentioned that there had been discussions about a preferred variable name that you might like better than SOURCEDATE. Any preference? Where should we document this? --dkg ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On Thu 2015-06-04 10:38:53 -0400, Ximin Luo wrote: A few months ago I had an idea for this that would be more generalisable. See https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20150330/001317.html for details. TL;DR is to have SOURCEDATE as an environment variable in ISO8601 format. I'm fine with this part of your proposal. Your proposal also includes a bunch of workarounds with faketime, which i'm a little concerned about (i say this as the faketime maintainer in debian, not wanting its particular flakiness in the build path for everything). Can we separate the $SOURCEDATE part of your proposal from the faketime part and just work on $SOURCEDATE independently? What TZ should SOURCEDATE have? ISO8601 is capable of supplying a TZ as well, so the current time could be written in a wide variety of ways while meaning the same instant: 0 dkg@alice:~$ date '+%FT%T%z' date -u '+%FT%T%z' 2015-06-04T13:25:25-0400 2015-06-04T17:25:25+ 0 dkg@alice:~$ I feel like we should we always set it to UTC, so that the inbound parsed offset would be +. sound sensible? It would be awesome for help2man to support this. help2man (and any other tool that accepts $SOURCEDATE) would also need to ensure that it extracts the parts it wants in a TZ-independent fashion as well. (not parsing back to localtime) At some point, debhelper can even set this environment variable automatically. We should open a bug with debhelper requesting this feature as soon as we come to agreement on the name and semantics. --dkg ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On 04/06/15 19:30, Daniel Kahn Gillmor wrote: What TZ should SOURCEDATE have? ISO8601 is capable of supplying a TZ as well, so the current time could be written in a wide variety of ways while meaning the same instant: 0 dkg@alice:~$ date '+%FT%T%z' date -u '+%FT%T%z' 2015-06-04T13:25:25-0400 2015-06-04T17:25:25+ 0 dkg@alice:~$ I feel like we should we always set it to UTC, so that the inbound parsed offset would be +. sound sensible? I had thought about this a bit, and not yet decided the best way - hence why I haven't yet written this idea up on the Debian Wiki. FWIW, here are my thoughts: - Always setting it to UTC would be simplest, but then our format wouldn't be ISO8601 - we'd have to say ISO8601 but omit the offset / ignore any offset. RFC 2822 and 3339, the other formats mentioned in `man date`, also include an offset. - It's neater to keep the TZ-offset the same as in debian/changelog. Generated dates will then be consistent with debian/changelog. If debian/changelog says Mar 2015, then the generated documentation will say Mar 2015. - However, it seems awkward to get date(1) to preserve the original offset. I suppose this is an artefact of using localtime(3) as you mentioned. In general the libc time stuff seems to have disastrous behaviour if you want to play with time zones other than local time or UTC, and this affects some other languages like python. - One can maybe hack around it with the TZ envvar, but it has a very weird syntax [1], and the hack is dependent on the actual value: $ date -d 2015-06-04T20:18:13+0800 --iso-8601=seconds 2015-06-04T14:18:13+0200 # nope, my local TZ is different $ TZ=UTC+08 date -d 2015-06-04T20:18:13+0800 --iso-8601=seconds 2015-06-04T04:18:13-0800 $ TZ=UTC-08 date -d 2015-06-04T20:18:13+0800 --iso-8601=seconds 2015-06-04T20:18:13+0800 # argh, TZ expects the opposite sign [1] http://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html - A possible workaround is to advise people to just extract the information directly out of SOURCEDATE using a regex (or strip off the last 5 chars of SOURCEDATE before giving it to localtime, maybe). But such advice is extra mental work for developers who may not bother reading the document that we put such advice on. It would be awesome for help2man to support this. help2man (and any other tool that accepts $SOURCEDATE) would also need to ensure that it extracts the parts it wants in a TZ-independent fashion as well. (not parsing back to localtime) X -- GPG: 4096R/1318EFAC5FBBDBCE git://github.com/infinity0/pubkeys.git signature.asc Description: OpenPGP digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On Thu 2015-06-04 08:17:59 -0400, Brendan O'Dea wrote:
The idea was that unless that environment variable was set, the
behaviour would be unchanged: the current date would be used. It
seems unlikely that DEB_BUILD_CHANGELOG would be set accidentally.
right, but this wouldn't be terribly useful for folks like fedora, who
also ship help2man and might want to be reproducible:
https://admin.fedoraproject.org/pkgdb/package/help2man/
http://securityblog.redhat.com/2013/09/18/reproducible-builds-for-fedora/
If the build system is going to set environment variables, maybe it
could set an environment variable with a parseable date string
(ISO-8601?) and help2man could look at that environment variable and
extract a string from it? (ideally this extraction step would be
TZ-independent, too)
That would also work, and if your reproducible build environment was
to pull the changelog date and stick it into an environment variable,
then you should only need to patch the tools rather than the Makefile
of every package which uses them.
At worst, for the tools you can't patch, you're in the same place you
are now--having to patch all the Makefiles, and even that is perhaps
easier as you already have the date:
--date=${DEB_BUILD_CHANGELOG_DATE:-$(date --iso-8601=seconds)}
Yep, i think this makes sense; reproducible-build folks -- do we want to
take this tack with help2man? If so, what would the environment
variable be named? what would its contents be?
--dkg
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
Hi, On Donnerstag, 4. Juni 2015, Ximin Luo wrote: https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-2 0150330/001317.html for details. TL;DR is to have SOURCEDATE as an environment variable in ISO8601 format. It would be awesome for help2man to support this. At some point, debhelper can even set this environment variable automatically. sounds good to me! Niels, what do you think? cheers, Holger signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#787444: help2man: support externally-supplied --date for reproducibility
On 4 June 2015 at 00:19, Daniel Kahn Gillmor d...@fifthhorseman.net wrote:
On Wed 2015-06-03 08:24:51 -0400, Brendan O'Dea wrote:
My inclination is instead to do something fairly specific to your
project: If the environment variable DEB_BUILD_CHANGELOG (or something
similar) is set, then the file to which it refers will be used to find
the latest revision date, and that date will be used on the manual
pages. Presumably the build system could set this prior to
build/test?
i think this could work, but it might not apply particularly well to
anyone outside of debian who uses help2man. I note that help2man is a
native package -- do you know if it's used outside of debian at all?
help2man is part of the GNU project, but since I'm the upstream and
the debian maintainer I build it as a native package, dupload the
results to Debian and upload the tarball to ftp.gnu.org at the same
time.
The idea was that unless that environment variable was set, the
behaviour would be unchanged: the current date would be used. It
seems unlikely that DEB_BUILD_CHANGELOG would be set accidentally.
Making the help2man run dpkg-parsechangelog also seems a little hairier
than it ought to be, and introduces a dependency on dpkg-dev which seems
a little weird.
I wasn't going to use dpkg-parsechangelog, it is a sufficiently
standard format that a simple capturing regex would suffice.
If the build system is going to set environment variables, maybe it
could set an environment variable with a parseable date string
(ISO-8601?) and help2man could look at that environment variable and
extract a string from it? (ideally this extraction step would be
TZ-independent, too)
That would also work, and if your reproducible build environment was
to pull the changelog date and stick it into an environment variable,
then you should only need to patch the tools rather than the Makefile
of every package which uses them.
At worst, for the tools you can't patch, you're in the same place you
are now--having to patch all the Makefiles, and even that is perhaps
easier as you already have the date:
--date=${DEB_BUILD_CHANGELOG_DATE:-$(date --iso-8601=seconds)}
Regards,
Brendan
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
