Re: [Reproducible-builds] Support for --ignore-profile flag in diffoscope
Ximin Luo: > This is quite an open-ended problem and there is no single "correct" > answer. I don't even know myself what would be best, at this stage. I think what we need to come up with now is a list of use cases. Then we can decide which one we want to support and how easy it should be. Is anyone willing to share examples where being able to ignore stuff would have made their life easier? The last one I spotted that could go on the list, ignoring irrelevant differences in two Android App packages: https://github.com/WhisperSystems/Signal-Android/blob/master/apkdiff/apkdiff.py -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] [diffoscope] Support for --ignore-profile flag in diffoscope
Ximin Luo: > Concretely I have some suggestions: > > 1. instead of calling this "ignore" we call it "hide". and instead of > "irrelevant" we say "common"/"minor"/"known" Great suggestions! :) > 2. diffoscope --ignore-* (or --hide-*) MUST NOT return 0 or otherwise > give the impression that two non-identical files are the same, even if > all differences are "hidden". It should report "n differences hidden". There's a tradeoff here: you don't really know how many differences are going to be hidden until you've computed them, and that might be costly. Use case: Jane is trying to understand if her patch removing __TIME__ from the source code is enough to make the binary reproducible. As she knows how this affect the binary, she get faster results and avoid clutter by adding the flag `--hide=debug-symbols` or `--hide-pattern='*.debug'`. I'm writing “faster results” because we can avoid unpacking the debug symbols and comparing them which are costly operations. If we display “n differences hidden”, then we always have to compare everything… -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#829113: Bug#829113: reprotest: Should unset $DISPLAY to avoid GUI popups from some build tools
Hi Axel! Axel Beckert: > Running > > $ reprotest 'dpkg-buildpackage -b' > ../debian-paketmanagement-buch_0\~2016.06.29_all.deb > > from a git checkout of commit 5f069a920df4e6f20a8eb9309c20c39ad60e6132 > under X to see if my $SOURCE_DATE_EPOCH implementation in that commit is > working correctly. > >* What was the outcome of this action? > > Two GUI popups from ebook-convert (from the package calibre) moaning > about non-existent $HOME and hence unreadable $HOME/.config/… > >* What outcome did you expect instead? > > No interactivity, especially no GUI interactivity at all. > > Unsetting $DISPLAY should this issue. > > (I'm not sure if setting and not setting $DISPLAY is or should be one of > the not explicitly listed variations. But since I got that popup twice, > I assume not.) I don't think this is a bug in reprotest. As far as I can see by looking in dpkg source tree, `dpkg-buildpackage` doesn't do anything to the DISPLAY variable etiher. So I would assume building the package might fail just as well without reprotest. If building the package requires to unset DISPLAY, I think this should be done in `debian/rules`… One take though, maybe reprotest should ensure that $HOME is set to an existing (temporary) directory; other building tools might get unhappy. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Remaining reprotest variations
Ceridwen: > For most of the variations I've done so far, I've been either > depending on external utilities or had POSIX-compliant ways to execute > them. The rest of the variations pose more problems. > > 1. user_group. The POSIX standard includes the notion of user/group > ids, but the only ways it defines to change the uid/gid of processes > are C functions. Unfortunately, there's also nothing in the POSIX > standard for creating users, though at least `chown` is. At the > moment, my plan is to use `useradd`, `groupadd`, `userdel`, > `groupdel`, and `su`. (Using 'sudo' like prebuilder does is more > complicated and less likely to be supported everywhere.) I'm going to > set the ids to something like 2. This page, > http://bhami.com/rosetta.html, indicates that this won't work on > FreeBSD or MacOS X, and thatis supported by these pages, > http://www.math.utah.edu/~beebe/unix/g/groupadd.html and > http://www.math.utah.edu/~beebe/unix/u/useradd.html. > - What works on FreeBSD/MacOS X? > - Are there any other problems I'm likely to encounter using `su`? If the code abstract user creation/removal depending on the system of the execution environment, we could implement (more) specific behaviors when required. Maybe that could be an answer: make it so experts in other operation system can easily add support? This applies to several variations you've listed as well, IMHO. > 6. time: tests.reproducible-builds.org sets the system clock into the > future for some machines, which doesn't work here. I can try to use > libfaketime for non-qemu environments and make the VM clock > independent for qemu, but this is likely to get very tricky. How much > time should I spend on this? We found many many problems that were related to time once all the checks were in place, so it would be good if reprotest could have similar support. Being able to run a different clock in qemu would be best, I think. > Beyond the specific variations, I have some questions about reversion > that apply to several variations. I'm trying to have reprotest revert > all the changes it makes on its own, to make it more useful for > simpler execution environments, but guaranteeing this under all > conditions is tricky. > > * If reprotest is called as root without any virtualization > (i.e. null), should it attempt to run the variations that require > root privileges on the host system? These are host, domain, bin_sh, > and user_group. I'm reluctant to disable this functionality > altogether, because I can see someone wanting it, but it definitely > has the potential to cause undesirable side effects if something > goes wrong. > > * Along the same line, `chsh` affects all user shells. This doesn't > require root privileges, but should it be tested without > virtualization? I think we should mandate isolated environment for anything that could make the system go wrong. Or require a “--i-dont-care-if-this-system-gets-broken” extra flag. ;) Thanks for your insightful summary! :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#838260: diffoscope: Reduce noise from offsets deltas in readelf(1) diffs
Chris Lamb: >> Could these offset differences in readelf(1) output be ignored, at least >> optionally? > > Love the idea! However, my gut cautions against ignoring them. even with an > option. > > Perhaps there is a perfect solution whereby we would normalise these two > offsets to — making it up here! — relative values, but simply need to > nclude that we have done that once in the diff. That way, we have a) still > captured the underlying issue, b) reduced the noise, and c) avoided a > cumbersome option flag. One idea that crossed my mind at some point that might be able to solve this as well: be able to record other kinds of differences than just line-oriented ones. Initially, I thought of this as a way to add image comparison as I felt sad not knowing any free software that could easily provide similar features to what GitHub offers [1]. But why stop with images? In the precise case of the readelf output, having line-oriented diff means we are carrying around a useless and confusing information: the line numbers are not helpful in anyway to locate and undrstand the differences. But what if we could replace the line numbers by the instruction addresses? Then the noise mentioned by Daniel disappears. Meanwhile, the actual output will become even more relevant. Such an approach would require some structural changes to the code, but could have benefits on many fronts. [1]: https://help.github.com/articles/rendering-and-diffing-images/ Hope that's any useful, -- Lunar.''`. lunar at debian.org : :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: OpenPGP digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#838260: diffoscope: Reduce noise from offsets deltas in readelf(1) diffs
Daniel Shahaf: > Jérémy Bobbio wrote on Tue, Sep 20, 2016 at 13:18:49 +: >> But why stop with images? In the precise case of the readelf output, >> having line-oriented diff means we are carrying around a useless and >> confusing information: the line numbers are not helpful in anyway to >> locate and undrstand the differences. >> >> But what if we could replace the line numbers by the instruction >> addresses? Then the noise mentioned by Daniel disappears. Meanwhile, the >> actual output will become even more relevant. > > In the example in the OP, the (source code) line numbers and instruction > addresses are the same between both builds. It is the .rodata addresses > embeddded into the instructions that differ. Thanks for pointing this out, I had actually misunderstood the problem at hand. :) -- Lunar.''`. lunar at debian.org : :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: OpenPGP digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [dpkg] 08/08: document 1.18.10.0~reproducible1
Mattia Rizzolo: > The reason we like to have a static string in there (and in jenkins we > use the host architecture), is that otherwise diffoscope would be noisy > about the difference in the name if we compared the .changes files […]. Would it still be the case after Satyam's work which resulted in https://anonscm.debian.org/cgit/reproducible/diffoscope.git/commit/?id=c61f41c5128094508d6fff8b746cecc3841f6b27 ? This change has been shipped as part of diffoscope/60 from what I can see. -- Lunar.''`. lunar at debian.org : :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: OpenPGP digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: package uploaded to our repo
Mattia Rizzolo: > Next steps: > it would be great if somebody could figure what's the real gain of > having .dsc in Checksums-Sha256. This is how .buildinfo files have linked a source package in an unambiguous manner to the resulting binaries. I see how the current specification is giving you trouble, but it's kinda their “raison d'être”. -- Lunar.''`. lunar at debian.org : :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: OpenPGP digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#848049: diffoscope: Add detection of order-only differences in plain text formats
Маша Глухова:
> I believe the attached patch would provide the requested functionality.
Nice work! :)
> From: Maria Glukhova
> Date: Sat, 24 Dec 2016 12:29:57 +0200
> Subject: [PATCH] Add detection of order-only difference in plain text format.
>
> Detect if the text files' contents differ only in line ordering, and give an
> appropriate comment.
> […]
> +def order_only_difference(unified_diff):
> +diff_lines = unified_diff.splitlines()
> +added_lines = [line[1:] for line in diff_lines if line.startswith('+')]
> +removed_lines = [line[1:] for line in diff_lines if line.startswith('-')]
> +# Faster check: does number of lines match?
> +if len(added_lines) != len(removed_lines):
> +return False
> +# Counter stores line and number of its occurrences.
> +return sorted(added_lines) == sorted(removed_lines)
I guess it's a fine approach to the problem, but I wonder if it would
not be better to use a slightly less accurate strategy that would be
nicer to memory and CPU.
What I have in mind is to incrementally compute a hash value that would
give the same result even if the lines are in different order.
Drawing from discussions on StackOverflow [1], I think doing a sum of
Python's hash() would work. My test was:
def unordered_hash(lines):
h = 0
for line in lines:
h += hash(line)
return h
h1 = unordered_hash(open('tests/data/text_order1').readlines())
h2 = unordered_hash(open('tests/data/text_order2').readlines())
print(h1, h2, h1 == h2)
That way, it could probably be implemented directly in the difference
module and work for other file types than just text files.
[1]:
https://stackoverflow.com/questions/30734848/order-independant-hash-algorithm
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#849142: [PATCH] Update dex_expected_diffs and test requirement to ensure test compatibility with enjarify >= 1.0.3. (Closes: #849142)
Chris Lamb:
> Daniel Shahaf wrote:
>
> > > +if subprocess.call(
> > > +('python3', '-c', 'import enjarify.typeinference'),
> >
> > Use sys.executable instead of hardcoding 'python3', to handle the case
> > that there's more than one python3 binary on the system?
>
> I deliberately used python3 to match the behaviour of what the
> /usr/bin/enjarify script does.
Guess it's worth adding a comment about it for future readers. :)
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#848049: diffoscope: Add detection of order-only differences in plain text formats
Hi! Маша Глухова: > The reason why I did not use some algorihm like that is that it requires to > read files for the second time. Right now, all the actual work with the > content of the files (except for the quick check for has_same_content) is > delegated to diff, and on big files, it occupies most of the time. Assuming > that for big files, reading them from drive would be the bottleneck, I > tried to avoid reading them again, instead working with the result of the > diff. > Still, I would be happily mistaken. I will implement your version and > compare the performance. You would not have to read the file twice as long as you do the hash in the difference module, when each line is actually fed to diff. A similar trick is already used to cope with files that are too long, see diffoscope.difference.make_feeder_from_raw_reader() I don't know if my suggestions is a good one. It might not be a good idea at all. Feel free to discuss it with your mentor before spending too much time on it. > Thank you again :) PS: Please call me Lunar. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Bug#888237: Fwd: Re: diffoscope and file renames
Chris Lamb: > > I had no idea I should have installed it, and indeed I was missing it. > > … which is itself a bug! However, the solution is not very obvious - why > can't really detect whether we need would benefit from fuzzy matching without > doing said fuzzy matching! > > Will have a think :) In that case, it might have worked to run difflib.SequenceMatcher on archive paths to compare their distance. I'm not sure if it's a good idea at all. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#796039: diffoscope: please recognize big-endian Haskell .hi files
Package: diffoscope Severity: normal diffoscope currently doesn't know how to parse Haskell .hi files written on big-endian architectures as such. This should be fixed. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] SOURCE_DATE_EPOCH specification document
Chris Lamb:
> Mattia wrote:
> > > After a hint from doko, I've started work on an official-looking spec
> > > for SOURCE_DATE_EPOCH
> >
> > Lamby pointed out that this is not something debian-related, so it would
> > be great to have it outsite a debian.{org,net} site.
> >
> > OTOH also the howto is not strictly debian-related, and both documents
> > are related to reproducibly.
> >
> > My personal proposal is to merge the two documents and move them to a new
> > more meaningful url (under rb.d.n).
>
> I believe this would handicap adoption of SOURCE_DATE_EPOCH in the wider
> software community.
Strongly seconded. The HOWTO is meant to be a general document about all
things related to “reproducible builds”. It's an “how to”: it's intent
is to describe several way to achieve reproducible builds.
It is way different than a specification which mention the one true way
on how things should work on specific behaviors.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] package uploaded to our repo
dpkg_1.18.3~reproducible1.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] reproducible .buildinfo inclusion in the archive and output by dpkg
Ansgar Burchardt: > Holger Levsen writes: > > together with Lunar we four sad together on the last Saturday of DebConf15 > > in > > Heidelberg and discussed the next steps forward to achieve the inclusion of > > .buildinfo inclusion in the Debian archive and output by dpkg. > > > > On the ftpmaster side we agreed that: > > > > - dak/queued has to be changed to accept .buildinfo files > > - will be stored on ftp-master, concatted and compressed it will be exposed > > to > > the mirrors > > - one per arch + suite, aka for each Packages file > > How large are these? I'm sure the snapshot.d.o maintainers would prefer > something that does not change with each mirror push, or is not part of > the dists/ tree pushed to mirrors. Holger tested it to be about 14 MB for amd64 once compressed. This included Arch:all packages. If we use a single tar archive, with `gzip --rsyncable`, I assume the difference on each mirror push will be small. But we probably could use reproducible.debian.net to come up with actual numbers if needed. > > - Packages file gets a certfied-by field: > > Build-Signed-Off-By: 0603CCFD91865C17E88D4C798382C95C29023DF9 > > Jérémy > > Bobbio which should include the checksum of the > > .buildinfo > > file (or maybe not, see above) > > I think having an external service for these is better: otherwise we > have to deal with who can add signatures, and probably would limit it to > people in Debian's keyring (I don't want ftp-master to deal with > external parties). A seperate service could accept signatures from > everybody, including parties not directly involved in Debian or > automated systems. > > Also adding even more data to the Packages indicies is something I would > like to avoid: the files are already quite large. I understand the latter concern, so maybe having this information in Packages is a bad idea and we need another index, or none as efficiency is maybe not a problem. But I'm quite convinced that the .buildinfo files need signatures to be any useful. With reproducible builds, we can easily provide a trust path from the source to the binary that goes beyond the archive signing key. I think limiting addition of these certifications (I assert that I have successfuly built the given binary packages with the given source in the mentioned environment = I sign the .buildinfo) to the Debian archives to Debian Developers is a fine thing to do. Derivatives that partially mirror the Debian archives are free to add their own. And the tools should also support external signature repositories. But the Debian archive should provide signatures of the .buildinfo so at the very least we know which buildd came up with these binary packages in the first place. Please note that solving the question of signatures is not strictly needed to move “reproducible builds” forward for now. We would like to see the changes merged in dpkg as this makes all Debian packages unreproducible currently. And this is currently blocking on having build information available. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#797525: Bug#797525: diffoscope: multiarch mode
clone 797525 -1 retitle -1 diffoscope: provide option to disable fuzzy-matching thanks Hi! Jakub Wilk: > I want to use diffoscope to compare two "Multi-Arch: same" debs of the same > version but different architecture, to see differences that will cause > co-installation conflicts. > > This almost works currently, but there's a bit of noise that could be > avoided. I'd like an option that does the following: > - Ignores all differences in control.tar. I had already in mind to provide pluggable ignore modules. So having support for this use case is on my personal roadmap. > - Ignores timestamp differences in data.tar. I can offer to ignore difference in metadata in the data.tar tarball. Ignoring all timestamp differences in not doable, they could take any form in any files. > - Disables fuzzy matching. That will be a separate option. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#792687: gettext: please support timestamps from environment
Santiago Vila: > The problem here is that some packages execute xgettext *every* single > time the package is built (creating a different .pot file each time), > and then they also perform msgmerge with the new .pot file and the > existing .po files, creating different .po files each time. > > None of that is necessary at all: > > For a given source package and debian revision, you only need to > create the .pot file once, for example, just before dpkg-buildpackage. Thanks for working on this and I think I agree. But I myself hit a wall while trying to handle software using intltool. Do you have an idea on how to handle them? If I remember correctly, gdm3 is one example. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#792687: gettext: please support timestamps from environment
Santiago Vila: > Excluding .pot files from what is considered to be the "source" might > be part of the problem. See tor-monitor upstream's reaction, for example: https://lists.alioth.debian.org/pipermail/pkg-privacy-maintainers/Week-of-Mon-20150824/15.html Quoting Alan from there: > I didn't add the .pot in git because it's generated at build time. > Adding it duplicates info and adds the risk that it's not up-to-date. I'd be happy to hear more arguments for/against. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#797759: Bug#797759: diffoscope: tar.gz and tar.bz2 archives compared wrongly
Control: retitle -1 diffoscope: implement fuzzy-matching across containers Control: severity -1 wishlist Michele Alessandrini: > When comparing tar.gz or tar.bz2 archives containing text files, some > of which different, the output is a binary diff. Perhaps it only > applies decompression and compares the two tar files, instead of their > contents? That can be implemented by adding support for fuzzy-matching across containers. This would also detect and compare properly a file being compressed from one version to the next. It's doable. The trickiest part in my mind so far is where to attach them in the tree of differences. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#797759: Bug#797759: Bug#797759: diffoscope: tar.gz and tar.bz2 archives compared wrongly
Michele Alessandrini: > Il 02/09/2015 14:38, Jérémy Bobbio ha scritto: > >Control: retitle -1 diffoscope: implement fuzzy-matching across containers > >Control: severity -1 wishlist > > > >Michele Alessandrini: > >>When comparing tar.gz or tar.bz2 archives containing text files, some > >>of which different, the output is a binary diff. Perhaps it only > >>applies decompression and compares the two tar files, instead of their > >>contents? > > > >That can be implemented by adding support for fuzzy-matching across > >containers. This would also detect and compare properly a file being > >compressed from one version to the next. > > > >It's doable. The trickiest part in my mind so far is where to attach > >them in the tree of differences. > > > > But isn't reading files inside archives a main feature of diffoscope? It is. But .gz and .bz2 are two different file formats and diffoscope treat them as such. It's like switching from .tar to .zip. It's quite a difference and diffoscope will not blindly compare their content currently. Trust me, fuzzy matching accross containers is doable and will solve exactly this problem, but also others. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#797560: diffoscope: option to treat absent files as empty
Control: tag -1 + pending Jakub Wilk: > I'd like an option for treating absent files as if they were empty, similar > to "diff -N". This has not been easy, but this will be in the next release (--new-file is the switch). -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#788364: libmagic1: misdetect some Coreboot images as text
retitle 788364 diffoscope: garbled output when comparing some Coreboot images clone 788364 -1 reassign -1 libmagic1 severity -1 libmagic1 normal retitle -1 libmagic1: misdetect Coreboot images as text files thanks Hi Christoph, diffoscope is the tool that we have created as part of the “reproducible builds” effort to understand differences between two builds. We now also use it to compare builds of Coreboot images. diffoscope uses libmagic (through its Python bindings) to identify the format of the files its trying to compare. Some coreboot images are misdetected as text files which results in garbled diffoscope output. Proper way to detect Coreboot images is probably to look for a CBFS header. cbfs_find_header() is how upstream does it: http://review.coreboot.org/gitweb?p=coreboot.git;a=blob;f=util/cbfstool/cbfs_image.c;h=c40bd6641 I could tell diffoscope to detect Coreboot images with a similar mechanism but it would probably be better to teach libmagic to do it. Is that easily doable? Reiner Herrmann: > file detects them as plain-text: > > > /tmp/b1_coreboot.rom: ISO-8859 text, with very long lines, with no line > > terminators > > /tmp/b2_coreboot.rom: ISO-8859 text, with very long lines, with no line > > terminators > > That's why diffoscope also treats them as text. > I'm not sure this can/should be fixed inside diffoscope, as we rely on > libmagic detecting them correctly. Reiner, I remember you had a look into this during DebConf. Have you made any progress? -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#797525: Bug#797525: diffoscope: multiarch mode
Control: retitle -1 diffoscope: provide a way to ignore all differences in control.tar Jérémy Bobbio: > Jakub Wilk: > > I want to use diffoscope to compare two "Multi-Arch: same" debs of the same > > version but different architecture, to see differences that will cause > > co-installation conflicts. > > > > This almost works currently, but there's a bit of noise that could be > > avoided. I'd like an option that does the following: > > - Ignores all differences in control.tar. > > I had already in mind to provide pluggable ignore modules. So having > support for this use case is on my personal roadmap. Let's retitle this bug accordingly. > > - Disables fuzzy matching. > > That will be a separate option. Pending upload, this will be doable with `--fuzzy-threshold=0`. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#798384: strip-nondeterminism: should preserve file permissions
Package: strip-nondeterminism Severity: important Hi! I'm opening a bug report to track the following issue discussed on the reproducible-builds mailing list: Eugene Zhukov: > On Mon, Sep 7, 2015 at 8:11 PM, Andrew Ayer wrote: > > On Mon, 7 Sep 2015 19:49:56 +0300 > > Eugene Zhukov wrote: > >> I was testing my epubcheck package built with ExperimentalToolchain > >> and noticed that jar file executable permissions get stripped. > >> If I drop dh_strip_nondeterminism line from > >> /usr/share/cdbs/1/rules/debhelper.mk everything works as expected. > >> I don't really understand how this is related so I'd appreciate if you > >> could have a look. > > > > Do you mean the jar file itself, or files within the jar? > The jar file itself. > > > (dh-)strip-nondeterminism should leave the execute bit alone within the > > jar, but I just checked and confirmed that it doesn't preserve the > > execute bit on the jar itself. I can fix this. Thanks Andrew! -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] [report] Week 20 in Stretch cycle
[ Chris Lamb suggested to post the weekly reports that has been so far ] [ available on Planet Debian to the mailing-list. Please tell us if] [ you think that's not a good idea.] What happened in the [1]reproducible builds effort this week: Media coverage Motherboard published an [2]article on the project inspired by the [3]talk at the Chaos Communication 15. Journalists [4]sadly rarely pick their headlines. The sensationalist “How Debian Is Trying to Shut Down the CIA” got started a few rants here and there. [5]One from OpenBSD developper Ted Unangst lead to a good email contact and some [6]thorough comments. Toolchain fixes * Emmanuel Bourg uploaded maven-ant-helper/7.11 which improved the reproducibility of the Javadoc by removing the timestamps and using the English locale. * Thomas Schmitt uploaded libisoburn/1.4.0-2 which adds to the ISO image creator xorriso new flags for -alter_date to avoid update ctimes. [7]Report by Daniel Kahn Gillmor. * Florian Schlichting uplodaded libmodule-build-perl/0.421400-2 which makes linked file ordering deterministic. [8]Original patch by Niko Tyni. The modified version of gettext has been [9]removed from the experimental toolchain. Fixing individual package seems a better approach for now. Chris Lamb sent two patches for abi-compliance-checker: one to [10]drop the timestamp from generated HTML reports and another to [11]make umask and timestamps deterministic in the abi tarball. [12]Bugs submitted by Dhole lead to a discussion on the best way to adapt pod2man now that we have [13]SOURCE_DATE_EPOCH specified. There is really a whole class of issues that are currently undiscovered waiting for tests running on a different date. This is likely to should happen soon. Chris Lamb uploaded a new version of debhelper in the [14]“reproducible” repository, cherry-picking a fix for interactions between ddebs and udebs. Packages fixed The following packages became reproducible due to changes in their build dependencies: aspic, django-guardian, erlang-sqlite3, etcd, libnative-platform-java, mingw-ocaml, nose2, oar, obexftp, py3cairo, python-dugong, python-secretstorage, python-setuptools, qct, qdox, recutils, s3ql, wine. The following packages became reproducible after getting fixed: * bochs/2.6-4 by Santiago Vila. * codec2/0.4-3 by A. Maitland Bottoms. * coquelicot/0.9.4-1 by Lunar. * criticalmass/1:1.0.0-3 by Santiago Vila. * ekg/1:1.9~pre+r2855-3 by Santiago Vila. * eterm/0.9.6-3 by Santiago Vila. * fbi/2.10-2 by Moritz Muehlenhoff. * fsvs/1.2.6-2 by Santiago Vila. * glhack/1.2-3 by Santiago Vila. * httraqt/1.4.6-2 by Anton Gladky. * libapache-authznetldap-perl/0.07-6 by gregor herrmann, [15]original patch by Dhole. * libkinosearch1-perl/1.01-3 uploaded by Florian Schlichting, [16]original patch by Niko Tyni. * liblucy-perl/0.3.3-6 uploaded by Florian Schlichting, [17]original patch by Niko Tyni. * slony1-2/2.2.4-1 by Christoph Berg. * slrn/1.0.2-3 uploaded by Moritz Muehlenhoff, [18]original patch by Dmitry Bogatov. * svtplay-dl/0.10.2015.08.24-1 uploaded by Olof Johansson, fixed upstream. * swh-plugins/0.4.15+1-8 uploaded by Jaromír Mikeš, [19]original patch by Chris Lamb. * sysstat/11.1.6-2 uploaded by Robert Luberda, [20]original patch by Chris Lamb. * uhd/3.9.0-3 by A. Maitland Bottoms. * volk/1.1-3 by A. Maitland Bottoms. * yadifa/2.1.3-2 uploaded by Markus Schade, [21]original patch by Santiago Vila. Some uploads fixed some reproducibility issues but not all of them: * dict-jargon/4.4.7-3 uploaded by Ruben Molina, [22]original patch by Dhole. * ferret-vis/6.9.3-3 uploaded by Alastair McKinstry, [23]original patch by Chris Lamb. Patches submitted which have not made their way to the archive yet: * 798366 on lilo by Dmitry Bogatov: remove usage of __TIME__ and __DATE__ macros. * 798557 on libapache-dbi-perl by Dhole: set date of the manpage to the latest debian/changelog entry. * 798776 on testdisk by [24]upstream! reproducible.debian.net The configuration of all remote armhf and amd64 nodes in now finished. The remaining reproducibility tests running on the Jenkins host has been removed. armhf results and graphs are now visible in [25]dashboard. We can now test the whole archive in 2-3 weeks using the current 12 amd64 jobs and 3 months using the current 6 armhf builders. We will be looking at improving the armhf sitation, maybe using more native systems or via arm64. (h01ger) The Jenkins UI is now more responsive since all jobs building packages have been moved to remote hosts. (h01ger) [26]A new job has been added to
Re: [Reproducible-builds] [Debian Wiki] Update of "ReproducibleBuilds/TimestampsProposal" by BenBoeckel
Daniel Kahn Gillmor:
> Wiki page:
> > - EXECUTE_PROCESS(COMMAND "date" "-u" "-d @${SOURCE_DATE_EPOCH}"
> > "+%d/%m/%Y" OUTPUT_VARIABLE BUILD_DATE)
> […]
> Is there a reason above that this example code to translate from
> SOURCE_DATE_EPOCH to a build date doesn't produce ISO-8601-compliant
> date strings in the BUILD_DATE variable?
I assume it's because it was initially in a piece of software that used
the above format. I'd say feel free to replace by a ISO-8601 format
string if you feel it matters.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] .buildinfo should contain source hashes (as well as binary hashes)
Ximin Luo: > With our current .buildinfo setup, the above process is more > complicated, because we *only* store hashes of the binary build > environment. I'm sorry but this is not accurate regarding the current specification [1]. It says: Build-Environment List of all packages forming the build environment, their architecture if different from build architecture, and their version. The idea to put a hash of the binary package in the Build-Environment is a late addition to the original idea. It came as a way to make `srebuild` job easier: retrieving a specific binary package with its hash is already part of snapshot.debian.org interface. It also makes unecessary to find the relevant repository snapshot and the related headaches with how to handle expired signatures. In any cases, we currently don't have code to store any hash of the Build-Environment. If we wanted to store hashes of binary packages, then we would need to have them in /var/lib/dpkg/status and it's not done yet, even if Guillem said this would be a good thing to have. > Currently, to run a DDC test, we would have to read the buildinfo > file, find the hashes of the binary build-deps, lookup the source > packages that corresponds to these hashes, find a different binary > build-deps for these hashes, and run our DDC-checker. This takes many > round trips, and contacting external infrastructure that isn't > necessary. You would not need to lookup the source packages using hashes. Using package and version gives you enough info to retrieve a specific source package from the archive. > If .buildinfo files contained source hashes, the DDC-checker could > work more directly, without requiring a remote repository of source > hash <-> binary hash mappings. I'm interested in `.buildinfo` in the context of the Debian project. The Debian archive is designed to be immutable. A specific version of a package will always correspond to the same source and binary files. So I don't see why one would do complex “source hash - binary hash mapping” when you can just rely on what is in the archive (and what has been archived by snapshot.debian.org). If by building thing that ought to match a specific package version you get different result, then you will have to investigate in any cases. Implementation-wise, getting the hash of the .dsc in the .buildinfo is going to be very tricky. dpkg does not know about what's available in the archive. It just knows about packages which are or were installed. [1]: https://wiki.debian.org/ReproducibleBuilds/BuildinfoSpecification -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#774274: fontforge: please allow scripts to set the font modification time
Jérémy Bobbio: > For font compilation to be deterministic, we need a way to set the font > modification time to a defined value instead of the current time. > > We thus add a new function “SetModificationTime” to the scripting language. > It takes a single string argument looking like “2014-12-30 21:18:33”. Now that we have SOURCE_DATE_EPOCH defined [1], I wonder if this is the right approach. Just looking for the variable in the environment and using it as the font modification time sounds an easier approach to implement and to make many Debian packages reproducible without any modifications. [1]: https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] package uploaded to our repo
dpkg_1.18.3.0~reproducible1.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] .buildinfo should contain source hashes (as well as binary hashes)
Ximin Luo: > > Implementation-wise, getting the hash of the .dsc in the .buildinfo is > > going to be very tricky. dpkg does not know about what's available in > > the archive. It just knows about packages which are or were installed. > > > > `apt-cache showsrc [pkg]` has the right information in there, but it's a bit > messy. I need to test this without a deb-src line, though. Building Debian packages doesn't involve APT in any ways. There is currently no coupling in the direction dpkg → APT. (That's why we need to get hash of the binary packages in /var/lib/dpkg/status before they can be written in dpkg-genbuildinfo.) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#799690: file: regression in MIME encoding lookups
Package: file Version: 1:5.25-1 Severity: important Hi! The latest upload of file broke diffoscope test suite. With file/1:5.24-2: $ file --mime-encoding tests/data/binary1 tests/data/binary1: binary With file/1:5.25-1: $ file --mime-encoding tests/data/binary1 tests/data/binary1: application/octet-streambinary The test file is only 16 bytes long and can be downloaded at: https://anonscm.debian.org/cgit/reproducible/diffoscope.git/plain/tests/data/binary1 Thanks, -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Week 21 in Stretch cycle
What happened in the [1]reproducible builds effort this week: Media coverage Nathan Willis covered [2]our DebConf15 status update in Linux Weekly News. Access to non-LWN subscribers will be given on Thursday 24th. Linux Journal published a [3]more general piece last Tuesday. Unexpected praise for reproducible builds appeared this week in the form of several iOS applications identified as including spyware. The malware was undetected by Apple screening. This actually happened because application developers had simply [4]downloaded a trojaned version of XCode through an unofficial source. While reproducible builds can't really help users of non-free software, this is exactly the kind of attacks that we are trying to prevent in our systems. Toolchain fixes * Mathieu Malaterre uploaded abi-compliance-checker/1.99.11-1 which [5]drops the timestamps from the generated HTML reports and [6]makes the generated .abi.tar.gz files reproducible. Original patches by Chris Lamb. Niko Tyni wrote and uploaded [7]a better patch for the source order problem in libmodule-build-perl. Tristan Seligmann [8]identified how the code generated by python-cffi could be emitted in random order in some cases. Upstream has already [9]fixed the problem. Packages fixed The following 24 packages became reproducible due to changes in their build dependencies: apache-curator, checkbox-ng, gant, gnome-clocks, hawtjni, jackrabbit, jersey1, libjsr305-java, mathjax-docs, mlpy, moap, octave-geometry, paste, pdf.js, pyinotify, pytango, python-asyncssh, python-mock, python-openid, python-repoze.who, shadow, swift, tcpwatch-httpproxy, transfig. The following packages became reproducible after getting fixed: * apparmor/2.10-2 uploaded by intrigeri, fixed upstream by Christian Boltz, with the same change [10]suggested by Reiner Herrmann. * ardour/1:4.2~dfsg-2 by IOhannes m zmölnig. * dcmtk/3.6.1~20150629-1 uploaded by Andreas Tille, [11]original patch by akira. * deap/1.0.1-4 by Daniel Stender. * firebird2.5/2.5.4.26856.ds4-2 by Damyan Ivanov. * gamera/3.4.2+svn1437-1 by Daniel Stender. * genometools/1.5.7-1 by Sascha Steinbiss. * golang-github-go-xorm-core/0.4.4-1 by Alexandre Viau. * klibc/2.0.4-4 by Ben Hutchings. * libgtk2-perl/2:1.2496-3 by intrigeri. * lsof/4.89+dfsg-0.1 uploaded by Laurent Bigonville, [12]original patch by Lunar. * monotone/1.1-6 by Markus Wanner. * ndisc6/1.0.1-4 by Santiago Vila. * privoxy/3.0.23-4 by Roland Rosenfeld. * ruby-flexmock/2.0.0~rc1-1 by Antonio Terceiro. * ruby-html2haml/2.0.0-1 by Lunar. * tunnelx/20140102-3 uploaded by Wookey, [13]original patch by Chris Lamb. * wtforms/2.0.2-1 by Orestis Ioannou, [14]original patch by Juan Picca. Some uploads fixed some reproducibility issues but not all of them: * maxima/5.37-1 by Camm Maguire, [15]report by akira. Patches submitted which have not made their way to the archive yet: * 783152 on kmod by Lunar: export SOURCE_DATE_EPOCH in debian/rules. * 799010 on 389-ds-base by Chris Lamb: use SOURCE_DATE_EPOCH value as the build date. * 799206 on python-sqlalchemy-utils by Chris Lamb: sort the list of extra requirement. * 799330 on cappuccino by Chris Lamb: pass a fixed seed to polygen. * 799410 on segment by Chris Lamb: use date of the latest debian/changelog entry as build date. reproducible.debian.net Tests for [16]Coreboot, [17]OpenWrt, [18]NetBSD, and [19]FreeBSD now runs weekly (instead of monthly). diffoscope development Python 3 offers new features (namely yield from and concurrent.futures) that could help implement parallel processing. The clear separation of bytes and unicode strings is also likely to reduce encoding related issues. Mattia Rizolo thus kicked the effort of porting diffoscope to Python 3. tlsh was the only dependency missing a Python 3 module. This got quickly fixed by a new upload. The rest of the code has been [20]moved to the point where only incompatibilities between Python 2.7 and Pyhon 3.4 had to be changed. The commit stream still require some cleanups but all tests are now passing under Python 3. Documentation update The documentation on [21]how to assemble the weekly reports has been updated. (Lunar) The example on how to use [22]SOURCE_DATE_EPOCH with CMake has been improved. (Ben Beockel, Daniel Kahn Gillmor) The solution for [23]timestamps in man pages generated by Sphinx now uses SOURCE_DATE_EPOCH. (Mattia Rizzolo) Package reviews 45 [24]reviews have been removed, 141 added and 62 updated this week. 67 new FTBFS reports have been filled by Chris Lamb, Niko Tyni, and Lisandro Damián Nicanor Pérez Meyer. New issues added this week: [25
[Reproducible-builds] package uploaded to our repo
diffoscope_36.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Adding a 'c' build to test with DEB_BUILD_OPTIONS=nocheck ?
Chris Lamb: > I just ran into yet another package where the contents can vary > depending on whether the tests are run or not. > > As an example, without tests a given Python package entirely vanilla and > is thus reproducible in our toolchain. However, executing the tests > creates various intermediary files that are genuinely useful (eg. > compiled versions of grammars, not .pyc files). These files are then > installed to the binary package. > > I'm only really discovering these when these files themselves are > unreproducible/non-deterministic, otherwise they are completely > invisible. > > So, does this matter to us? It's strictly more of a general QA issue if > we are declare that DEB_BUILD_OPTIONS does not contain nocheck from an > reproducibility PoV.. but on the other hand, our testing framework would > make this almost trivial to detect. > > (Why another build? Whilst adding `nocheck` to our current `b` build > could work, it would be a bad regression as I would dearly miss having > the tests run in an exotic locale and timezone, etc., hence proposing a > `c` build). My take on this: I want to wait until we can rebuild packages taken from directly the archive. We can easily run these later rebuilds with “nocheck”. That should enable us to spot these problems. What do you think? -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] [report] Week 22 in Stretch cycle
What happened in the [1]reproducible builds effort this past week: Toolchain fixes * Ben Hutchings uploaded linux-tools/4.2-1 which makes the tarball generated by genorig.py reproducible. Packages fixed The following 22 packages became reproducible due to changes in their build dependencies: breathe, cdi-api, geronimo-jpa-2.0-spec, geronimo-validation-1.0-spec, gradle-propdeps-plugin, jansi, javaparser, libjsr311-api-java, mac-widgets, mockito, mojarra, pastescript, plexus-utils2, powerline, python-psutil, python-sfml, python-tldap, pythondialog, tox, trident, truffle, zookeeper. The following packages became reproducible after getting fixed: * cloudprint/0.14-1 uploaded by David Steele, [2]original patch by Chris Lamb. * cpl-plugin-sinfo/2.6.5+dfsg-2 by Ole Streicher. * fonts-stix/1.1.1-4 uploaded by Hugo Lefeuvre, [3]original patch by Dhole. * gstreamermm-1.0/1.4.3+dfsg-5 by Philip Rinn. * hspell/1.2-3 uploaded by Tzafrir Cohen, [4]original patch by Reiner Herrmann. * libmodule-extractuse-perl/0.33-2 by gregor herrmann. * mariadb-10.0/10.0.21-1 by Otto Kekäläinen. * mkvtoolnix/8.4.0-1 uploaded by Christian Marillat, fixed upstream. * mlpack/1.0.12-5 by Barak A. Pearlmutter. * module-assistant/0.11.8 by Andreas Beckmann. * pitivi/0.94-4 uploaded by Sebastian Dröge, [5]reported by Scott Kitterman. * privoxy/3.0.23-4 by Roland Rosenfeld. * qtop/2.3.1-1 uploaded by Hugo Lefeuvre, fixed upstream. * seyon/2.20c-32 by Santiago Vila. * subvertpy/0.9.3-2 by Jelmer Vernooij. * twitter-bootstrap/2.0.2+dfsg-8 by Santiago Vila, [6]reported by Chris Lamb. * vdr-plugin-remote/0.7.0-1 by Tobias Grimm. Some uploads fixed some reproducibility issues but not all of them: * fldigi/3.23.01-1 by Kamal Mostafa. Patches submitted which have not made their way to the archive yet: * 799871 on console-data by Chris Lamb: grep all keymap files as text. * 87 on anarchism by Holger Levsen: use C locale when converting HTML to text files. * 800107 on dutch by Chris Lamb: grep wordlist files as text. diffoscope development The changes to make [7]diffoscope run under Python 3, along with many small fixes, entered the archive with version 35 on September 21th. Another release was made the very next day fixed two encoding-related issues discovered when running diffoscope on more Debian packages. strip-nondeterminism development Version 0.12.0 now [8]preserves file permissions on modified zip files and dh_strip_nondeterminism has been made [9]compatible with older debhelper. disorderfs development Version 0.3.0 implemented a “multi-user” mode that was required to build Debian packages using disorderfs. It also added command line options to control the ordering of files in directory (either shuffled or reversed) and another to do arbitrary changes to the reported space used by files on disk. A couple days later, version 0.4.0 was released to support locks, flush, fsync, fsyncdir, read_buf, and write_buf. Almost all known issues have now been fixed. reproducible.debian.net disorderfs is now used during the second build. This makes file ordering issue very easy to identify as such. (h01ger) Work has been done on making the distributed build setup more reliable. (h01ger) Documentation update Matt Kraii fixed the example on [10]how to fix issues related to dates in Sphinx. Recent Sphinx versions should also be compatible with [11]SOURCE_DATE_EPOCH. Package reviews 53 [12]reviews have been removed, 85 added and 13 updated this week. 46 packages failing to build from source has been identified by Chris Lamb, Chris West, and Niko Tyni. Chris Lamb was the lucky reporter of bug #80 on vdr-plugin-prefermenu. Issues related to disorderfs are being [13]tracked with a new issue. References 1. https://wiki.debian.org/ReproducibleBuilds 2. https://bugs.debian.org/797505 3. https://bugs.debian.org/792602 4. https://bugs.debian.org/796275 5. https://bugs.debian.org/799677 6. https://bugs.debian.org/776299 7. http://diffoscope.org/ 8. https://bugs.debian.org/798384 9. https://bugs.debian.org/799732 10. https://wiki.debian.org/ReproducibleBuilds/TimestampsInManpagesGeneratedBySphinx 11. https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal 12. https://reproducible.debian.net/unstable/amd64/index_notes.html 13. https://reproducible.debian.net/issues/unstable/ftbfs_due_to_disorderfs_issue.html signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] dpkg 1.18.3.0~reproducible1 is broken (?)
Santiago Vila: > Using > > deb http://reproducible.alioth.debian.org/debian/ ./ > > even the most simple package created makes lintian to complain: > […] I will not be able to look into this at least before the week-end. So please beat me to it. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] CI testbuilds LC_ALL vs. LANG
Andreas Metzler: > If we are aiming for locale agnostic builds we seem to take a > unnecessary work intensive approach by adding > export LC_ALL=C > to dozens (hundreds?) of debian/rules or changing > -sort > +env LC_ALL= sort > instead of simply patching dpkg-buildpackage. Using `dpkg-buildpackage` is not a requirement to build Debian packages. If using `fakeroot debian/rules binary` starts to give different results than running `dpkg-buildpackage`, some maintainers are likely to have bad surprises. Pushing for such a change is much larger than the scope of the reproducible builds effort. Feel free to work on it, though! -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] [report] Week 23 in Stretch cycle
What happened in the [1]reproducible builds effort this past week: Toolchain fixes Barry Warsaw uploaded wheel/0.26.0-1 which now uses [2]SOURCE_DATE_EPOCH instead of WHEEL_FORCE_TIMESTAMP and [3]uses time.gmtime() to avoid timezone issues. Patches by Chris Lamb and Reiner Herrmann. Andreas Metzler uploaded autogen/1:5.18.6-1 in experimental with [4]several patches for reproducibility issues written by Valentin Lorentz. Groovy upstream has merged a [5]change proposed by Emmanuel Bourg to remove timestamps generated by groovydoc. Ben Hutchings submitted a patch to [6]add support for SOURCE_DATE_EPOCH in linux-kbuild as an alternate way to specify the build timestamp. Reiner Herrman has sent a [7]patch adding support for [8]SOURCE_DATE_EPOCH in docbook-utils. Packages fixed The following packages became reproducible due to changes in their build dependencies: commons-csv. fest-reflect, sunxi-tools, xfce4-terminal, The following packages became reproducible after getting fixed: * httpcomponents-client/4.5.1-1 by Emmanuel Bourg. * jhead/1:3.00-2 by Ludovic Rousseau. * libvigraimpex/1.10.0+dfsg-10 by Daniel Stender. * linux/4.2-1~exp1 by Ben Hutchings. * maelstrom/1.4.3-L3.0.6+main-7 by Santiago Vila. * nedit/1:5.6a-3 by Paul Gevers. * pitivi/0.94-4 by Sebastian Dröge, [9]reported by Scott Kitterman. * procenv/0.40-2 by James Hunt. * seyon/2.20c-32 by Santiago Vila. * slib/3b1-5 by Santiago Vila. * spock/0.7-groovy-2.0-1 by Emmanuel Bourg. * u-boot/2015.10~rc4+dfsg1-1 by Vagrant Cascadian. * vdr-plugin-remote/0.7.0-1 by Tobias Grimm. Some uploads fixed some reproducibility issues but not all of them: * dutch/1:2.10-4 uploaded by Thijs Kinkhorst, [10]original patch by Chris Lamb. Patches submitted which have not made their way to the archive yet: * 800776 on cluster-glue: exports SOURCE_DATE_EPOCH in debian/rules. Tomasz Rybak uploaded pycuda/2015.1.3-1 which should fix reproducibility issues. The package has not been tested as it is in contrib. akira found an [11]embedded code copy of texi2html in fftw. reproducible.debian.net Email notifications are now only sent once a day per package, instead of on each status change. (h01ger) disorderfs has been temporarily disabled to see if it had any impact on the disk space issues. (h01ger) When running out of disk space, build nodes will now automatically detect the problem. This means test results will not be recorded as “FTBFS” and the problem will be reported to Jenkins maintainers. (h01ger) The navigation menu of package pages has been improved. (h01ger) The two amd64 builders now use two different kernel versions: 3.16 from stable and 4.1 from backports on the other. (h01ger) We now [12]graph the number of packages which needs to be fixed. (h01ger) Munin now creates graphs on how many builds were performed by build nodes ([13]example). (h01ger) A migration plan has been agreed with [14]DSA on how to turn Jenkins into an official Debian service. A backport of jenkins-job-builder for Jessie is currently missing. (h01ger) Package reviews 119 [15]reviews have been removed, 103 added and 45 updated this week. 16 “fail to build from source” issues were reported by Chris Lamb and Mattia Rizzolo. New issue this week: [16]timestamps_in_manpages_generated_by_docbook_utils. Misc. Allan McRae has submitted a patch to make [17]ArchLinux pacman [18]record a .BUILDINFO file. References 1. https://wiki.debian.org/ReproducibleBuilds 2. https://bugs.debian.org/795942 3. https://bugs.debian.org/789981 4. https://bugs.debian.org/794892 5. https://github.com/apache/incubator-groovy/pull/121 6. http://www.spinics.net/lists/linux-kbuild/msg11728.html 7. https://bugs.debian.org/800797 8. https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal 9. https://bugs.debian.org/799677 10. https://bugs.debian.org/800107 11. https://bugs.debian.org/800653 12. https://reproducible.debian.net/unstable/amd64/stats_pkgs_to_fix.png 13. https://jenkins.debian.net/munin/debian.net/profitbricks-build1-amd64.debian.net/jenkins_reproducible_builds.html 14. https://dsa.debian.org/ 15. https://reproducible.debian.net/unstable/amd64/index_notes.html 16. https://reproducible.debian.net/issues/unstable/timestamps_in_manpages_generated_by_docbook_utils_issue.html 17. https://wiki.archlinux.org/index.php/Pacman 18. https://lists.archlinux.org/pipermail/pacman-dev/2015-October/020357.html signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] package uploaded to our repo
dpkg_1.18.3.0~reproducible2.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#801333: retitle
retitle 801333 diffoscope: UnicodeDecodeError with
haskell-authenticate-oauth/1.5.1.1-4
clone 801333 -1
retitle -1 diffoscope: UnicodeDecodeError in test_text_option_with_file
severity -1 minor
thanks
Holger Levsen:
> someone just reported the same problem on irc: […]
It was not the same problem. With a quick look at the stack trace,
you can see that the error is raised in a different location of the
code. Just like when you see “Segmentation fault”, it's a bad idea to
assume that it's due to the same bug.
> ==
> FAILURES
> ==
> _
> test_text_option_with_file
> _
>
> tmpdir = local('/tmp/pytest-bnewbold/pytest-4/test_text_option_with_file0'),
> capsys = <_pytest.capture.CaptureFixture object at 0x7f643f784c18>
>
> def test_text_option_with_file(tmpdir, capsys):
> report_path = str(tmpdir.join('report.txt'))
> args = ['--text', report_path, TEST_TAR1_PATH, TEST_TAR2_PATH]
> with pytest.raises(SystemExit) as excinfo:
> main(args)
> assert excinfo.value.code == 1
> out, err = capsys.readouterr()
> assert err == ''
> assert out == ''
> with open(report_path, 'r') as f:
> > assert f.read().startswith('--- ')
>
> tests/test_main.py:105:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> self =
> input = b'--- /home/bnewbold/diffoscope/tests/data/test1.tar\n+++
> /home/bnewbold/diffoscope/tests/data/test2.tar\n\xe2\x94\x9c...n the Ends of
> Goods and Evils, or alternatively [About]\n\xe2\x94\x82 +The
> Purposes of Good and Evil).\n\xe2\x95\xb5\n'
> final = True
>
> def decode(self, input, final=False):
> > return codecs.ascii_decode(input, self.errors)[0]
> E UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
> 102: ordinal not in range(128)
>
> /usr/lib/python3.4/encodings/ascii.py:26: UnicodeDecodeError
>
> 1 failed, 11 passed in 0.61 seconds
> =
>
>
> [10:40] < bnewbold> this is with the diffoscope 36 package installed, running
> the tests in a git checkout
> [10:42] < bnewbold> this is in a just-created 'sid' lxc container created on
> a debian jessie laptop (created with lxc-create)
The test suite assumes to be run under a UTF-8 locale. That needs to be
either enforced or fixed.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#801855: ongl: test strings depend on default character encoding of the build system
Source: ognl Version: 2.7.3-6 Severity: minor User: reproducible-builds@lists.alioth.debian.org Usertags: locale Hi! It seems that depending on the build system default character encoding, the non-ASCII characters in org/ognl/test/QuotingTest.java might get mistranslated. This also prevents ongl from building reproducibly when one locale is ISO-8859-15 and the other is UTF-8. See: https://reproducible.debian.net/dbd/unstable/amd64/ognl_2.7.3-6.debbindiff.html (also supplied as attachment) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- ognl_2.7.3-6.debbindiff.html.xz Description: Binary data signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Notes on OpenWRT Reproducibility
bnewb...@robocracy.org:
> 1. There's a statement that diffoscope "cannot detect .bin files as
> squashfs". I don't think that's accurate, I could rename .squashfs files to
> .bin and diffoscope found the magic and worked fine.
diffoscope currently recognizes Squashfs images using file(1). The
extension should not matter.
> I think the issue is the huge number of hardware-specific ".bin" and
> ".img" formats OpenWRT generates[1]. Many of these are in uImage
> format (detectable by magic), others are just {kernel, bootloader, fs}
> at fixed offsets. For the former it would be great if diffoscope could
> pick up on the format in any way. I found a random script[2] that
> tries to extract uImage binaries into the original files, but it would
> be a good start to just present the metadata and offsets.
The easiest way is probably to add a new “container-style” type of file
in diffoscope. If you don't feel comfortable doing it yourself by
looking at the existing code I can probably do it, but I would welcome
good pointers and test data. Feel free to open a new wishlist bug for
this.
> 3. I was sort of surprised to see linux-latest passing reproducibility in
> Debian on armhf. Was anything special necessary for this, eg changes to
> mkimage to generate uImage without a timestamp? Any advice or HOWTO out
> there on reproducible kernel builds? I see Ben Hutchings's kbuild patch for
> SOURCE_DATE_EPOCH support; is that sufficient?
linux-latest is just a meta package to instal the latest kernel version.
The source package for the kernel is called “linux”. It has not been
tested on armhf yet:
https://reproducible.debian.net/rb-pkg/unstable/armhf/linux.html
> 5. diffoscope has squashfs support, but is there actually a way to generate
> squashfs files reproducibly? Perhaps there is a way to strip the metadata
> after the fact?
It's most probable that changes in mksquashfs itself will be required.
Thanks for your work! :)
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] On expiring packages and issues
Holger Levsen: > On Freitag, 16. Oktober 2015, Santiago Vila wrote: > > Hi. Do we expire issues (not packages) when no package currently have them? > > https://reproducible.debian.net/index_issues.html -> scroll down to the > bottom. > > > (I would prefer to keep them). > > it depends, i'd say. they also stay in git history. Packages we definitly > remove once they are fixed or removed from sid. I would say that it's better to keep any issue which other free software projects might bump into. I know it's far from a clear guideline. In any cases, we have the history, as Holger said. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] On expiring packages and issues
Santiago Vila: > On Fri, Oct 16, 2015 at 01:40:22PM +0200, Jérémy Bobbio wrote: > > I would say that it's better to keep any issue which other free software > > projects might bump into. > > Agreed. > > > I know it's far from a clear guideline. In any > > cases, we have the history, as Holger said. > > I'd like to avoid having to recover an old issue from git history, if > possible. > > So I propose this simple guideline to remove issues: Do it only when > they were due to bugs in our CI. I think it's sound to also remove issues due to Debian-specific toolchain once it has been fixed. I don't think there's much value in keeping tabs on something like ordering problems in debhelper the required `sort` has been added. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] On expiring packages and issues
Santiago Vila: > On Fri, Oct 16, 2015 at 03:22:34PM +0200, Jérémy Bobbio wrote: > > I think it's sound to also remove issues due to Debian-specific > > toolchain once it has been fixed. I don't think there's much value in > > keeping tabs on something like ordering problems in debhelper the > > required `sort` has been added. > > What do you mean by "Debian-specific"? We have lots of derived distros. > In case they are interested in their own reproducibility, it would be nice > for them that we do not expire issues so happily. Good point. > We are not even tracking anywhere which version fixed the different > issues. I believe this is done in the monthly reports. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Weird localedef failures
Hi! Esa, thanks for looking into these problems. Esa Peuha: > Santiago Vila: > > For the same reason (not being build-essential), it is ok that it's > > not pre-installed in pbuilder chroot, and in fact, usually it's not. > > Huh? How is the second build supposed to be able to set the locale to > fr_CH without the locales package? We install the locales-all package which contains all locales already pre-built, and `Provides: locales`. > No, that is not the problem. If you look at > > https://reproducible.debian.net/rbuild/unstable/amd64/apertium-en-ca_0.9.3~r61232-1.rbuild.log > > you can see that locales is in the Build-Depends list, but pbuilder > doesn't try to install it, so either there is something wrong with > the installed packages database (which would be a bug in our pbuilder > setup) or locales is installed but some of its files are somehow > inaccessible (which would also be a bug in our pbuilder setup). We already had an issue where locales-all was not providing everything that locales would provide. See #788352. I guess the fact that locales-all doesn't ship localedef can be considered as a bug. But the whole point of locales-all is that you don't need to build anything. In the present case, the fact that apertium locale packages ships compiled locale definition files feels like a hack. I wonder if there's a better solution. I also wonder if compiled locale definitions works accross different versions of the glibc or if there's potential upgrade issues hiding here. (I must admit that I am also puzzled at the idea that there's a source package *per locale*, that must be a hell of a burden to update and keep in sync.) So, probably there's a minor bug to fill against locales-all about localedef. A quick work-around for apertium locale packages would be to check if the system already has a compiled locale definition and copy it instead of always using localedef. But I think it would be worthwhile to revisit how apertium locale packages are built and work. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Weird localedef failures
Lunar: > We already had an issue where locales-all was not providing everything > that locales would provide. See #788352. > > I guess the fact that locales-all doesn't ship localedef can be > considered as a bug. But the whole point of locales-all is that you > don't need to build anything. > […] > So, probably there's a minor bug to fill against locales-all about > localedef. I just noticed that this very issue might be already fixed in experimental. It would be worthwhile work to verify the situation for apertium locale packages. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] [report] Week 25 in Stretch cycle
What happened in the [1]reproducible builds effort this week: Toolchain fixes * Scott Kitterman uploaded python3-defaults/3.4.3-7 which changes py3versions to list versions in a consistent order. Issue [2]reported by Santiago Vila with a tentative patch by Chris Lamb. Sadly, it appears the problem is not entirely solved. * Martin Pitt uploaded init-system-helpers/1.24 which makes the order of unit files in maintainer scripts stable. [3]Original patch by Reiner Herrmann. * Niko Tyni uploaded libextutils-xsbuilder-perl/0.28-3 which [4]makes the generated XS code reproducible. Niko Tyni wrote a new [5]patch adding support for [6]SOURCE_DATE_EPOCH in Pod::Man. This would complement or replace the previously implemented POD_MAN_DATE environment variable in a more generic way. Niko Tyni [7]proposed a fix to prevent [8]mtime variation in directories due to debhelper usage of cp --parents -p. Packages fixed The following 119 packages became reproducible due to changes in their build dependencies: aac-tactics, aafigure, apgdiff, bin-prot, boxbackup, calendar, camlmix, cconv, cdist, cl-asdf, cli-common, cluster-glue, cppo, cvs, esdl, ess, faucc, fauhdlc, fbcat, flex-old, freetennis, ftgl, gap, ghc, git-cola, globus-authz-callout-error, globus-authz, globus-callout, globus-common, globus-ftp-client, globus-ftp-control, globus-gass-cache, globus-gass-copy, globus-gass-transfer, globus-gram-client, globus-gram-job-manager-callout-error, globus-gram-protocol, globus-gridmap-callout-error, globus-gsi-callback, globus-gsi-cert-utils, globus-gsi-credential, globus-gsi-openssl-error, globus-gsi-proxy-core, globus-gsi-proxy-ssl, globus-gsi-sysconfig, globus-gss-assist, globus-gssapi-error, globus-gssapi-gsi, globus-net-manager, globus-openssl-module, globus-rsl, globus-scheduler-event-generator, globus-xio-gridftp-driver, globus-xio-gsi-driver, globus-xio, gnome-control-center, grml2usb, grub, guilt, hgview, htmlcxx, hwloc, imms, kde-l10n, keystone, kimwitu++, kimwitu-doc, kmod, krb5, laby, ledger, libcrypto++, libopendbx, libsyncml, libwps, lprng-doc, madwimax, maria, mediawiki-math, menhir, misery, monotone-viz, morse, mpfr4, obus, ocaml-csv, ocaml-reins, ocamldsort, ocp-indent, openscenegraph, opensp, optcomp, opus, otags, pa-bench, pa-ounit, pa-test, parmap, pcaputils, perl-cross-debian, prooftree, pyfits, pywavelets, pywbem, rpy, signify, siscone, swtchart, tipa, typerep, tyxml, unison2.32.52, unison2.40.102, unison, uuidm, variantslib, zipios++, zlibc, zope-maildrophost. The following packages became reproducible after getting fixed: * autoconf2.13/2.13-67 uploaded by Ben Pfaff, [9]original patch by Santiago Vila. * ding/1.8-3 by Roland Rosenfeld. * dropbear/2015.68-1 by Guilhem Moulin. First set of patches (#777324, #793006) by Chris Lamb and akira. * nvram-wakeup/1.1-3 by Tobias Grimm. * original-awk/2012-12-20-5 by Santiago Vila. * resiprocate/1:1.10.0-1 uploaded by Daniel Pocock, [10]patch by Reiner Herrmann merged upstream. * sbuild/0.66.0-5 by Johannes Schauer, [11]reported by Jakub Wilk. * scribus/1.4.5+dfsg1-4 by Mattia Rizzolo. * sgmltools-lite/3.0.3.0.cvs.20010909-19 by Santiago Vila. * unicode-data/8.0-2 uploaded by Alastair McKinstry, [12]original patch by Esa Peuha. * xmoto/0.5.11+dfsg-3 by Stephen Kitt. Packages which could not be tested: * mp4h/1.3.1-11 by Axel Beckert (shared memory issue on reproducible.debian.net). * nvidia-graphics-drivers-legacy-304xx/304.128-5 by Andreas Beckmann (non-free). Some uploads fixed some reproducibility issues but not all of them: * libvirt/1.2.20-1 by Guido Günther. * libgpars-groovy-java/1.2.1-4 by Emmanuel Bourg. Patches submitted which have not made their way to the archive yet: * 801520 on libapache2-mod-perl2 by Niko Tyni: sort the list of files used to build the documentation. * 801523 on perl by Niko Tyni: sort the list of input files in PPPort_xs.PL. [13]Forwarded upstream. * 801864 on ncurses by Esa Peuha: use C locale when sorting the list of keys. * 802042 on libchado-perl by Niko Tyni: sort keys in installed configuration file. Lunar reported that [14]test strings depend on default character encoding of the build system in ongl. reproducible.debian.net The 189 packages composing the Arch Linux “core” repository are [15]now being tested. No packages are currently reproducible, but most of the time the difference is limited to metadata. This has already gained some interest in the Arch Linux community. An explicit log message is now visible when a build has been killed due to the 12 hours timeout. (h01ger) Remote build setup has been made more robust and self m
[Reproducible-builds] [report] Week 26 in Stretch cycle
What happened in the [1]reproducible builds effort this past week: Toolchain fixes * Stefano Rivera uploaded python-cffi/1.3.0-1 which makes the generated code order deterministic for anonymous unions and anonymous structs. [2]Reported by Tristan Seligmann, and [3]fixed uptream. Mattia Rizzolo created a [4]bug report to continue the discussion on storing cryptographic checksums of the installed .deb in dpkg database. This follows the [5]discussion that happened in June and is a pre-requisite to add checksums to [6].buildinfo files. Niko Tyni [7]identified why the Vala compiler would generate code in varying order. A better patch than his initial attempt still needs to be written. Packages fixed The following 15 packages became reproducible due to changes in their build dependencies: alt-ergo, approx, bin-prot, caml2html, coinst, dokujclient, libapreq2, mwparserfromhell, ocsigenserver, python-cryptography, python-watchdog, slurm-llnl, tyxml, unison2.40.102, yojson. The following packages became reproducible after getting fixed: * 389-ds-base/1.3.3.13-1 uploaded by Timo Aaltonen, [8]original patch by Chris Lamb. * apache2/2.4.17-1 by Stefan Fritsch. * ben/0.7.3 by Mehdi Dogguy. * cdo/1.6.9+dfsg.1-3 by Alastair McKinstry. * epubcheck/4.0.0-2 by Eugene Zhukov. * grads/2:2.0.2-8 by Alastair McKinstry. * litl/0.1.7+dfsg-1 uploaded by Samuel Thibault, [9]original patch by Chris Lamb. * mia/2.2.5-1 by Gert Wollny. * powerline/2.2-2 by Jerome Charaoui. * python-oslotest/1:1.11.0-2 by Thomas Goirand. * tth/4.05+ds-2 uploaded by Jerome Benoit, [10]original patch by Reiner Herrmann. * xbae/4.60.4-7 uploaded by Nicholas Breen, [11]original patch by Chris Lamb. * xdmf/2.1.dfsg.1-13 by Alastair McKinstry. Some uploads fixed some reproducibility issues but not all of them: * foxeye/0.10.2-1 by Andriy Grytsenko. * jaxe/3.5-6 by Samuel Thibault. * ncurses/6.0+20151017-1 by Sven Joachim, [12]original patch by Esa Peuha. * olap4j/1.2.0-1 by Emmanuel Bourg. * tomcat8/8.0.28-1 by Emmanuel Bourg. reproducible.debian.net pbuilder has been updated to version 0.219~bpo8+1 on all eight build nodes. (Mattia Rizzolo, h01ger) Packages that FTBFS but for which no open bugs have been recorded are now tested again after 3 days. Likewise for “depwait” packages. (h01ger) Out of disk situations will not cause IRC notifications anymore. (h01ger) Documentation update Lunar continued to work on writing documentation for the future reproducible-builds.org website. Package reviews 44 [13]reviews have been removed, 81 added and 48 updated this week. Chris West and Chris Lamb identified 70 “fail to build from source” issues. Misc. h01ger presented the project in Mexico City at the [14]3er Congreso de Seguridad de la Información. It seems there is an interest for academic papers related to reproducible builds. Bryan has been doing hard work to improve reproducibility for [15]OpenWrt. He wrote a [16]report linking to the [17]patches and [18]test results he published. References 1. https://wiki.debian.org/ReproducibleBuilds 2. https://bugs.debian.org/799278 3. https://bitbucket.org/cffi/cffi/commits/1cfe8c7a59e88186f1a07a9dde28e1787fd900d0 4. https://bugs.debian.org/802241 5. https://lists.debian.org/debian-dpkg/2015/06/msg00011.html 6. https://wiki.debian.org/ReproducibleBuilds/BuildinfoSpecification 7. https://bugs.debian.org/802528 8. https://bugs.debian.org/799010 9. https://bugs.debian.org/797508 10. https://bugs.debian.org/792181 11. https://bugs.debian.org/797871 12. https://bugs.debian.org/801864 13. https://reproducible.debian.net/unstable/amd64/index_notes.html 14. http://congresoseguridad.org/index.php?page=programa 15. https://www.openwrt.org/ 16. https://lists.openwrt.org/pipermail/openwrt-devel/2015-October/036688.html 17. https://github.com/bnewbold/openwrt-repro/compare/master...repro 18. http://repro.bnewbold.the-nsa.org/openwrt-results/ signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] [Pkg-octave-devel] Trying to fix random_buildpath_by_octave
Rafael Laboissiere: > I think that I found the culprit. The .oct files for octave are built using > the mkoctfile program, as you pointed out above. This program uses default > values for the the environment variables containing flags for the to C++ > compilation and binary link. For instance: > > $ mkdir tmp-1 > $ cp /usr/share/doc/liboctave-dev/examples/oregonator.cc tmp-1 > $ (cd tmp-1 ; mkoctfile -v oregonator.cc) > g++ -c -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/octave-4.0.0/octave/.. > -I/usr/include/octave-4.0.0/octave -I/usr/include/hdf5/serial > -I/usr/include/mpi -pthread -fopenmp -mieee-fp -g -O2 > -fstack-protector-strong -Wformat -Werror=format-security oregonator.cc -o > oregonator.o > g++ -shared -Wl,-Bsymbolic -o oregonator.oct oregonator.o > -L/usr/lib/i386-linux-gnu/octave/4.0.0 -L/usr/lib/i386-linux-gnu -loctinterp > -loctave -Wl,-z,relro > > I discovered that the "-g" is responsible for the difffrences in the > resulting binaries. Here is the proof: > > $ (cd ../tmp-1 ; CXXFLAGS="-g -O2 -fstack-protector-strong -Wformat > -Werror=format-security" mkoctfile oregonator.cc ; md5sum oregonator.o*) > e10495ec145b6c3f946c1728fa64afd2 oregonator.o > 641e6c9a7ee23b38a4452e6337f8a6b5 oregonator.oct Is `mkoctfile` creating a randomly-named temporary file? g++ will write the full path to the source file in the debug symbols. That would explain why removing `-g` solve the problem. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] [Pkg-octave-devel] Trying to fix random_buildpath_by_octave
Sébastien Villemot: > My understanding is that .oct files are stripped before creating the > Debian package (by dh_strip). Hence they no longer contain debugging > symbol. So removing -g at compilation time should make no difference. The build id is generated by ld using a checksum of the whole binary before it is being stripped. (The build id is used to tie a binary to its debug symbols, so that's exactly what we want.) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] binNMU detection for generating Changes field in buildinfo (was: Re: binNMU or reproducible builds (choose only one))
Johannes Schauer:
> Lunar:
> > Simon McVittie:
> > > BinNMUs don't upload any source at all. They instruct the autobuilders
> > > to run sbuild with some non-default options ("sbuild --binNMU=2
> > > --make-binNMU "Rebuild with foo 3" foo_1.2-3" will result in
> > > foo_1.2-3+b2_i386.changes, I think), and sbuild on each autobuilder
> > > downloads the foo_1.2-3.dsc that already exists in the archive.
> > >
> > > The only inherent conflict that I can see between binNMUs and
> > > reproducible builds is that all attempts to reproduce the original build
> > > need to prepend the same changelog entry as the original build, for
> > > example by copying them from the build info that will already be
> > > necessary to be able to use the same build-dependency versions.
> >
> > This is already taken in account in the current `.buildinfo`
> > specification [1]:
> >
> > Changes
> >
> > Close to the one in `*.changes`. When source and binary versions
> > differ, the field is added with the content of the extra changelog
> > entries.
> >
> > The field is already created by the experimental `dpkg-genbuildinfo` [2].
> > It is not yet implement in the `srebuild` script but it shouldn't be too
> > hard.
> >
> > [1]:
> > https://wiki.debian.org/ReproducibleBuilds/BuildinfoSpecification#buildinfo_field_descriptions
> > [2]:
> > https://anonscm.debian.org/cgit/reproducible/dpkg.git/tree/scripts/dpkg-genbuildinfo.pl?h=pu/reproducible_builds&id=c4665b80d7a042216145652ea3d1259b78ac6237#n246
>
> I don't think this is the right way to do it. Looking at the dpkg-genbuildinfo
> script, it seems that this behaviour is triggered whenever the source version
> differs from the binary version. But binNMUs are not the only case for which
> the source and binary versions differ. To convince yourself you can run the
> following (get all packages that are not binNMUs (detected by them having a bX
> at the end of their version) and which have a versioned Source field):
>
> grep-dctrl --not --field Version --eregex '\+b[0-9]+$' --and \
> -F Source --eregex '\(' \
>
> /var/lib/apt/lists/http.debian.net_debian_dists_sid_main_binary-amd64_Packages
> \
> -n -s Package,Source,Version
>
> It shows that there are several binary packages that have a different source
> package version even though they are not binNMUs. The version of any binary
> package can be done during the build with dh_gencontrol or directly via
> `dpkg-gencontrol -v$version`.
Thanks for the explaination.
> The right way to find out whether a binNMU is done is to check
> $changelog->{'Binary-Only'} as it is done in dpkg-genchanges. During a binNMU,
> sbuild sets binary-only=yes in the top changelog entry.
I've changed the `if` to use $changelog->{'Binary-Only'}, but it was
already working as intended. The code read as follow:
my $sourceversion = $changelog->{'Binary-Only'} ?
$prev_changelog->{'Version'} : $changelog->{'Version'};
my $binaryversion = $changelog->{'Version'};
[…]
if ($binaryversion ne $sourceversion) { # handle binNMU
> In fact, what you call $binaryversion in that script is really the source
> version because your $binaryversion comes from d/changelog. So maybe you want
> to rename that variable. That also then remove the ambiguity from a later part
> in the code when you set the Version field in the buildinfo to $binaryversion.
> This is ambiguous because a source package can build binary packages of
> several
> different versions.
The naming is coming straight from dpkg-genchanges, so both should be
changed if you fill they are ill-named.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] package uploaded to our repo
dpkg_1.18.4~reproducible1.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Repro Build report I do not understand
Dirk Eddelbuettel:
> One of my packages which still didn't build reproduciby is littler -- for
> which I am upstream. I rewrote the build process, and even though it produces
> a small binary (which embeds R for use in #! scripts etc) it now ships as an
> R package on CRAN. Which all build reproducibly.
>
> Yet I have this:
>https://reproducible.debian.net/rb-pkg/unstable/amd64/littler.html
>
> And I don't understand the 'dbgsym' part. What turns that on? How can I turn
> it off?
You might want to read the latest status update to learn about them:
https://lists.debian.org/debian-devel/2015/08/msg00443.html
But they are not the source of reproducibility, just a symptom.
From the quick look I had, it seems symbols are sorted in a different
order depending on the locale:
https://reproducible.debian.net/dbd/unstable/amd64/littler_0.3.0-2.debbindiff.html#r-cran-littler_0.3.0-2_amd64.deb/data.tar.xz/data.tar/./usr/bin/r/objdump%20--disassemble%20--full-contents%20{}
(It's my guess because both are sorted by the second build has lowercase
'a' grouped together with uppercase 'A'.)
The symbols match the ones in
https://sources.debian.net/src/r-base/3.2.2-1/src/library/datasets/data/
Hope that helps,
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Repro Build report I do not understand
Dirk Eddelbuettel:
> | From the quick look I had, it seems symbols are sorted in a different
> | order depending on the locale:
> |
> https://reproducible.debian.net/dbd/unstable/amd64/littler_0.3.0-2.debbindiff.html#r-cran-littler_0.3.0-2_amd64.deb/data.tar.xz/data.tar/./usr/bin/r/objdump%20--disassemble%20--full-contents%20{}
> | (It's my guess because both are sorted by the second build has lowercase
> | 'a' grouped together with uppercase 'A'.)
> |
> | The symbols match the ones in
> | https://sources.debian.net/src/r-base/3.2.2-1/src/library/datasets/data/
> |
> | Hope that helps,
>
> That is very good too. Should I ensure a locale during the build? Any other
> heavy hand?
You can try to set LC_ALL (or set LC_COLLATE and unset LC_ALL). But you
might want to identify where the sorting happens first.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] GCC patch reviewed. Proposed mail for gcc-patches mailing list
Santiago Vila: > I have very mixed feelings about this kind of patches. > > I fear that by modifying gcc to hide the improper usage of __DATE__ > and __TIME__, we could be removing an incentive for maintainers and > authors to write software which is truly reproducible, i.e. we run > the risk of people thinking in the line of "Oh, I will not care about > my program using __TIME__ or __DATE__ because gcc will take care of > that". Well, I still would like to push to add `-Wdate-time` to our default set of CFLAGS. Even with Dhole's patch, developpers would see the warning and ask themselves if it's the right thing to do. I think for this precise case, that's a good compromise. Also, I'm pretty sure that if the GCC team merges support for SOURCE_DATE_EPOCH, we could get a similar patch merged to LLVM. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] [R-B WEBSITE PATCH] native en_US speaker website tweaks
Daniel Kahn Gillmor: > index.html | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) Thanks, it's now live! :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] [report] Week 29 in Stretch cycle
What happened in the [1]reproducible builds effort this past week: Toolchain fixes Emmanuel Bourg uploaded eigenbase-resgen/1.3.0.13768-2 which uses of the scm-safe comment style by default to make them deterministic. Mattia Rizzolo started a [2]new thread on debian-devel to ask a wider audience for issues about the -Wdate-time compile time flag. When enabled, GCC and clang print warnings when __DATE__, __TIME__, or __TIMESTAMP__ are used. Having the [3]flag set by default would prompt maintainers to remove these source of unreproducibility from the sources. Packages fixed The following packages have become reproducible due to changes in their build dependencies: bmake, cyrus-imapd-2.4, drobo-utils, eigenbase-farrago, fhist, fstrcmp, git-dpm, intercal, libexplain, libtemplates-parser, mcl, openimageio, pcal, powstatd, ruby-aggregate, ruby-archive-tar-minitar, ruby-bert, ruby-dbd-odbc, ruby-dbd-pg, ruby-extendmatrix, ruby-rack-mobile-detect, ruby-remcached, ruby-stomp, ruby-test-declarative, ruby-wirble, vtprint. The following packages became reproducible after getting fixed: * bino/1.6.1-1 by Daniel Schaal. * debconf/1.5.58 by Colin Watson. * fte/0.50.2b6-8 uploaded by Axel Beckert, [4]original patch by Reiner Herrmann. * gnome-desktop3/3.18.2-1 uploaded by Michael Biebl, fix by Laurent Bigonville based on Andreas Henriksson's suggestion. * libxmu/2:1.1.2-2 uploaded by Julien Cristau, fix by Andreas Boll. * megaglest/3.11.1-3 by Markus Koschany. * p4vasp/0.3.29+dfsg-2 uploaded by Graham Inggs, [5]original patch by Reiner Herrmann. * pd-iemmatrix/0.3-1 by IOhannes m zmölnig. * performous/1.0+git150721-2 by Markus Koschany. * python-sqlalchemy-utils/0.30.12-2 uploaded by Thomas Goirand, [6]original patch by Chris Lamb. * pywavelets/0.3.0-1 uploaded by Daniele Tricoli, [7]original patch by Juan Picca. * torus-trooper/0.22.dfsg1-10 uploaded by Markus Koschany, [8]original patch by Reiner Herrmann. * tycho2/2-3 by Ole Streicher. * xenomai/2.6.4+dfsg-0.1 by Leopold Palomo-Avellaneda. * xfonts-mona/2.90-8 by Nobuhiro Iwamatsu. Some uploads fixed some reproducibility issues, but not all of them: * apache-mime4j/0.7.2-4 by Emmanuel Bourg. * ispell-lt/1.2.1-5 by Chris Lamb (#778263. * xscavenger/1.4.5-1 uploaded by Hwei Sheng Teoh, [9]original patch by Chris Lamb. Patches submitted which have not made their way to the archive yet: * 804729 on pbuilder by Reiner Herrmann: tell dblatex to build in a deterministic path. reproducible.debian.net The fifth and sixth armhf build nodes have been set up, resulting in five more builder jobs for armhf. More than 10,000 packages have now been identified as reproducible with the [10]“reproducible” toolchain on armhf. (Vagrant Cascadian, h01ger) Helmut Grohne and Mattia Rizzolo now have root access on all 12 build nodes used by [11]reproducible.debian.net and [12]jenkins.debian.net. (h01ger) [13]reproducible-builds.org is now linked from all package pages and the reproducible.debian.net dashboard. (h01ger) profitbricks-build5-amd64 and profitbricks-build6-amd64, responsible for running amd64 tests now run 398.26 days in the future. This means that one of the two builds that are being compared will be run on a different minute, hour, day, month, and year. This is not yet the case for armhf. FreeBSD tests are also done with 398.26 days difference. (h01ger) The design of the [14]Arch Linux test page has been greatly improved. (Levente Polyak) diffoscope development Three releases of diffoscope happened this week numbered 39 to 41. It includes support for EPUB files (Reiner Herrmann) and Free Pascal unit files, usually having .ppu as extension (Paul Gevers). The rest of the changes were mostly targetting at making it easier to run diffoscope on other systems. The tlsh, rpm, and debian modules are now all optional. The test suite will properly skip tests that need optional tools or modules when they are not available. As a result, diffosope is [15]now available on PyPI and thanks to the work of Levente Polyak [16]in Arch Linux. Getting these versions in Debian was a bit cumbersome. Version 39 was uploaded with an expired key (according to the keyring on ftp.debian.org which will hopefully be updated soon) which is currently handled by keeping the files in the queue [17]without REJECTing them. This prevented any other Debian Developpers to upload the same version. Version 40 was uploaded as a source-only upload… but failed to build from source which had the [18]undesirable side effect of removing the previous version from unstable. The package faild to build from source because it was built passing -I to debbuild. This excl
[Reproducible-builds] Bug#806149: diffoscope TypeError processing openjfx/8u60-b27-4
Control: merge 805774 806149 Control: tag 805774 pending Mattia Rizzolo: > Seen on rb.d.n with openjfx/8u60-b27-4 on testing (trying it on unstable > resulted with a FTBFS of the package...) Same problem as #805774. Fixed in Git. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#806321: coreutils: please make the build reproducible (timestamps)
Source: coreutils Version: 8.23-4 Severity: wishlist Tags: patch User: reproducible-builds@lists.alioth.debian.org Usertags: timestamps Hi! While working on the “reproducible builds” effort [1], we have noticed that coreutils could not be built reproducibly. The build process uses help2man to create some of manpages. help2man author added support for the SOURCE_DATE_EPOCH environment variable [2] in version 1.47.1 which makes it possible to use a pre-defined value instead of the time of the build. As coreutils source currently contains an old version of help2man, the attached patch will copy the version in the help2man Debian package before running ./configure on top of setting SOURCE_DATE_EPOCH to the date of the latest debian/changelog entry. Once applied, coreutils can be built reproducibly in our current experimental framework. [1]: https://wiki.debian.org/ReproducibleBuilds [2]: https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- diff -u coreutils-8.23/debian/changelog coreutils-8.23/debian/changelog --- coreutils-8.23/debian/changelog +++ coreutils-8.23/debian/changelog @@ -1,3 +1,14 @@ +coreutils (8.23-4.0~reproducible1) UNRELEASED; urgency=low + + * Non-maintainer upload. + * Make the build reproducible: +- Replace in-tree help2man by the one in the help2man package. Add + the required Build-Depends. The in-tree help2man doesn't support + SOURCE_DATE_EPOCH. +- Set SOURCE_DATE_EPOCH to the date of the latest debian/changelog entry. + + -- Jérémy Bobbio Thu, 26 Nov 2015 12:43:39 +0100 + coreutils (8.23-4) unstable; urgency=low * [33] remove chroot optimization that avoids the actual chroot when diff -u coreutils-8.23/debian/control coreutils-8.23/debian/control --- coreutils-8.23/debian/control +++ coreutils-8.23/debian/control @@ -3,7 +3,7 @@ Section: utils Priority: required Standards-Version: 3.9.6.0 -Build-Depends: gettext (>= 0.10.37), debhelper (>= 5.0.0), autotools-dev, dh-buildinfo, texinfo (>= 4.2), groff, dpatch, libattr1-dev [linux-any], libacl1-dev [linux-any], libselinux1-dev (>= 1.32) [linux-any], gperf, bison +Build-Depends: gettext (>= 0.10.37), debhelper (>= 5.0.0), autotools-dev, dh-buildinfo, texinfo (>= 4.2), groff, dpatch, libattr1-dev [linux-any], libacl1-dev [linux-any], libselinux1-dev (>= 1.32) [linux-any], gperf, bison, help2man XS-Testsuite: autopkgtest Package: coreutils diff -u coreutils-8.23/debian/rules coreutils-8.23/debian/rules --- coreutils-8.23/debian/rules +++ coreutils-8.23/debian/rules @@ -20,6 +20,9 @@ endif endif +# ensure reproducible output +export SOURCE_DATE_EPOCH = $(shell date -d "$$(dpkg-parsechangelog -SDate)" +%s) + # implement no optimization build option CFLAGS = $(shell dpkg-buildflags --get CFLAGS) LDFLAGS = $(shell dpkg-buildflags --get LDFLAGS) @@ -46,6 +46,9 @@ dh_testdir dh_autotools-dev_updateconfig + # replace in-tree help2man by the one in Debian + cp /usr/bin/help2man man/help2man + CFLAGS="$(CFLAGS)" CPPFLAGS="$(CPPFLAGS)" \ LDFLAGS='$(LDFLAGS)' ./configure \ --build=$(DEB_BUILD_GNU_TYPE) --host=$(DEB_HOST_GNU_TYPE) \ signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#806328: xz-utils: please ship xz.pot
Source: xz-utils Version: 5.1.1alpha+20120614-2.1 Severity: wishlist User: reproducible-builds@lists.alioth.debian.org Usertags: timestamps X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org, sanv...@debian.org Hi! While working on the “reproducible builds” effort [1], we have noticed that xz-utils could not be built reproducibly. One of the problem is that the Gettext files always contain a different value for POT-Creation-Date. That's because the template for translations, xz.pot, is currently not shipped as part of the source. So xgettext will be run everytime, resulting in a new POT-Creation-Date and this slight loss of information. This is not considered a good practice, as Santiago Vila has explained in another bug report [2]. Upstream do ship xz.pot in its release tarballs. They also reference the file to potential translators in README. So it would probably be better to ship the file in Debian source. Thanks! [1]: https://wiki.debian.org/ReproducibleBuilds [2]: https://bugs.debian.org/792687#15 -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#806331: xz-utils: make the selected POSIX shell stable accross build environments
Source: xz-utils Version: 5.1.1alpha+20120614-2.1 Severity: normal User: reproducible-builds@lists.alioth.debian.org Usertags: environment X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org, sanv...@debian.org Hi! While working on the “reproducible builds” effort [1], we have noticed that xz-utils could not be built reproducibly. When dash is the default shell, the configure script m4/posix-shell.m4 will select /bin/bash as the “conforming POSIX shell”. When bash is the default shell, /bin/sh will be selected. The binary package currently in sid on amd64 uses /bin/bash. As bash is currently Essential:yes (#103284), this is probably not a problem. But I wonder if they would not be troubles if the package was built on an environment were bash is the default shell, but later installed on a system where /bin/dash is /bin/sh. So for reproducibility and safety reason, it would be best if the selected path to the shell would not depend on the build environment. Thanks! [1]: https://wiki.debian.org/ReproducibleBuilds -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] On which arch should Arch:all packages be built? addition to scsh-0.6, we should not be trying to build these packages on amd64:
Santiago Vila: > Let's take cmucl as an example: > > Package: cmucl > Architecture: i386 all > > It has "all", yes, but implicitly I take the Architecture line > as saying that I should only try to build the "Arch: all" package > using a i386 autobuilder. > > > Does this make sense? I think the question can be phrased as: should architecture-independent packages be buildable on all architectures? My own answer would be: yes, as long as they don't mandate a particular architecture in their Build-Depends-Indep. Is there something wrong with this answer? -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] On which arch should Arch:all packages be built? addition to scsh-0.6, we should not be trying to build these packages on amd64:
Santiago Vila: > On Fri, Nov 27, 2015 at 11:12:50AM +0100, Jérémy Bobbio wrote: > > I think the question can be phrased as: should architecture-independent > > packages be buildable on all architectures? > > > > My own answer would be: yes, as long as they don't mandate a particular > > architecture in their Build-Depends-Indep. > > > > Is there something wrong with this answer? > > My answer would be: Not necessarily. > > In theory, the build process could need to compile a program which is > used to create some file which is later used in the binary-indep target. > > Such program does not need to be portable because, well, some programs > are just not portable. Not in the sense that it's inherently non-portable > but just in the sense that nobody had the time to "port" it yet. > > If we had a case like this, an architecture field in the source > package saying "i386 all" would mean that we could do all this: > > "dpkg-buildpackage" under i386 to build the i386.deb and the all.deb. > "dpkg-buildpackage -A" under i386 to build only the all.deb. > "dpkg-buildpackage -B" under i386 to build only the i386.deb. > > and at the same time it would be possible that the source package is > just not designed or ready to build the all.deb under, say, amd64. > > In most cases it will probably work, but I do not see it as a > requirement that we should take for granted. For example, I would not > submit it as a bug (unless I find a way to fix it first). While I agree, I really think you've identified a hole in the policy here. Either we need a formal agreement that "i386 all" = "Arch: all" packages must be built on i386 or an extra field somewhere to indicate that. I would suggest moving the discussion to a better suited communication channel as this is a general issue. Maybe debian-dpkg@l.d.o? Thanks for finding this. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] diffoscope dedicated mailing-list
Hi! diffoscope is really growing bigger than just the Debian community, so it felt better to create a dedicated mailing-list. Please subscribe: https://lists.reproducible-builds.org/listinfo/diffoscope -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#806891: [Reproducible-commits] [diffoscope] 01/01: Multi-file HTML output
Joachim Breitner:
> Multi-file HTML output
Really great idea. :) Thanks for the initial patch!
> +parser.add_argument('--jquery', metavar='url', dest='jquery_url',
> +help='link to the jquery url, with --html-dir. By
> default, a symlink to /usr/share/javascript/jquery/jquery.js is created')
To make the Suggests a reality, I think it would be better to only
display “By default, a symlink…” if the file is already present on the
filesystem. Otherwise, if `--html-dir` is specified, the software should
exit with an error, asking users to specify `--jquery`.
Related question: is the browser going to visit the subpage if I have
JavaScript disabled? In that case, one other option is to only add the
JavaScript code when we have access to jQuery. I know some people don't
want JavaScript in their browser, so specifying something like
`--no-jquery` or `--jquery=none` could also be a way to turn it off.
> +# no output desired? print text
> +if not parsed_args.text_output and not parsed_args.html_output and
> not parsed_args.html_output_directory:
> +parsed_args.text_output = "-"
Maybe it would be nicer to write:
if not any(parsed_args.text_output, parsed_args.html_output,
parsed_args.html_output_directory):
> +def output_unified_diff(print_func, directory, anchor, unified_diff):
> +if directory and len(unified_diff) >
> Config.general.separate_file_diff_size:
> +# open a new file for this table
> +filename="%s.html" % hashlib.md5(anchor.encode('utf-8')).hexdigest()
I'm not entirely sure the anchor as it's working right now will be
unique…
> +logger.debug('separate html output for diff of %s (size %d)',
> anchor, len(unified_diff))
> +with file_printer(directory, filename) as new_print_func:
> +output_unified_diff_table(new_print_func, unified_diff)
So I think it would be great to crash here instead of overwrite if the file
aleardy exists. What do you think?
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Broken HTML on diffoscope.org
Axel Beckert: > there seems to be a missing "" on https://diffoscope.org/ behind > the link to http://brew.sh/. Well, actually it's a "" which > behaves like a missing one. Just fixed it. Thanks! -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808002: Bug#808002: diffoscope: Add support for Mozilla-optimized ZIPs
Mike Hommey: > It would be useful for diffoscope to output differences in omni.ja files as > for other Zip files, instead of ending up with a diff of an hexdump. > > The attached patch implements a minimal support for this. It however doesn't > look at the difference in the `preload` value. Great! I think it's fine to just skip the preload value. It will show up in the fallback binary comparison if that's the only remaining difference. I was going to merge this, but actually I have to ask: would you be kind enough to amend the test suite as well? Thanks! -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808103: Bug#808103: diffoscope: Truncated symbols in ELF diffs
Control: tag -1 + pending Mike Hommey: > When there are differences in symbols-related sections of ELF files, > symbols can be truncated if they are long enough. That happens often > with C++ mangled symbols. > > The following patch fixes the issue: […] Applied, thanks! -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808104: Bug#808104: diffoscope 43 KeyError's with several packages
Control: tag -1 + pending
Mattia Rizzolo:
> Seen on rb.d.n, with several packages, e.g.
> dhcpdump/1.8-2 on testing/amd64
> console-data/2:1.12-5 on testing/amd64
>
>
> Wed Dec 16 01:52:13 UTC 2015 - diffoscope 43 will be used to compare the two
> builds:
> Traceback (most recent call last):
> […]
> File "/usr/lib/python3/dist-packages/diffoscope/comparators/libarchive.py",
> line 150, in get_member
> raise KeyError('%s not found in archive', member_name)
> KeyError: ('%s not found in archive', './md5sums')
Thanks for the report. Fix pushed.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808121: Bug#808121: diffoscope: HTML output is bloated
Close: tag -1 + pending Mike Hommey: > Looking at the HTML in the HTML output, one can see that it is needlessly > large. > > Specifically, there appears to be a lot of e.g. > following each other, without even a separation between them. This conflates > the amount of memory necessary for browsers to render those pages. I've commited a fix for this specific issue. The HTML presenter borrowed a lot of code from diff2html which was probably not much optimized in the first place. I guess the output could be vastly improved, but I'd rather focus on other part of the code for now. Patches highly welcome in the meantime. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808002: Bug#808002: diffoscope: Add support for Mozilla-optimized ZIPs
Control: tag -1 + pending Mike Hommey: > On Tue, Dec 15, 2015 at 04:31:46PM +0100, Jérémy Bobbio wrote: > > Mike Hommey: > > > It would be useful for diffoscope to output differences in omni.ja files > > > as > > > for other Zip files, instead of ending up with a diff of an hexdump. > > > > > > The attached patch implements a minimal support for this. It however > > > doesn't > > > look at the difference in the `preload` value. > > > > Great! I think it's fine to just skip the preload value. It will show up > > in the fallback binary comparison if that's the only remaining difference. > > > > I was going to merge this, but actually I have to ask: would you be kind > > enough to amend the test suite as well? > > Attached. Awesome! Applied. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808121: Bug#808121: Bug#808121: diffoscope: HTML output is bloated
Mike Hommey: > Here's another easy win, attached. Nice. Commited. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808121: Bug#808121: Bug#808121: diffoscope: HTML output is bloated
Esa Peuha: > While we are at it, let's convert HTML character entity references > (which each use 6-8 characters and as many bytes in the HTML file) > to actual characters (which UTF-8 encodes as 2-3 bytes). Since all > diffoscope output files are peppered with abundant amounts of these > things, this could reduce the file sizes by a few percent at least. > I used Python string literals instead of the actual characters in > the Python file, because 1) the non-breaking and zero-width spaces > would be very hard to distinguish from ordinary space and missing > string content, respectively, and 2) it is impossible to be sure > that every piece of software that is ever going to be used to view > or edit the file would handle non-ASCII characters correctly. Thanks for the patch. It's been commited and push. I would be grateful if you could submit ready-to-merge Git changes next time (see git-format-patch(1)). -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808003: Bug#808003: diffoscope: Comparing directories shouldn't care about file order
Control: tag -1 + pending Mike Hommey: >* What led up to the situation? > > Comparing two directories > >* What exactly did you do (or not do) that was effective (or > ineffective)? > > Ran `diffoscope --html output.html a b` where a and b are directories > containing > more or less the same thing. > >* What was the outcome of this action? > > The output contained differences in the output for find due to > filesystem-dependent behavior wrt file ordering in readdir(3). > >* What outcome did you expect instead? > > The sorted list should be compared instead. I've just commited a fix. Thanks for the report. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808199: Bug#808199: diffoscope: Weird line numbering in diff output
Control: tag -1 + pending Mike Hommey: > Attached here is two files that can be compared directly, exposing the > same problem. That's essentially the output of tar --full-time -tvf on > both tarballs from the original bug report. > > The following patch fixes the numbering issue: > […] Commited, thanks. The bug was around since the very first version of the current HTML presenter code! > However, there's another issue, that the diff is actually incomplete: > the right hand side stops 3 lines too early. I've fixed this as well. The 'lines skipped' message was not written when we were skipping lines at the end of the diff. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808541: Bug#808541: diffoscope TypeError with openocd/0.9.0-1 in testing/amd64
Control: tag -1 + pending Mattia Rizzolo: > TypeError: unorderable types: bytes() < str() I've pushed a fix. Thanks for the report. (This only appeared when the default encoding was not Unicode-aware.) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808120: Bug#808120: diffoscope: Should use less memory
Control: tag -1 + pending Mike Hommey: >* What was the outcome of this action? > > A 533KB HTML file that, considering its size, doesn't contain much > differences. > Yet, while processing this, the diffoscope process (not its children readelf, > objdump or diff processes) sucked more than 4GB of memory. That tells me > something unexpectedly suboptimal is happening. Absolutely! The code was building a full list of lines to compare in memory instead of feeding them to diff as they were produced. The fix was trivial once the issue was understood. Thanks for the nudge. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] dpkg-dev: please add support for .buildinfo files
pecified). .IP \fB6.\fP 3 +Unless a source-only build has been requested, it runs the \fBbuildinfo\fP hook +and calls \fBdpkg\-genbuildinfo\fP to generate a \fB.buildinfo\fP file. +Several \fBdpkg\-buildpackage\fP options are forwarded to +\fBdpkg\-genbuildinfo\fP. +.IP \fB7.\fP 3 It runs the \fBchanges\fP hook and calls \fBdpkg\-genchanges\fP to generate a \fB.changes\fP file. Many \fBdpkg\-buildpackage\fP options are forwarded to \fBdpkg\-genchanges\fP. -.IP \fB7.\fP 3 +.IP \fB8.\fP 3 It runs the \fBpostclean\fP hook and if \fB\-tc\fP is specified, it will call \fBfakeroot debian/rules clean\fP again. -.IP \fB8.\fP 3 -It calls \fBdpkg\-source \-\-after\-build\fP. .IP \fB9.\fP 3 +It calls \fBdpkg\-source \-\-after\-build\fP. +.IP \fB10.\fP 3 It runs the \fBcheck\fP hook and calls a package checker for the \fB.changes\fP file (if a command is specified in \fBDEB_CHECK_COMMAND\fP or with \fB\-\-check\-command\fP). -.IP \fB10.\fP 3 +.IP \fB11.\fP 3 It runs the \fBsign\fP hook and calls \fBgpg2\fP or \fBgpg\fP to sign the \fB.dsc\fP file (if any, unless \fB\-us\fP is specified or on UNRELEASED builds), and the \fB.changes\fP file (unless \fB\-uc\fP is specified or on UNRELEASED builds). -.IP \fB11.\fP 3 +.IP \fB12.\fP 3 It runs the \fBdone\fP hook. . .SH OPTIONS @@ -317,6 +322,12 @@ The source package version (without the epoch). The upstream version. .RE .TP +.BI \-\-buildinfo-identifier= identifier +By default, \fBdpkg\-buildpackage\fP put the system hostname and the +current time in the name of the \fB.buildinfo\fP file. An arbitrary +identifier can be specified as a replacement (since dpkg 1.18.5). It must +contain only alphanumeric characters and hyphens. +.TP .BI \-p sign-command When \fBdpkg\-buildpackage\fP needs to execute GPG to sign a source control (\fB.dsc\fP) file or a \fB.changes\fP file it will run @@ -351,6 +362,10 @@ Passed unchanged to \fBdpkg\-source\fP. See its manual page. Pass option \fIopt\fP to \fBdpkg\-source\fP (since dpkg 1.15.6). Can be used multiple times. .TP +.BI \-\-buildinfo\-option= opt +Pass option \fIopt\fP to \fBdpkg\-genbuildinfo\fP (since dpkg 1.18.5). +Can be used multiple times. +.TP .BI \-\-changes\-option= opt Pass option \fIopt\fP to \fBdpkg\-genchanges\fP (since dpkg 1.15.6). Can be used multiple times. @@ -422,6 +437,7 @@ and initial arguments for .BR dpkg\-source (1), .BR dpkg\-architecture (1), .BR dpkg\-buildflags (1), +.BR dpkg\-genbuildinfo (1), .BR dpkg\-genchanges (1), .BR fakeroot (1), .BR lintian (1), diff --git a/man/dpkg-genbuildinfo.1 b/man/dpkg-genbuildinfo.1 new file mode 100644 index 000..77f2a76 --- /dev/null +++ b/man/dpkg-genbuildinfo.1 @@ -0,0 +1,98 @@ +.\" dpkg manual page - dpkg-genbuildinfo(1) +.\" +.\" Copyright © 1995-1996 Ian Jackson +.\" Copyright © 2000 Wichert Akkerman +.\" Copyright © 2008-2010 Raphaël Hertzog +.\" Copyright © 2006-2014 Guillem Jover +.\" Copyright © 2015 Jérémy Bobbio +.\" +.\" This is free software; you can redistribute it and/or modify +.\" it under the terms of the GNU General Public License as published by +.\" the Free Software Foundation; either version 2 of the License, or +.\" (at your option) any later version. +.\" +.\" This is distributed in the hope that it will be useful, +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +.\" GNU General Public License for more details. +.\" +.\" You should have received a copy of the GNU General Public License +.\" along with this program. If not, see <https://www.gnu.org/licenses/>. +. +.TH dpkg\-genbuildinfo 1 "2015-01-01" "Debian Project" "dpkg utilities" +.SH NAME +dpkg\-genbuildinfo \- generate Debian .buildinfo files +. +.SH SYNOPSIS +.B dpkg\-genbuildinfo +.RI [ option ...] +.br +. +.SH DESCRIPTION +.B dpkg\-genbuildinfo +reads information from an unpacked and built Debian source tree and +from the files it has generated and generates a Debian control +file describing the build environment and the build products +.RB ( .buildinfo " file)." +. +.SH OPTIONS +.TP +.BI \-c controlfile +Specifies the main source control file to read information from. The +default is +.BR debian/control . +.TP +.BI \-l changelog-file +Specifies the changelog file to read information from. The +default is +.BR debian/changelog . +.TP +.BI \-f files-list-file +Specifies where is the list of files that have been produced by the build, +rather than using +.BR debian/files . +.TP +.BI \-F changelog-format +Specifies the format of the changelog. See \fBdpkg\-parsechangelog\fP(1) +for information about alternative formats. +.TP +.BI \-u upload-files-dir +Look for the files to be uploaded in +.I upload-files-dir +rather than +.B .. +.RB ( dpkg\-genbuildinfo +needs to find these files so that it can include their sizes and +c
[Reproducible-builds] package uploaded to our repo
dpkg_1.18.5~reproducible1.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] dpkg-dev: please make mtimes of packaged files deterministic
reassign 759886 dpkg-dev
retitle 759886 dpkg-dev: please make mtimes of packaged files deterministic
thanks
Hi!
The attached patch series is an attempt to make the mtimes of packaged
files deterministic. It is taken from the `pu/reproducible_builds`
dpkg branch maintained by the “reproducible builds” folks [1].
The first two patches introduce the idea of a canonical build timestamp
that will be used throughout dpkg-deb.
The first patch will make use of this timestamp to set the mtime in ar
headers (that's #75). All headers will thus get the same timestamp
instead of recording the current time as they are added.
The second patch will use the --mtime and --clamp-mtime option of tar to
clamp the mtime of files recorded in control.tar and data.tar to the
build timestamp: files created at a later time will see their mtime
set to the build timestamp (that's #759886). As --clamp-mtime is only
available since tar/1.28-1 and has not yet been merged upstream,
dpkg-deb will first look for its availability by looking for the option
in the output of “tar --help”. If it's not available, it will fallback
to the previous behavior.
The third patch adds the ability to set the aforementioned build
timestamp using the SOURCE_DATE_EPOCH environment variable [2].
The forth patch changes dpkg-buildpackage to set SOURCE_DATE_EPOCH to
the time of the latest debian/changelog entry if it hasn't been already
set, effectively making the timestamps recorded by dpkg-deb in the most
common build process deterministic.
[1]:
https://anonscm.debian.org/cgit/reproducible/dpkg.git/log/?h=pu/reproducible_builds
[2]: https://reproducible-builds.org/specs/source-date-epoch/
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
From e56a69e2fac8531c9a45008470ca8aa8dd9f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Bobbio?=
Date: Tue, 27 Aug 2013 22:38:31 +0200
Subject: [PATCH 1/4] dpkg-deb: Use a single timestamp for ar headers when
building a .deb
In order to make build reproducible in the future, we now use a single
timestamp in all ar headers when creating a .deb.
Previously, each ar header would have the current time of its creation.
This level of precision is not really needed and the time of the beginning of
the build is good enough.
Address: #75
---
dpkg-deb/build.c | 10 +++---
dpkg-split/split.c | 4 ++--
lib/dpkg/ar.c | 13 +++--
lib/dpkg/ar.h | 4 ++--
4 files changed, 18 insertions(+), 13 deletions(-)
diff --git a/dpkg-deb/build.c b/dpkg-deb/build.c
index 8d9f066..117e424 100644
--- a/dpkg-deb/build.c
+++ b/dpkg-deb/build.c
@@ -37,6 +37,7 @@
#include
#include
#include
+#include
#include
#include
@@ -529,6 +530,7 @@ do_build(const char *const *argv)
int arfd;
int p1[2], gzfd;
pid_t c1, c2;
+ time_t build_timestamp;
/* Decode our arguments. */
dir = *argv++;
@@ -559,6 +561,8 @@ do_build(const char *const *argv)
}
m_output(stdout, _(""));
+ build_timestamp = time(NULL);
+
/* Now that we have verified everything its time to actually
* build something. Let's start by making the ar-wrapper. */
arfd = creat(debar, 0644);
@@ -636,8 +640,8 @@ do_build(const char *const *argv)
compressor_get_extension(control_compress_params.type));
dpkg_ar_put_magic(debar, arfd);
-dpkg_ar_member_put_mem(debar, arfd, DEBMAGIC, deb_magic, strlen(deb_magic));
-dpkg_ar_member_put_file(debar, arfd, adminmember, gzfd, -1);
+dpkg_ar_member_put_mem(debar, arfd, DEBMAGIC, deb_magic, build_timestamp, strlen(deb_magic));
+dpkg_ar_member_put_file(debar, arfd, adminmember, gzfd, build_timestamp, -1);
} else {
internerr("unknown deb format version %d.%d", deb_format.major, deb_format.minor);
}
@@ -679,7 +683,7 @@ do_build(const char *const *argv)
if (lseek(gzfd, 0, SEEK_SET))
ohshite(_("failed to rewind temporary file (%s)"), _("data member"));
-dpkg_ar_member_put_file(debar, arfd, datamember, gzfd, -1);
+dpkg_ar_member_put_file(debar, arfd, datamember, gzfd, build_timestamp, -1);
close(gzfd);
}
diff --git a/dpkg-split/split.c b/dpkg-split/split.c
index 8137654..d132e3e 100644
--- a/dpkg-split/split.c
+++ b/dpkg-split/split.c
@@ -210,13 +210,13 @@ mksplit(const char *file_src, const char *prefix, off_t maxpartsize,
(intmax_t)st.st_size, (intmax_t)partsize,
curpart, nparts, pkg->available.arch->name);
dpkg_ar_member_put_mem(file_dst.buf, fd_dst, PARTMAGIC,
- partmagic.buf, partmagic.used);
+ partmagic.buf, time(NULL), partmagic.used);
varbuf_reset(&partmagic);
/* Write the data part. */
varbuf_printf(&partname, "data.%d", curpart);
dpkg_ar_member_put_file(file_dst.buf, fd_dst, partname.buf,
- fd_src, cur_
Re: [Reproducible-builds] package uploaded to our repo
Jérémy Bobbio: > dpkg_1.18.5~reproducible1.dsc has just been uploaded to > https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain With this upload, we should not need a modified debhelper anymore. :) See #759886. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Communicating about the change in behaviour for ar
Manoj Srivastava:
> I was trying to import the new version of make into unstable, and
> I discovered that t/10 tests about the archive related part of makes
> test suite failed. The reason was a change in the behaviour of ar,
> which is now configured with --enable-deterministic-archives. When
> adding files and the archive index use zero for UIDs, GIDs, timestamps,
> and use consistent file modes for all files. When this option is used,
> if ar is used with identical options and identical input files,
> multiple runs will create identical output files regardless of the
> input files' owners, groups, file modes, or modification times. This
> seems like good news for reproducible builds.
>
> Unfortunately, when using makes libxx(*.a) syntax for creating
> archives, make needs the timestamp of the file in order to decide to
> update it or not. With the current deterministic behavior of ar, the
> timestamp is always 0. This is behaviour that is not backwards
> compatible (as can be seen in the bug report #798804 and
> #798913). Some operations, instead of being no-ops, now rebuild theg/Lunar
> archive.
>
> T think the change, because of the benefits of the
> reproducible builds, are a good thing, but I also think that they are
> not visible to the general user base (not all the users of ar and make
> are debian developers, and the release is not the only thing built
> using ar and make). My recommendations is a NEWS file entry for
> binutils and make; and a mention in the release announcement for
> reproducible builds.
>
> I have uploaded a version of make the defaults to adding U to the
> ARFLAGS, but, on research and reflection, I would be open to reverting
> that and adding a NEWS entry.
In any case, I think we should communicate to users that something
unexpected (from the point view of make) is happening so they can adapt
their Makefile. What would you think of the attached patch series?
The warning is actually implemented in the second patch, but the first
one is required to be able to differentiate a non-existent archive or
member from a member with a timestamp set to 0.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
From a40d198fb5a3a0387ce5b4f0f40e23cab2a535bb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Bobbio?=
Date: Sun, 17 Jan 2016 10:55:40 +
Subject: [PATCH 1/2] Make ar_member_date compatible with archives with
timestamps set to 0
ar_scan() scanning function uses 0 to indicate that scanning should continue.
This made ar_member_date() unable to differentiate when it was unable to find
the requested member from a member with a timestamp set to 0.
We thus change its prototype to have its return value indicate if it has been
able to find the requested member. The timestamp is set through the newly given
pointer.
---
ar.c | 33 +
commands.c | 5 -
dir.c | 7 +--
makeint.h | 2 +-
remake.c | 7 +++
5 files changed, 38 insertions(+), 16 deletions(-)
diff --git a/ar.c b/ar.c
index 675572a..d10a8df 100644
--- a/ar.c
+++ b/ar.c
@@ -68,25 +68,39 @@ ar_parse_name (const char *name, char **arname_p, char **memname_p)
/* This function is called by 'ar_scan' to find which member to look at. */
+struct member_date_lookup
+{
+ const char *name;
+ time_t *member_date;
+};
+
/* ARGSUSED */
static long int
ar_member_date_1 (int desc UNUSED, const char *mem, int truncated,
long int hdrpos UNUSED, long int datapos UNUSED,
long int size UNUSED, long int date,
int uid UNUSED, int gid UNUSED, int mode UNUSED,
- const void *name)
+ const void *data)
{
- return ar_name_equal (name, mem, truncated) ? date : 0;
+ const struct member_date_lookup *lookup_data = data;
+ if (ar_name_equal (lookup_data->name, mem, truncated))
+{
+ *lookup_data->member_date = date;
+ return 1;
+}
+ return 0;
}
-/* Return the modtime of NAME. */
+/* Read the modtime of NAME in MEMBER_DATE.
+ Returns 1 if NAME exists, 0 otherwise. */
-time_t
-ar_member_date (const char *name)
+int
+ar_member_date (const char *name, time_t *member_date)
{
char *arname;
char *memname;
- long int val;
+ int found;
+ struct member_date_lookup lookup_data;
ar_parse_name (name, &arname, &memname);
@@ -107,11 +121,14 @@ ar_member_date (const char *name)
(void) f_mtime (arfile, 0);
}
- val = ar_scan (arname, ar_member_date_1, memname);
+ lookup_data.name = memname;
+ lookup_data.member_date = member_date;
+ found = ar_scan (arname, ar_member_date_1, &lookup_data);
free (arname);
- return (val <= 0 ? (time_t) -1 : (time_t) val);
+ /* return 0 (not found) if the archive does not exist or has inva
[Reproducible-builds] Bug#808207: diffoscope: Filter objdump --disassemble output before diffing it
Hi Mike! Mike Hommey: > When comparing large ELF binaries, some minor differences can end up hurting > the visibility of more important differences. > > Specifically, objdump --disassemble displays symbols+offsets for addresses > it derives from IP-relative addressing, like the following: > >9d2be2: 48 8d 05 42 65 24 02lea0x2246542(%rip),%rax# > 2c1912b <_fini@@xul45a1+0x1d803> I would be grateful if you could try again using the master branch. Dhole made diffoscope compare ELF sections individually and I wonder how much it helped with this problem. If it doesn't, would you be so kind to provide example binaries? Thanks, -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808197: diffoscope: readelf --debug-dump can be brutal
Control: tag -1 + pending Mike Hommey: > However, if each debug section was compared individually, that give more > chance for diffoscope to work. (--debug-dump takes values allowing to treat > each debug section independently) Implemented by Dhole and currently available in the master branch. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#808267: diffoscope: Redundant information in ELF comparisons
Hi Mike, Mike Hommey: > When comparing ELF files, the following commands are used: > - readelf --all > - readelf --debug-dump > - objdump --disassemble --full-contents > > objdump --disassemble --full-contents is actually redundant in itself. For > example, it will dump both an hexdump and a disassembly of the .text section. > It's also redundant with the output of readelf --debug-dump because it does an > hexdump of the .debug_* sections that readelf --debug-dump does a dwarf dump > of. The master branch now compare ELF files section by section. If you could test it and see if there's still redundancies, I would be grateful. Thanks! -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812428: libgcrypt20: please make the build reproducible (timestamps)
Source: libgcrypt20
Version: 1.6.4-4
Severity: wishlist
Tag: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps
X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
Hi!
While working on the “reproducible builds” effort [1], we have noticed
that libgcrypt20 could not be built reproducibly.
The attached patch adds support for SOURCE_DATE_EPOCH to set the value
of BUILD_TIMESTAMP defined in the configure script. Once applied,
libgcrypt20 is nearly reproducible in our current experimental
framework—their might be remaining timestamps in the PDF documentation.
[1]: https://wiki.debian.org/ReproducibleBuilds
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
diff --git a/debian/control b/debian/control
index ff31343..e3a19c4 100644
--- a/debian/control
+++ b/debian/control
@@ -6,7 +6,7 @@ Uploaders: Andreas Metzler ,
Eric Dorland , James Westby ,
Simon Josefsson
Build-Depends: debhelper (>= 9.20150628),
- libgpg-error-dev (>= 1.11), autotools-dev
+ libgpg-error-dev (>= 1.11), autotools-dev, dh-autoreconf
Build-Depends-Indep: texlive-latex-base, texlive-generic-recommended,
texinfo (>= 4.6-0)
Standards-Version: 3.9.6
diff --git a/debian/patches/30_support_source_date_epoch.diff b/debian/patches/30_support_source_date_epoch.diff
new file mode 100644
index 000..8e43921
--- /dev/null
+++ b/debian/patches/30_support_source_date_epoch.diff
@@ -0,0 +1,18 @@
+Description: support setting BUILD_TIMESTAMP using SOURCE_DATE_EPOCH
+ Enable reproducible builds by supporting setting the value of BUILD_TIMESTAMP
+ through the SOURCE_DATE_EPOCH environment variable. More information at:
+ https://reproducible-builds.org/specs/source-date-epoch/
+Author: Jérémy Bobbio
+Last-Update: 2016-01-23
+
+--- libgcrypt20-1.6.4.orig/configure.ac
libgcrypt20-1.6.4/configure.ac
+@@ -1993,7 +1993,7 @@ changequote([,])dnl
+ BUILD_FILEVERSION="${BUILD_FILEVERSION}mym4_revision_dec"
+ AC_SUBST(BUILD_FILEVERSION)
+
+-BUILD_TIMESTAMP=`date -u +%Y-%m-%dT%H:%M+ 2>/dev/null || date`
++BUILD_TIMESTAMP=`date -d"@$SOURCE_DATE_EPOCH" -u +%Y-%m-%dT%H:%M+ 2>/dev/null || date -u +%Y-%m-%dT%H:%M+ 2>/dev/null || date`
+ AC_SUBST(BUILD_TIMESTAMP)
+ AC_DEFINE_UNQUOTED(BUILD_TIMESTAMP, "$BUILD_TIMESTAMP",
+[The time this package was configured for a build])
diff --git a/debian/patches/series b/debian/patches/series
index d5154e9..2f0a91b 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1,3 +1,4 @@
12_lessdeps_libgcrypt-config.diff
15_multiarchpath_in_-L.diff
20_fedora_libgcrypt-1.6.3-aliasing.patch
+30_support_source_date_epoch.diff
diff --git a/debian/rules b/debian/rules
index 27614f9..1b137a5 100755
--- a/debian/rules
+++ b/debian/rules
@@ -48,4 +48,4 @@ override_dh_gencontrol:
dh_gencontrol --remaining-packages
%:
- dh $@ --parallel --with autotools_dev
+ dh $@ --parallel --with autoreconf --with autotools_dev
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812524: Bug#812524: UnicodeDecodeError with glob2/0.9.4.4-2.4 on unstable/amd64
Control: tag -1 + pending Mattia Rizzolo: > File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode > return codecs.ascii_decode(input, self.errors)[0] > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 95: > ordinal not in range(128) Fixed in 00b26a6. -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812534: Bug#812534: TypeError with python-expyriment/0.7.0+git34-g55a4e7e-3.1 on unstable/amd64
Control: tag -1 + pending
Mattia Rizzolo:
> File "/usr/lib/python3/dist-packages/diffoscope/comparators/debian.py",
> line 185, in recognizes
> for d in buildinfo.get('Checksums-Sha256'):
> TypeError: 'NoneType' object is not iterable
Already fixed in af6bcf80.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812762: findutils: FTBFS if fr_FR or other locales are installed
Source: findutils
Version: 4.6.0-2
Severity: important
User: reproducible-builds@lists.alioth.debian.org
Usertag: ftbfs
Hi!
findutils will FTBFS due to test errors if fr_FR or a couple other
locales are installed. The failing tests are test-mbrtowc{1,2,3,4}.sh.
The log contains:
test-mbrtowc.c:49: assertion 'ret == (size_t)(-2)' failed
Aborted
An easy way to test this is to install the locales-all package in the
build environment.
Maybe mbrtowc() behavior when n == 0 has changed in recent libc?
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#807669: dh-strip-nondeterminism: Breaks some jar file
Control: tag -1 + patch
Raphael Hertzog:
> On Mon, 14 Dec 2015, Raphael Hertzog wrote:
> > Your analysis is correct but dh_strip_nondeterminisn should detect the
> > signature and avoid messing up with the file in that case.
> >
> > That's what this bug is about.
>
> And we got another case where dh_strip_nondeterminism actually broke a
> working package... https://bugs.kali.org/view.php?id=3019
>
> Is there anything we can do to ensure that this bug gets a timely fix?
Attached is a patch which I think could work. I'm not confident enough
in my Perl skills to commit directly though.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
From e2dfd6d97a2f0af21f5d113d7eed12d90ebe2384 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Bobbio?=
Date: Tue, 26 Jan 2016 13:59:14 +
Subject: [PATCH] Don't process signed Jar file
Otherwise, we will break the signature and that's not a good thing.
I guess it would be better if we had a way to pass a warning back. But that's
something for the future.
Closes: #807669
---
lib/File/StripNondeterminism/handlers/jar.pm | 6 ++
1 file changed, 6 insertions(+)
diff --git a/lib/File/StripNondeterminism/handlers/jar.pm b/lib/File/StripNondeterminism/handlers/jar.pm
index e136395..4af06a0 100644
--- a/lib/File/StripNondeterminism/handlers/jar.pm
+++ b/lib/File/StripNondeterminism/handlers/jar.pm
@@ -87,6 +87,12 @@ sub _jar_normalize_member {
sub normalize {
my ($jar_filename) = @_;
+ my $jar = Archive::Zip->new($jar_filename);
+ my @filenames = $jar->memberNames();
+ for my $filename (@filenames) {
+ # don't process signed jars
+ return 0 if $filename =~ /\AMETA-INF\/[^\/]+\.SF\Z/i;
+ }
return File::StripNondeterminism::handlers::zip::normalize($jar_filename,
filename_cmp => \&_jar_filename_cmp,
member_normalizer => \&_jar_normalize_member);
--
2.6.1
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#138409: dpkg-dev: please add support for .buildinfo files
Jérémy Bobbio:
> The attached patch will enable dpkg-buildpackage to create .buildinfo
> files as specified on the Debian wiki [1]. They have two main purposes:
>
> * recording information about the system environment used during a
>particular build—versions of the build dependencies installed, system
>architecture, etc. for easier forensics/debugging;
> * describe how to recreate (partially or in full) the original
>environment when trying to reproduce a particular build.
I think the proposed patch is missing a field to record some environment
variables that can affect the build process. Right now, I'm thinking of
DEB_BUILD_OPTIONS and DEB_flag_{SET,STRIP,APPEND,PREPEND} (from
dpkg-buildflags). Maybe others? Build profiles?
Initially I was against recording such information, but now that we
understand the purpose of .buildinfo files better and not mandate that
they be reproducible themselves, it doesn't matter if one contains
`DEB_BUILD_OPTIONS=parallel=4` and the other
`DEB_BUILD_OPTIONS=parallel=16`. It is the responsibility of users
trying to recreate a given package to filter this out.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812876: glib2.0: please make the build reproducible (locale)
Source: glib2.0
Version: 2.46.2-3
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: locale
Hi!
While working on the “reproducible builds” effort [1], we have noticed
that glib2.0 could not be built reproducibly.
The attached patch ensure that functions are sorted using the C locale
when giotypefuncs.c is generated.
Once applied, glib2.0 can be built reproducibly in our current
experimental framework.
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
diff -Nru glib2.0-2.46.2/debian/patches/locale-independent-sort-for-giotypefuncs.patch glib2.0-2.46.2/debian/patches/locale-independent-sort-for-giotypefuncs.patch
--- glib2.0-2.46.2/debian/patches/locale-independent-sort-for-giotypefuncs.patch 1970-01-01 01:00:00.0 +0100
+++ glib2.0-2.46.2/debian/patches/locale-independent-sort-for-giotypefuncs.patch 2016-01-27 14:33:11.0 +0100
@@ -0,0 +1,13 @@
+Index: glib2.0-2.46.2/gio/tests/Makefile.am
+===
+--- glib2.0-2.46.2.orig/gio/tests/Makefile.am
glib2.0-2.46.2/gio/tests/Makefile.am
+@@ -558,7 +558,7 @@ giotypefuncs.c: Makefile
+ ${CPP} $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) xgen-giosrc.c | \
+ $(GREP) -o '\bg_[A-Za-z0-9_]*_get_type\b' | \
+ $(GREP) -v 'g_io_extension_get_type\|g_variant_get_type' | \
+- sort | uniq | \
++ LC_ALL=C sort | uniq | \
+ $(SED) -e 's/^/*tp++ = /' -e 's/$$/ ();/' >> xgen-gio && \
+ cp xgen-gio $@ # && rm -f xgen-gio xgen-giosrc.c
+
diff -Nru glib2.0-2.46.2/debian/patches/series glib2.0-2.46.2/debian/patches/series
--- glib2.0-2.46.2/debian/patches/series 2015-12-23 17:08:41.0 +0100
+++ glib2.0-2.46.2/debian/patches/series 2016-01-27 14:31:36.0 +0100
@@ -14,3 +14,4 @@
0001-Fix-trashing-on-overlayfs.patch
bug712848-volume-monitor-deadlock-kfreebsd.patch
disable-failing-test-for-pcre838.patch
+locale-independent-sort-for-giotypefuncs.patch
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] package uploaded to our repo
libxslt_1.1.28-2.1.0~reproducible2.dsc has just been uploaded to https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812895: libxml++2.6: please make the build reproducible (environment)
Source: libxml++2.6
Version: 2.40.1-1
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: environment
Hi!
While working on the “reproducible builds” effort [1], we have noticed
that libxml++2.6 could not be built reproducibly.
The reason is that the entire `examples` directory is shipped after
being built and so contains several libtool scripts which capture the
PATH. These files and others added during the build are actually not
useful as examples.
The attached patch replaces the wildcard in
`debian/libxml++2.6.examples` by the output of
`find examples -type f | LC_ALL=C sort` to ship only source files. Once
applied, libxml++2.6 can be built reproducibly in our current
experimental framework.
[1]: https://wiki.debian.org/ReproducibleBuilds
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
diff -Nru libxml++2.6-2.40.1/debian/libxml++2.6-doc.examples libxml++2.6-2.40.1/debian/libxml++2.6-doc.examples
--- libxml++2.6-2.40.1/debian/libxml++2.6-doc.examples 2012-02-06 04:48:29.0 +0100
+++ libxml++2.6-2.40.1/debian/libxml++2.6-doc.examples 2016-01-27 17:59:41.0 +0100
@@ -1 +1,65 @@
-examples/*
+examples/Makefile.am
+examples/Makefile.in
+examples/README
+examples/dom_build/main.cc
+examples/dom_parse_entities/example.dtd
+examples/dom_parse_entities/example.xml
+examples/dom_parse_entities/main.cc
+examples/dom_parser/example.dtd
+examples/dom_parser/example.xml
+examples/dom_parser/example_invalid.xml
+examples/dom_parser/example_with_namespace.xml
+examples/dom_parser/main.cc
+examples/dom_parser_raw/example.dtd
+examples/dom_parser_raw/example.xml
+examples/dom_parser_raw/example_invalid.xml
+examples/dom_parser_raw/main.cc
+examples/dom_read_write/README
+examples/dom_read_write/example.dtd
+examples/dom_read_write/example.xml
+examples/dom_read_write/main.cc
+examples/dom_update_namespace/example1.xml
+examples/dom_update_namespace/example2.xml
+examples/dom_update_namespace/main.cc
+examples/dom_xinclude/example.xml
+examples/dom_xinclude/include1.txt
+examples/dom_xinclude/include2.xml
+examples/dom_xinclude/main.cc
+examples/dom_xpath/example.xml
+examples/dom_xpath/main.cc
+examples/dtdvalidation/example.dtd
+examples/dtdvalidation/main.cc
+examples/import_node/example1.xml
+examples/import_node/example2.xml
+examples/import_node/main.cc
+examples/sax_exception/example.xml
+examples/sax_exception/main.cc
+examples/sax_exception/myparser.cc
+examples/sax_exception/myparser.h
+examples/sax_parser/example.xml
+examples/sax_parser/main.cc
+examples/sax_parser/myparser.cc
+examples/sax_parser/myparser.h
+examples/sax_parser_build_dom/README
+examples/sax_parser_build_dom/example.xml
+examples/sax_parser_build_dom/main.cc
+examples/sax_parser_build_dom/svgdocument.cc
+examples/sax_parser_build_dom/svgdocument.h
+examples/sax_parser_build_dom/svgelement.cc
+examples/sax_parser_build_dom/svgelement.h
+examples/sax_parser_build_dom/svggroup.h
+examples/sax_parser_build_dom/svgparser.cc
+examples/sax_parser_build_dom/svgparser.h
+examples/sax_parser_build_dom/svgpath.h
+examples/sax_parser_entities/example.xml
+examples/sax_parser_entities/main.cc
+examples/sax_parser_entities/myparser.cc
+examples/sax_parser_entities/myparser.h
+examples/schemavalidation/example.rng
+examples/schemavalidation/example.xml
+examples/schemavalidation/example.xsd
+examples/schemavalidation/main.cc
+examples/testutilities.cc
+examples/testutilities.h
+examples/textreader/example.xml
+examples/textreader/main.cc
--- libxml++2.6-2.40.1/debian/rules 2012-02-06 04:48:29.0 +0100
+++ libxml++2.6-2.40.1/debian/rules 2016-01-27 18:15:57.278176904 +0100
@@ -23,4 +23,3 @@
--libdir=\$${prefix}/lib/$(DEB_HOST_MULTIARCH)
DEB_MAKE_CHECK_TARGET := check
DEB_DH_MAKESHLIBS_ARGS_$(SHARED_PKG) += -V"$(SHARED_PKG) (>= $(SHVER))"
-DEB_INSTALL_EXAMPLES_$(DOC_PKG) += -XMakefile -X.deps -X.libs -X.o
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#812899: libsm: please make the build reproducible (locale)
Source: libsm Version: 2:1.2.2-1 Severity: wishlist Tags: patch User: reproducible-builds@lists.alioth.debian.org Usertags: locale Hi! While working on the “reproducible builds” effort [1], we have noticed that libsm could not be built reproducibly. The attached patch makes sure the text documentation is always generated using a UTF-8 locale. Once applied, libsm can be built reproducibly in our current experimental framework. [1]: https://wiki.debian.org/ReproducibleBuilds -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- From f23f58eb98288fc3178d582dc03a77a332dc82fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=A9my=20Bobbio?= Date: Wed, 27 Jan 2016 18:32:31 +0100 Subject: [PATCH] Make sure text documentation is generated as UTF-8 Otherwise, if the package is built on a system with a locale using another character encoding, the resulting text documentation might not be readable on other systems. --- docbook.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docbook.am b/docbook.am index bba4d54..0c1a086 100644 --- a/docbook.am +++ b/docbook.am @@ -43,7 +43,7 @@ if HAVE_XMLTO_TEXT shelf_DATA += $(docbook:.xml=.txt) %.txt: %.xml $(chapters) - $(AM_V_GEN)$(XMLTO) $(XMLTO_HTML_FLAGS) txt $< + LC_ALL=C.UTF-8 $(AM_V_GEN)$(XMLTO) $(XMLTO_HTML_FLAGS) txt $< endif HAVE_XMLTO_TEXT if HAVE_FOP -- 2.7.0 signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#138409: dpkg-dev: please add support for .buildinfo files
Guillem Jover: > I've some pending changes I'll be committing to master or a separate > branch, that I'd like to be tested on the reproducible setup (ideally > against the already generated and pre-existing reproducible binaries), > if that's possible, I'll ask about that when those land, I just need > to finish up fewm more unit tests. > > Here's a quick review: […] Thanks for the review! Just so I can organize my work, are your pending changes related to the .buildinfo? I can spend the next days improving the patch with your remarks, but I'd rather not duplicate your work. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
[Reproducible-builds] Bug#813052: Bug#813052: diffoscope takes more than an hour on foreign arch libc6
Helmut Grohne: > Even though I cannot reproduce the issue at hand, I think that the code > adding automatic debug symbols looks fishy to me. It appears to recurse > over /tmp here and that looks very wrong to me. I don't understand what you mean by that. Could you provide be (at least some) of the `--debug` output? -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Bug#138409: dpkg-dev: please add support for .buildinfo files
Guillem Jover:
> > One of the main change is that `.buildinfo` should now be named with an
> > arbitrary identifier. By default this defaults to $HOSTNAME-$TIMESTAMP
> > but can be set to an arbitrary value by the `--buildinfo-identifier`
> > command line flag.
>
> Hmmm, leaking the hostname seems slightly privacy-concerning? If the
> information therein is not relevant I'd rather use something like an
> UUID (although that would require increasing the pseudo-build-essential
> set), or just hashing the hostname-timestamp with something like md5
> or sha1 or similar.
My hunch is that having a timestamp visible in the file name might help
recognizing files quickly after a bunch of builds, especially to
identify the last one. So I'd rather keep it.
If privacy is the goal, hashing just the hostname would not be help
much, as any precomputed table would work.
How about $TIMESTAMP-$EIGHT_FIRST_CHARS_OF_BUILDINFO_MD5?
(I'm picking md5 because it's already used in dpkg-gensymbols.)
> Can we just simply use the package name rules instead? It also avoids
> potential problems with case and similar. (There's a
> pkg_name_is_illegal function in Dpkg::Package already.)
Sounds reasonable. I've updated the wiki page and prepared a patch for
dak.
> > +} else {
> > +warning(_g('no .dsc file, skipping .buildinfo generation'));
> > +}
> > }
>
> ISTR mentioning this before, but I don't see why generating the
> buildinfo file is tied to existing a source package at all? The source
> should be included if we are including a source in the upload, that's
> it.
The whole puprose of the reproducible builds effort is to provide a
verifiable path from sources to binaries. Signed .buildinfo files are
certifications that the listed binary packages have been built using the
described source and environment.
Only listing the source in .buildinfo when a source is included in the
upload would prevent us to have multiple builders certify the same
binaries. That would cut us from providing multiple certifications and
would undermine the purpose of reproducible builds.
So I could remove the limitation, but the resulting .buildinfo file
would not be very useful for reproducible builds.
> > +$fields->{'Source'} = $spackage;
> > +if ($changelog->{'Binary-Only'}) {
> > +$fields->{'Source'} .= ' (' . $sourceversion . ')';
> > +$fields->{'Changes'} = $changelog->{'Changes'} . "\n\n"
> > + . ' -- ' . $changelog->{'Maintainer'}
> > + . ' ' . $changelog->{'Date'};
> > +}
>
> Hmmm, it bothers me slightly that the Changes field here diverges in
> form from the one in the .changes file.
I can understand. It's been designed that way because it's actually only
there for binNMUs where the source is the same as the original and we
need a way to reconstruct the right changelog file.
Because sbuild might actually change its strings in the future, it felt
like plain copy/pasting was the safest. So recreating the changelog in
case of binNMU is about outputing the value of the Changes field in the
.buildinfo, a blank line, and the changelog from the original source.
> I think I'd prefer to have the Date as its own field, maybe always
> included. And also the Maintainer field. Any reason to not include
> them all the time or on their own?
I think they would be confusing. If we would to include the “Maintainer”
I guess we should call it “Changed-By” like in .changes. “Date”
as such would be a confusing name because I would tend to think of it
as the date of the build, and not as the date of the latest changelog
of a binNMU… So maybe “Changed-On” or “Change-Date”.
But this feels just more complicated than just the current
implementation, even if the format differs slightly. Maybe we can rename
that field instead to “Extra-Changelog-Entry” or something else so it's
clear they have different format?
> > +my $environment = Dpkg::Deps::AND->new();
> > +foreach my $pkg (sort keys %env_pkgs) {
> > +foreach my $installed_pkg (@{$facts->{pkg}->{$pkg}}) {
> > +my $version = $installed_pkg->{version};
> > +my $architecture = $installed_pkg->{architecture};
> > +my $pkg_name = $pkg;
>
> > +if (defined $architecture &&
> > +$architecture ne 'all' && $architecture ne $build_arch) {
> > +$pkg_name = "$pkg_name:$architecture";
> > +}
> […]
> Also this will include all Multi-Arch instances for a given package
> regardless of them being used or not, I don't think that's desirable.
Can we know for sure which one have been used?
I'm already working on other changes you suggested.
Thanks,
--
Lunar.''`.
lu...@debian.org: :Ⓐ : # apt-get install anarchism
`. `'`
`-
signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Repr
