diffoscope 269 released
Hi, The diffoscope maintainers are pleased to announce the release of version 269 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 269 includes the following changes: [ Chris Lamb ] * Allow Debian testing continuous integration builds to fail right now. [ Sergei Trofimovich ] * Amend 7zip version test for older versions that include the "[64]" string. (Closes: reproducible-builds/diffoscope#376) ## Download Version 269 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
diffoscope 268 released
Hi, The diffoscope maintainers are pleased to announce the release of version 268 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 268 includes the following changes: [ Chris Lamb ] * Drop apktool from Build-Depends; we can still test our APK code via autopkgtests. (Closes: #1071410) * Fix tests for 7zip version 24.05. * Add a versioned dependency for at least version 5.4.5 for the xz tests; they fail under (at least xz 5.2.8). (Closes: reproducible-builds/diffoscope#374) [ Vagrant Cascadian ] * Relax Chris' versioned xz test dependency (5.4.5) to also allow version 5.4.1. ## Download Version 268 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Debian NMU Sprint Thursday, June 6th 17:00 UTC!
Vagrant Cascadian wrote:
> I am hoping to schedule some Non-Maintainer Uploads (NMU) sprints,
> starting with two thursdays from now...
>
> Planning on meeting on irc.oftc.net in the #debian-reproducible channel
> at 17:00UTC and going for an hour or two or three. Feel free to start
> early or stay late, or even fix things on some other day!
See you there/then. :)
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 267 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 267 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 267 includes the following changes:
[ Chris Lamb ]
* Include "xz --verbose --verbose" (ie. double --verbose) output, not just
the single --verbose. (Closes: #1069329)
* Only include "xz --list" output if the xz has no other differences.
## Download
Version 267 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 266 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 266 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 266 includes the following changes:
[ Chris Lamb ]
* Use "xz --list" to supplement the output when comparing .xz archives;
essential when some underlying metadata differs. (Closes: #1069329)
* Actually append the xz --list after the container differences, as it
simplifies tests and the output.
* Add 7zip to Build-Depends in debian/control.
* Update copyright years.
[ James Addison ]
* Maintain an in-header boolean state to determine whether to drop
from-file/to-file lines. This fixes an issue where HTML differences were
being inadvertendly neglected. (Closes: reproducible-builds/diffoscope#372)
## Download
Version 266 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Reproducible Builds in April 2024
applied any
ameliorating fixes.
[10] https://www.openwall.com/lists/oss-security/2024/04/08/8
[11] https://www.openwall.com/lists/oss-security/
[12] https://www.openwall.com/lists/oss-security/2024/04/20/3
§
Website updates
---
There were a number of improvements made to our website this month,
including Chris Lamb updating the archive page [13] to recommend -X and
unzipping with TZ=UTC [14] and adding Maven, Gradle, JDK and Groovy
examples to the SOURCE_DATE_EPOCH page [15] [16]. In addition Jan
Zerebecki added a new /contribute/opensuse/ [17] page [18] and
Sertonix fixed the automatic RSS feed detection [19][20].
[13] https://reproducible-builds.org/docs/archive/
[14]
https://salsa.debian.org/reproducible-builds/reproducible-website/commit/d15f76b8
[15] https://reproducible-builds.org/docs/source-date-epoch/
[16]
https://salsa.debian.org/reproducible-builds/reproducible-website/commit/bfcbb9a2
[17] https://reproducible-builds.org/contribute/opensuse/
[18]
https://salsa.debian.org/reproducible-builds/reproducible-website/commit/4901c9ae
[19]
https://salsa.debian.org/reproducible-builds/reproducible-website/commit/5f311583
[20]
https://salsa.debian.org/reproducible-builds/reproducible-website/commit/54c80767
§
"Reproducible Builds and Insights from an Independent Verifier for Arch Linux"
--
Joshua Drexel, Esther Hänggi and Iyán Méndez Veiga of the School of
Computer Science and Information Technology, Hochschule Luzern (HSLU) in
Switzerland published a paper this month entitled "Reproducible Builds
and Insights from an Independent Verifier for Arch Linux" [22]. The
paper establishes the context as follows:
> Supply chain attacks have emerged as a prominent cybersecurity threat
> in recent years. Reproducible and bootstrappable builds have the
> potential to reduce such attacks significantly. In combination with
> independent, exhaustive and periodic source code audits, these measures
> can effectively eradicate compromises in the building process. In this
> paper we introduce both concepts, we analyze the achievements over the
> last ten years and explain the remaining challenges.
What is more, the paper aims to:
> … contribute to the reproducible builds effort by setting up a
> rebuilder and verifier instance to test the reproducibility of Arch
> Linux packages. Using the results from this instance, we uncover an
> unnoticed and security-relevant packaging issue affecting 16 packages
> related to Certbot […].
A PDF [23] of the paper is available.
[22] https://doi.org/10.18420/sicherheit2024_016
[23]
https://dl.gi.de/server/api/core/bitstreams/f8685808-2e51-4a53-acc0-2b45fa240e3b/content
§
libntlm now releasing 'minimal source-only tarballs'
Simon Josefsson [25] wrote on his blog this month that, going forward,
the libntlm [26] project will now be releasing what they call "minimal
source-only tarballs [27]":
> The XZUtils incident [28] illustrate that tarballs with files that are
> not included in the git archive offer an opportunity to disguise
> malicious backdoors. [The] risk of hiding malware is not the only
> motivation to publish signed minimal source-only tarballs. With pre-
> generated content in tarballs, there is a risk that GNU/Linux
> distributions [ship] generated files coming from the tarball into the
> binary *.deb or *.rpm package file. Typically the person packaging the
> upstream project never realized that some installed artifacts was
> not re-built[.]
Simon's post [29] goes into further details how this was achieved, and
describes some potential caveats and counters some expected responses as
well. A shorter version can be found in the announcement for the 1.8
release of libntlm [30].
[25] https://blog.josefsson.org/
[26] https://gitlab.com/gsasl/libntlm/
[27]
https://blog.josefsson.org/2024/04/13/reproducible-and-minimal-source-only-tarballs/
[28] https://en.wikipedia.org/wiki/XZ_Utils_backdoor
[29]
https://blog.josefsson.org/2024/04/13/reproducible-and-minimal-source-only-tarballs/
[30] https://lists.nongnu.org/archive/html/libntlm/2024-04/msg0.html
§
Distribution work
-
In Debian this month, Helmut Grohne filed a bug [31] suggesting the
removal of dh-buildinfo, a tool to generate and distribute .buildinfo-
like files within binary packages. Note that this is distinct from the
.buildinfo generation performed by dpkg-genbuildinfo. By contrast, the
entirely optional dh-buildinfo generated a debian/buildinfo file that
would be shipped within binary packages as
/usr/share/doc/package/buildinfo_$arch.gz.
In addition, 21 reviews of Debian pac
Re: Please review the draft for April's report
Chris Lamb wrote: > Please review the draft for April's Reproducible Builds report: This has now been published — thanks to all who contributed. :) If possible, please share the following link: https://reproducible-builds.org/reports/2024-04/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1788877087358476414 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for April's report
Hi all,
Please review the draft for April's Reproducible Builds report:
https://reproducible-builds.org/reports/2024-04/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2024-04.md
I intend to publish it no earlier than:
$ date -d 'Fri, 10 May 2024 10:00:00 +0100'
https://time.is/compare/1000_10_May_2024_in_BST
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2024-04.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2024-04.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: silx package from Debian
Vagrant Cascadian wrote: > Looks like it is probbaly some sort of sort ordering or randomness issue > in whatever is generating the documentation. Probably not silx specific, > but the tools it uses. Looking very very quickly, this looks like (at least): def get_unique_key(self): return str(uuid4()) ... in ./sphinx_panels/tabs.py in the sphinx-panels source package. Disabling parallelism won't help in this situation. Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Which conferences are folks attending these days?
Santiago Torres-Arias wrote:
> I wonder if co-location with another conf would help people prioritize
> the "right conf." It may just be a matter of signaling r-b presence in a
> conference as first class. For me, it'd help me choose which of the 20
> conferences that happen each year is the one I should aim for.
That's a great point. In retrospect, we really should perhaps have
made something bigger of the fact that the most recent RB summit was
reasonably close in time and space to PackagingCon, for example.
Anyway, we'll definitely keep this in mind when choosing future RB
locations and dates. The difficulty, as you no doubt are aware, is
that conference dates must perforce be picked long in advance, and
they are dictated in no small part by the blunt reality of the venue's
availability rather than organisers' good intentions. :(
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: diffoscope 265 released
Vagrant Cascadian wrote:
>> The diffoscope maintainers are pleased to announce the release of
>> version 265 of diffoscope.
>
> Signed tag please? :)
Salsa was having some connection issues earlier — scrollback indicates
that all interactions but the `git push --tags` succeeded. Done :)
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 265 released
Hi, The diffoscope maintainers are pleased to announce the release of version 265 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 265 includes the following changes: [ Chris Lamb ] * Ensure that tests with ">=" version constraints actually print the corresponding tool name. (Closes: reproducible-builds/diffoscope#370) * Prevent odt2txt tests from always being skipped due to an impossibly new version requirement. (Closes: reproducible-builds/diffoscope#369) * Avoid nested parens-in-parens when printing "skipping…" messages in the testsuite. ## Download Version 265 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Which conferences are folks attending these days?
Hey -general, I was talking to a bunch of RB folks yesterday, and we came to the loosely shared view that, after peak Covid and other industry-wide changes, conferences are no longer the "must attend" events they previously were… especially in the area of software supply-chain security. In rough, practical terms, it seems harder to justify conference travel today than it did in mid-2019. To that end, what conferences are folks on this list still going to, and, hopefully, still getting something from? I mean, there must be some exceptions other than FOSDEM… :) Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Reproducible Builds in March 2024
; Functional package managers (FPMs) and reproducible builds (R-B) are > technologies and methodologies that are conceptually very different > from the traditional software deployment model, and that have > promising properties for software supply chain security. This thesis > aims to evaluate the impact of FPMs and R-B on the security of the > software supply chain and propose improvements to the FPM model to > further improve trust in the open source supply chain. Full PDF: [15] Julien's paper poses a number of research questions on how the model of distributions such as GNU Guix [16] and NixOS [17] can "be leveraged to further improve the safety of the software supply chain", etc. [13] https://en.wikipedia.org/wiki/HAL_(open_archive [14] https://hal.science/hal-04482192 [15] https://hal.science/hal-04482192/document [16] https://guix.gnu.org/ [17] https://nixos.org/ [18] https://guix.gnu.org/ § Software and source code identification with GNU Guix [18] and reproducible builds -- In a long line of commendably detailed blog posts, Ludovic Courtès, Maxim Cournoyer, Jan Nieuwenhuizen and Simon Tournier have together published two interesting posts on the GNU Guix blog [19] this month. In early March, Ludovic Courtès, Maxim Cournoyer, Jan Nieuwenhuizen and Simon Tournier wrote about software and source code identification [20] and how that might be performed using Guix, rhetorically posing the questions: "What does it take to 'identify software'? How can we tell what software is running on a machine to determine, for example, what security vulnerabilities might affect it?" Later in the month, Ludovic Courtès wrote a solo post describing adventures on the quest for long-term reproducible deployment [21]. Ludovic's post touches on GNU Guix's aim to support "time travel", the ability to reliably (and reproducibly) revert to an earlier point in time, employing the iconic image of Harold Lloyd hanging off the clock in "Safety Last!" (1925) [22] to poetically illustrate both the slapstick nature of current modern technology and the gymnastics required to navigate hazards of our own making. [19] https://guix.gnu.org/en/blog/ [20] https://guix.gnu.org/en/blog/2024/identifying-software/ [21] https://guix.gnu.org/en/blog/2024/adventures-on-the-quest-for-long-term-reproducible-deployment/ [22] https://en.wikipedia.org/wiki/Safety_Last! § Two new Rust-based tools for post-processing determinism Zbigniew Jędrzejewski-Szmek announced "add-determinism" [23], a work-in- progress reimplementation of the Reproducible Builds project's own strip-nondeterminism [24] tool in the Rust programming language [25], intended to be used as a post-processor in RPM-based distributions such as Fedora [26] In addition, Yossi Kreinin [27] published a blog post titled "refix: fast, debuggable, reproducible builds" [28] that describes a tool that post-processes binaries in such a way that they are still debuggable with gdb [29], etc. Yossi post details the motivation and techniques behind the (fast) performance of the tool. [23] https://github.com/keszybz/add-determinism [24] https://salsa.debian.org/reproducible-builds/strip-nondeterminism [25] https://www.rust-lang.org/ [26] https://fedoraproject.org/ [27] https://yosefk.com/ [28] https://yosefk.com/blog/refix-fast-debuggable-reproducible-builds.html [29] https://sourceware.org/gdb/ § Distribution work - In Debian this month, since the testing framework no longer varies the build path [30], James Addison performed a bulk downgrade of the bug severity [31] for issues filed with a level of normal to a new level of wishlist. In addition, 28 reviews of Debian packages were added, 38 were updated and 23 were removed this month adding to ever-growing knowledge about identified issues [32]. As part of this effort, a number of issue types were updated, including Chris Lamb adding a new ocaml_include_directories toolchain issue [33] and James Addison adding a new filesystem_order_in_java_jar_manifest_mf_include_resource issue [34] and updating the random_uuid_in_notebooks_generated_by_nbsphinx to reference a relevant discussion thread [35]. [30] https://reproducible-builds.org/docs/build-path/ [31] https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/003257.html [32] https://tests.reproducible-builds.org/debian/index_issues.html [33] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/a052c30f [34] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/cc94c935 [35] https://salsa.debian.org/reproducible-builds/reproducible-notes/commit/55497f89 In addition, Roland Clobus posted
Re: Please review the draft for March's report
Holger Levsen wrote: > I also like the final order of the entries, though when I skimmed > through https://reproducible-builds.org/reports/2024-03/ I wondered > whether we should add a table of contents to the top of each post? > > What do y'all think? I used to always add one one, but haven't for a while. Still, the bumper amount of news in this month's report month does seem to ask for one... Added and pushed. Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 264 released
Hi, The diffoscope maintainers are pleased to announce the release of version 264 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 264 includes the following changes: [ Chris Lamb ] * Don't crash on invalid zipfiles, even if we encounter 'badness' through through the file. (Re: #1068705) [ FC (Fay) Stegerman ] * Add note when there are duplicate entries in ZIP files. (Closes: reproducible-builds/diffoscope!140) [ Vagrant Cascadian ] * Add an external tool reference for GNU Guix for zipdetails. ## Download Version 264 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Please review the draft for March's report
Chris Lamb wrote: > Please review the draft for March's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2024-03/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1778496263027093713 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Please review the draft for March's report
Holger Levsen wrote: > On Wed, Apr 10, 2024 at 10:02:56AM -0400, David A. Wheeler via rb-general > wrote: >> I agree, this one is HUGE news. There's been a lot of awesome work related >> to reproducible builds, but "minimal container userland is a 100% >> reproducible build in a real-world widely-used distro" is a big step forward >> and should be widely announced. > > agreed. > > I also think the news about Vagrant helping Debian to confirm the xz related > builds have been fine, deserves a bigger headline. Thank you for all the feedback so far. Unless someone makes these changes to the draft themselves, I will attend to this (and all the other critiques here and on Salsa) before publishing. Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for March's report
Hi all,
Sorry for the delay in getting this out — it was, quite genuinely, a
bumper amount of things that needed condensing, rewriting and
generally getting into readable shape. Anyway, if folks would be so
kind as to review the draft for last months report here:
https://reproducible-builds.org/reports/2024-03/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2024-03.md
I intend to publish it no earlier than:
$ date -d 'Thu, 11 Apr 2024 17:30:00 +0100'
https://time.is/compare/1730_11_Apr_2024_in_BST
§
As ever, please feel free and commit/push to drafts directly without the
overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2024-03.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2024-03.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Three bytes in a zip file
Larry Doolittle wrote:
> Yes. The -X isn't needed, sometimes, and then when you least expect it, it
> is.
> Classic reproducible-builds gotcha.
>
>> I'm happy to update this document myself if need be. :)
>
> Go for it.
Finally got around to doing this. Many thanks. :)
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 263 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 263 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 263 includes the following changes:
[ Chris Lamb ]
* Add support for the zipdetails(1) tool included in the Perl distribution.
Thanks to Larry Doolittle et al. for the pointer to this tool.
* Don't use parenthesis within test "skipping…" messages; PyTest adds its own
parenthesis, so we were ending up with double nested parens.
* Fix the .epub tests after supporting zipdetails(1).
* Update copyright years and debian/tests/control.
[ FC (Fay) Stegerman ]
* Fix MozillaZipContainer's monkeypatch after Python's zipfile module changed
to detect potentially insecure overlapping entries within .zip files.
(Closes: reproducible-builds/diffoscope#362)
## Download
Version 263 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Two questions about build-path reproducibility in Debian
James Addison wrote: > None of the remaining thirty-or-so (and in fact, none of the 66 updated so > far) > are usertagged both 'buildpath' and 'toolchain'. > > I would say that a few of them _are_ 'toolchain packages' -- mono, > binutils-dev > and a few others -- but for these bugs the buildpath issues are internal to > each package at build-time and do not affect the construction of other > packages in their ecosystem. You are absolutely right to distinguish between a package that is itself unreproducible and a package that is causing other packages to be unreproducible. These are very much orthogonal concepts as you imply, and a package can certainly be in both categories at once. What might be confusing to folks is that our "toolchain" usertag in the Debian BTS does not refer to a toolchain *package* in the usual, Debian sense, i.e. Mono, libc, Bison, documentation generators and so on. But rather that (loosely speaking) "if this usertag is applied to a bug, its denoting that that particular *bug* is affecting the reproducibility of other packages." Unfortunately, the tag is actually an excellent example of that general trend in tech where something was badly named in the spur of the moment, and then the name just sticks around forever due to some combination of muscle memory, inertia and, frankly, priority: as in, this metadata is not *all* that visible nor A++ important to begin with… outside of threads like this. :) Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Two questions about build-path reproducibility in Debian
Hi James, > Approximately thirty are still set to other severity levels, and I plan to > update those with the following adjusted messaging […] Looks good to me. :) Completely out of interest, are any of those 30 bugs tagged both "buildpath" and "toolchain"? It's written nowhere in Policy (and I can't remember if it's ever been discussed before), but if package X is causing package Y to be unreproducible, I feel that has some bearing on the severity of the bug for that issue filed against X… completely independent of whether package X is reproducible itself or not. :) Just to underscore that this is simply my curiosity before you reassign: in the particular case of *buildpath* AND toolchain, these should almost certainly be wishlist anyway because, as discussed, we "aren't testing buildpath". Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 262 released
Hi, The diffoscope maintainers are pleased to announce the release of version 262 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 262 includes the following changes: [ Chris Lamb ] * Factor out Python version checking in test_zip.py. (Re: #362) * Also skip some zip tests under 3.10.14 as well; a potential regression may have been backported to the 3.10.x series. The underlying cause is still to be investigated. (Re: #362) ## Download Version 262 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Arch Linux minimal container userland 100% reproducible - now what?
Hey kpcyrd,
Super excited about the energy in this thread. :)
I'll probably reply to a different part of the conversation
tomorrow, but just to very quickly append something to this bit:
> This kind of [archive] service is crucial for implementing
> reproducible builds (because this is used to setup the build
> environment described in BUILDINFO files), and
> reproducible-builds.org has recently received $350k to implement an
> analogous service for Debian (to be able to catch up with Arch
> Linux).
I think h01ger already talked to you a bit on IRC, but the long and
short of it is that, well, reproducible-builds.org wishes it had those
kind of resources to dedicate towards building such a service! Yes, we
did manage to secure some funding recently, and no doubt some of that
will help kickstart an analogous snapshot service. But the amount,
timeframe and associated deliverables don't quite, alas, match your
summary. Still, the important thing here is that your passion is
infectious. :)
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 261 released
Hi, The diffoscope maintainers are pleased to announce the release of version 261 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 261 includes the following changes: [ Chris Lamb ] * Don't crash if we encounter an .rdb file without an equivalent .rdx file. (Closes: #1066991) * In addition, don't identify Redis database dumps (etc.) as GNU R database files based simply on their filename. (Re: #1066991) * Update copyright years. ## Download Version 261 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Three bytes in a zip file
Hey, > echo "Forcing timestamp $SOURCE_DATE_EPOCH" > touch --date="@$SOURCE_DATE_EPOCH" fab/* > TZ=UTC zip -X --latest-time "$zipfile" fab/* > # Note the -X flag; to be pedantic about timestamps, > # that means you should unpack with TZ=UTC unzip "$zipfile". See > # > https://lists.reproducible-builds.org/pipermail/rb-general/2023-April/002927.html Ah, interesting! Does that -X mean that https://reproducible-builds.org/docs/archives/ ... is incomplete? I'm happy to update this document myself if need be. :) Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Reproducible Builds in February 2024
challenging and costly to perform manually. > (HAL Portal [18], full PDF [19]) [16] https://inria.hal.science/hal-04441579v2 [17] https://www.inria.fr/en/inria-centre-rennes-university [18] https://inria.hal.science/hal-04441579v2 [19] https://inria.hal.science/hal-04441579/file/msr24.pdf § Mailing list highlights --- >From our mailing list [20] this month: * User "cen" posted a query asking "How to verify a package by rebuilding it locally on Debian [21]" which received a followup from Vagrant Cascadian [22]. * James Addison asked "Two questions about build-path reproducibility in Debian [23]" regarding the differences in the testing performed by Debian's GitLab continuous integration (CI) pipeline [24] and the Debian-specific testing performed by the Reproducible Builds project itself [25], and followed this with a separate but related question regarding misconfigured *reprotest* [26] configurations. [20] https://lists.reproducible-builds.org/listinfo/rb-general/ [21] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003238.html [22] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003240.html [23] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003246.html [24] https://salsa.debian.org/salsa-ci-team/pipeline [25] https://tests.reproducible-builds.org/debian/reproducible.html [26] https://salsa.debian.org/reproducible-builds/reprotest § Distribution work - In Debian this month, 5 reviews of Debian packages were added, 22 were updated and 8 were removed this month adding to Debian's knowledge about identified issues [27]. A number of issue types were updated as well. In addition, Roland Clobus posted his 23rd update of the status of reproducible ISO images [28] on our mailing list. In particular, Roland helpfully summarised that "all major desktops build reproducibly with "bullseye", "bookworm", "trixie" and "sid" provided they are built for a second time within the same DAK run (i.e. [within] 6 hours)" and that there will likely be further work at a MiniDebCamp in Hamburg [29]. Furthermore, Roland also responded in- depth [30] to a query about a previous report [31]. [27] https://tests.reproducible-builds.org/debian/index_issues.html [28] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003251.html [29] https://wiki.debian.org/DebianEvents/de/2024/MiniDebCampHamburg [30] https://lists.reproducible-builds.org/pipermail/rb-general/2024-February/003233.html [31] https://lists.reproducible-builds.org/pipermail/rb-general/2024-January/003217.html Fedora [32] developer Zbigniew Jędrzejewski-Szmek [33] announced a work- in-progress script called fedora-repro-build [34] that attempts to reproduce an existing package within a koji [35] build environment. Although the projects' README file [36] lists a number of "fields will always or almost always vary" and there is a non-zero list of other known issues [37], this is an excellent first step towards full Fedora reproducibility. [32] https://fedoraproject.org/ [33] https://github.com/keszybz [34] https://github.com/keszybz/fedora-repro-build [35] https://pagure.io/koji/ [36] https://github.com/keszybz/fedora-repro-build#readme [37] https://pagure.io/fedora-reproducible-builds/project/issues?tags=irreproducibility Jelle van der Waa introduced a new linter rule [38] for Arch Linux [39] packages in order to detect cache files leftover by the Sphinx documentation generator [40] which are unreproducible by nature and should not be packaged. At the time of writing, 7 packages in the Arch repository are affected by this. [38] https://gitlab.archlinux.org/pacman/namcap/-/merge_requests/64 [39] https://archlinux.org/ [40] https://www.sphinx-doc.org/en/master/ Elsewhere, Bernhard M. Wiedemann posted another monthly update [41] for his work elsewhere in openSUSE. [41] https://lists.opensuse.org/archives/list/fact...@lists.opensuse.org/thread/I66U56F5R3TR4ZTLYGPSGWINNOLZ7XP4/ § diffoscope -- diffoscope [43] is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes such as uploading versions 256, 257 and 258 to Debian and made the following additional changes: * Use a deterministic name instead of trusting gpg's --use-embedded- filenames. Many thanks to Daniel Kahn Gillmor (dkg) for reporting this issue and providing feedback. [44][45] * Don't error-out with a traceback if we encounter struct.unpack- related errors when parsing Python .pyc files. (#1064973). [47] * Don't try and compare rdb_expected_diff on non-GNU systems as %p formatting can vary, especially with res
Re: Please review the draft for February's report
Chris Lamb wrote: > Please review the draft for February's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2024-02/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1766508612887744550 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Three bytes in a zip file
Hey Fay, > The original reproducibility issue this thread started with was traced back to > the atime back then, my tool just hopefully makes doing that a bit easier :) Oh! I somehow missed that this was an atime-related issue at the time… in addition to missing your PoC to call out to repro-apk from diffoscope as well. Sorry about both of those. Learning that it was an atime issue was especially interesting as it connected a few things in my head, including the fact that I could never reproduce a few .zip-related issues in the past — which I now realise is because I mount my filesystems with noatime. I'll implement zipdetails support shortly. As you say, it will be quicker/straightforward to integrate. :) Chris > I almost forgot: zipdetails (that comes with perl) can also show this > difference > (and quite a lot of other things, though its output is not usually so easy to > diff which is why I tend to prefer my own tools -- diff-zip-meta, zipinfo.py, > apksigtool -- but it might be easier to use that for diffoscope, at least for > now): > > $ diff -Naur <( zipdetails atime1.zip ) <( zipdetails atime2.zip ) > @@ -15,7 +15,7 @@ > 0023 Length 0009 > 0025 Flags '03 mod access' > 0026 Mod Time65EB87EA 'Fri Mar 8 22:49:30 2024' > -002A Access Time 65EB87EA 'Fri Mar 8 22:49:30 2024' > +002A Access Time 65EB87EE 'Fri Mar 8 22:49:34 2024' > 002E Extra ID #00027875 'ux: Unix Extra Type 3' > 0030 Length 000B > 0032 Version 01 -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 260 released
Hi, The diffoscope maintainers are pleased to announce the release of version 260 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 260 includes the following changes: [ Chris Lamb ] * Actually test 7z support in the test_7z set of tests, not the lz4 functionality. (Closes: reproducible-builds/diffoscope#359) * In addition, correctly check for the 7z binary being available (and not lz4) when testing 7z. * Prevent a traceback when comparing a contentful .pyc file with an empty one. (Re: Debian:#1064973) ## Download Version 260 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Three bytes in a zip file
Hey Fay, Oh this is great work! So, using your tool, did you manage to solve the underlying non-determinism? :) Based on the output (which labels the field as an 'extra atime' or similar), it seems like you've managed to work out which part of your toolchain is making the build reproducible — or am I being too optimistic? Best wishes, Chris ps. Separate to that, how amenable would you be to working with me getting this extra .ZIP metadata support built directly into diffoscope at some point…? Fay Stegerman wrote: > * Larry Doolittle [2023-04-06 23:59]: >> Do you know of any tooling that can help decode zip file contents in general? > > I know this thread is almost a year old now, but I finally got to my backlog > working on diff-zip-meta.py [1], which is a tool specifically intended to > elucidated differences in ZIP/APK metadata. And as of today, the master > branch > supports showing the kind of timestamp differences you reported in > human-readable form, not just as a difference in the raw data: > > $ diff-zip-meta foo.zip bar.zip > --- foo.zip > +++ bar.zip > entry foo: > extra (entry): > - 55540900035164ea655164ea6575780b000104e80304e803 > + 55540900035164ea655464ea6575780b000104e80304e803 > - extra (entry) atime=2024-03-08 01:05:21 > + extra (entry) atime=2024-03-08 01:05:24 > > - Fay > > [1] https://github.com/obfusk/reproducible-apk-tools#diff-zip-metapy -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for February's report
Hi all, Please review the draft for February's Reproducible Builds report: https://reproducible-builds.org/reports/2024-02/?draft ⦠or, via the Git repository itself: https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2024-02.md I intend to publish it no earlier than: $ date -d 'Sat, 09 Mar 2024 14:15:00 +' https://time.is/compare/1415_09_Mar_2024_in_GMT § Please feel free and commit/push to drafts directly without the overhead of sending patches or merge requests. You should make your changes to the "_reports/2024-02.md" file in the "reproducible-website" repository: $ git clone https://salsa.debian.org/reproducible-builds/reproducible-website $ cd reproducible-website $ sensible-editor _reports/2024-02.md I am happy to reword and/or rework additions prior to publishing. If you currently do not have access to the above repository, you can request access by following the instructions at: https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Re: reprotest: inadvertent misconfiguration in salsa-ci config
James Addison wrote: > I've opened a merge request[1] to explore this error-treatment approach; it > lacks useful error messaging so far, but I'll attempt to add that soon. In your enthusiasm I think you neglected to included the actual "[1]" URL later in your mail. However, allow me to do that for you: [1] https://salsa.debian.org/reproducible-builds/reprotest/-/merge_requests/23 Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 259 released
Hi, The diffoscope maintainers are pleased to announce the release of version 259 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 259 includes the following changes: [ Chris Lamb ] * Don't error-out with a traceback if we encounter "struct.unpack"-related errors when parsing .pyc files. (Closes: #1064973) * Fix compatibility with PyTest 8.0. (Closes: reproducible-builds/diffoscope#365) * Don't try and compare rdb_expected_diff on non-GNU systems as %p formatting can vary. (Re: reproducible-builds/diffoscope#364) ## Download Version 259 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Re: Two questions about build-path reproducibility in Debian
Vagrant Cascadian wrote:
> There are real-world build path issues, and while it is possible to work
> around them in various ways, I think they are still issues worth fixing
> to make it easier to debug other issues, although deprioritizing them
> makes sense, given buildd.debian.org now normalizes them.
+1.
And for this reason, I think we should keep the buildpath-related
bugs as well. They should all be 'wishlist' priority anyway, and I
wouldn't like to bet my hat that the usertag metadata is accurate and
comprehensive enough to blindly close them in the first place. (We
only really used the usertags to do some rough-and-ready statistics
on broad issue categories.)
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: reprotest: inadvertent misconfiguration in salsa-ci config
Vagrant Cascadian wrote: > This almost makes me want to entirely deprecate --variations, and switch > to recommending "--vary=-all,+whatever" or "--vary=-all > --vary=+whatever" instead of ever using --variations. This is also a very tempting option. I mean, if we're going to emit an error (ie. break some existing configurations), then we might as properly fix the core of this UI issue. And this would also save us working out which --variations invocations are "bad" and which are acceptable. > I'm not sure the variations syntax enables much that cannot be more > unambiguously expressed with --vary. Indeed. And, y'know, if there was a call for it, we could add a new and less confusing version of the --variations option under a different, unambiguous name — perhaps something like --only-vary=a,b. > I am not sure what sort of refactoring will be needed to make this > possible. In particular, how --auto-build is implemented […] (I think this is implemented internally to reprotest, on a different abstraction layer to the command-line argument handling.) Any strong opinions from elsewhere...? Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: reprotest: inadvertent misconfiguration in salsa-ci config
Hi James,
Great post, thank you. So, I'm in two minds re. the way forward:
> * Update reprotest to handle a single-disabled-varations-value as a
> special case - treating it as vary and/or emitting a warning.
On whether to magically/transparently fix this, needless to say, it's
considered bad practice to change the behaviour of software that has
already been released — I would, as a rule, subscribe to that idea.
However, we should bear in mind that this idea revolves around what
users are *expecting*, not necessarily what the software actually
does.
I say that because I hazard that all 400 usages are indeed expecting
that `--variations=-foo` functions the same as `--variations=all,-foo`
(or `--vary=-foo`), and so this proposed change would merely be
modifying reprotest to reflect their existing expectations. It would
not therefore be a violation of the "don't break existing
functionality" dictum.
(Saying that, the addition of a warning that we are doing so would
definitely not go amiss.)
> * Treat removal of a variance factor from an already-empty-context
> as an error.
I'm also tempted by this as well. :) How would this be experienced by
most DDs? Would their new pushes to Salsa now suddenly fail in the
reprotest job of the pipeline? If so, that's not too awful, given that
the prominent error message would presumably let them know precisely
how to fix it.
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 258 released
Hi, The diffoscope maintainers are pleased to announce the release of version 258 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 258 includes the following changes: [ Chris Lamb ] * Use the 7zip package (over p7zip-full) after package transition. (Closes: #1063559) * Update debian/tests/control. [ Vagrant Cascadian ] * Fix a typo in the package name field (!) within debian/changelog. ## Download Version 258 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
diffoscope 257 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 257 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 257 includes the following changes:
[ James Addison ]
* Parse the header and hunksize of diffs strictly before parsing the context
below. (Closes: reproducible-builds/diffoscope#363)
* Reformat code to comply with the latest version of Black (24.1.1).
[ Chris Lamb ]
* Expand the previous changelog entry to include the CVE number that was
subsequently assigned.
* Bump the miniumum Black requirement to run the "Black clean" test and make
test_zip.py Black clean.
## Download
Version 257 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 256 released
Hi, The diffoscope maintainers are pleased to announce the release of version 256 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 256 includes the following changes: * Use a determistic name when extracting content from GPG artifacts instead of trusting the value of gpg's --use-embedded-filenames. This prevents a potential information disclosure vulnerability that could have been exploited by providing a specially-crafted GPG file with an embedded filename of, say, "../../.ssh/id_rsa". Many thanks to Daniel Kahn Gillmor for reporting this issue and providing feedback. (Closes: reproducible-builds/diffoscope#361) * Temporarily fix support for Python 3.11.8 re. a potential regression with the handling of ZIP files. (See reproducible-builds/diffoscope#362) ## Download Version 256 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Reproducible builds in January 2024
o ⬋ ⬊ January 2024 in Reproducible Builds o o ⬊ ⬋ https://reproducible-builds.org/reports/2024-01/ o Welcome to the January 2024 report from the Reproducible Builds project. In these reports we outline the most important things that we have been up to over the past month. If you are interested in contributing to the project, please visit our 'Contribute' [1] page on our website. [1] https://reproducible-builds.org/contribute/ § "How we executed a critical supply chain attack on PyTorch" --- John Stawinski [2] and Adnan Khan [3] published a lengthy blog post detailing how they executed a supply-chain attack [4] against PyTorch [5], a popular machine learning platform "used by titans like Google, Meta, Boeing, and Lockheed Martin": > Our exploit path resulted in the ability to upload malicious PyTorch > releases to GitHub, upload releases to [Amazon Web Services], > potentially add code to the main repository branch, backdoor PyTorch > dependencies – the list goes on. In short, it was bad. Quite bad. The attack pivoted on PyTorch's use of "self-hosted runners [7]" as well as submitting a pull request to address a trivial typo in the project's README file to gain access to repository secrets and API keys that could subsequently be used for malicious purposes. [2] https://johnstawinski.com/ [3] https://adnanthekhan.com/ [4] https://johnstawinski.com/2024/01/11/playing-with-fire-how-we-executed-a-critical-supply-chain-attack-on-pytorch/ [5] https://pytorch.org/ [7] https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners § New Arch Linux forensic filesystem tool --- On our mailing list [8] this month, long-time Reproducible Builds developer kpcyrd announced a new tool [9] designed to forensically analyse Arch Linux [10] filesystem images. Called archlinux-userland-fs-cmp [11], the tool is "supposed to be used from a rescue image (any Linux) with an Arch install mounted to, [for example], /mnt." Crucially, however, "at no point is any file from the mounted filesystem eval'd or otherwise executed. Parsers are written in a memory safe language." More information about the tool can be found on their announcement message [12], as well as on the tool's homepage [13]. A GIF of the tool in action [14] is also available. [8] https://lists.reproducible-builds.org/pipermail/rb-general/ [9] https://lists.reproducible-builds.org/pipermail/rb-general/2024-January/003232.html [10] https://archlinux.org/ [11] https://github.com/kpcyrd/archlinux-userland-fs-cmp [12] https://lists.reproducible-builds.org/pipermail/rb-general/2024-January/003232.html [13] https://github.com/kpcyrd/archlinux-userland-fs-cmp [14] https://asciinema.org/a/MFefYEdvU2O5LlIzseQnyBky5 § Issues with our SOURCE_DATE_EPOCH code? --- Chris Lamb started a thread on our mailing list [15] summarising some potential problems with the source code snippet the Reproducible Builds project has been using to parse the SOURCE_DATE_EPOCH [16] environment variable: > I'm not 100% sure who originally wrote this code, but it was probably > sometime in the ~2015 era, and it must be in a huge number of codebases > by now. > > Anyway, Alejandro Colomar was working on the shadow security tool and > pinged me regarding some potential issues with the code. You can see > this conversation here: [17]. Chris ended his message with a request that those with intimate or low- level knowledge of time_t, C types, overflows and the various parsing libraries in the C standard library (etc.) contribute with further info. [15] https://lists.reproducible-builds.org/pipermail/rb-general/2024-January/003225.html [16] https://reproducible-builds.org/docs/source-date-epoch/ [17] https://github.com/shadow-maint/shadow/commit/cb610d54b47ea2fc3da5a1b7c5a71274ada91371#r136407772 § Distribution updates In Debian this month, Roland Clobus posted another detailed update of the status of reproducible ISO images [18] on our mailing list. In particular, Roland helpfully summarised that "all major desktops build reproducibly with bullseye, bookworm, trixie and sid provided they are built for a second time within the same DAK run (i.e. [within] 6 hours)". Additionally 7 of the 8 bookworm images from the official download link [19] build reproducibly at any later time. In addition to this, three reviews of Debian packages were added, 1
Re: Please review the draft for January's report
Chris Lamb wrote: > Please review the draft for January's Reproducible Builds report: This has now been published — thanks to all who contributed. We didn't know what to do with the FOSDEM stuff (technically February, not January so I will do a separate post tomorrow). Anyway, please share the following link: https://reproducible-builds.org/reports/2024-01/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1755356173442965599 Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for January's report
Hey folks,
Please glance over the draft for January's Reproducible Builds report:
https://reproducible-builds.org/reports/2024-01/?draft
… which you can also do via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2024-01.md
I intend to publish it no earlier than:
$ date -d 'Wed, 07 Feb 2024 14:15:00 -0800'
https://time.is/compare/1415_07_Feb_2024_in_PST
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2024-01.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ vi _reports/2024-01.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 255 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 255 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 255 includes the following changes:
[ Vekhir ]
* Add/fix compatibility for Python progressbar 2.5 & 3.0 etc.
[ Chris Lamb ]
* Update copyright years.
## Download
Version 255 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Potential issues with the snippet to parse SOURCE_DATE_EPOCH in C
Hey folks, We've been using the following snippet of code to parse the value of SOURCE_DATE_EPOCH in the C programming language: https://reproducible-builds.org/docs/source-date-epoch/#c I'm not 100% sure who originally wrote this code, but it was probably sometime in the ~2015 era, and it must be in a huge number of codebases by now. Anyway, Alejandro Colomar was working on the shadow security tool and pinged me regarding some potential issues with the code. You can see this conversation here: https://github.com/shadow-maint/shadow/commit/cb610d54b47ea2fc3da5a1b7c5a71274ada91371#r136407772 ... but for context here, it kicks off with: > Was there any reason to reject >ULONG_MAX? I'm touching this code, > and don't see a reason for it; it looks very arbitrary; especially > since some systems can have 32-bit long, but 64-bit time_t. Should I > just drop that check, or keep it? And why? As you can see from the exchange (re. 32-bit/64-bit time_t etc.), I am not 100% sure of myself in this area. But I did promise I'd bring it up here on this list and solicit input, etc. Note that Alejandro goes onto say: > I have written a set of fixes for this function, in this PR: #893 > (still a draft) > [See: > https://github.com/shadow-maint/shadow/commit/cb610d54b47ea2fc3da5a1b7c5a71274ada91371#r136481852 > ] … and on that PR, Alejandro writes: > I was wondering... maybe I could write a library, libgetnum, and add > these functions there. That way, all those code bases in Debian that > have the problem we found in gettime() could be fixed easily by just > calling getnum() internally as I did here. > > I also fixed so many other hidden bugs in this PR, that other > projects could benefit from such a library. Speaking just for myself, I worry that most codebases won't rush to introduce a new library dependency "just" to parse this value (even if they should), but no doubt we should iron out any problems in the snippet regardless. Either way, I hope some folks who are better at this kind of C / systems programming can jump in and iron out any potential issues with Alejandro. Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 254 released
Hi, The diffoscope maintainers are pleased to announce the release of version 254 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 254 includes the following changes: [ Chris Lamb ] * Reflow some code according to black. [ Seth Michael Larson ] * Add support for comparing the 'eXtensible ARchive' (.XAR/.PKG) file format. [ Vagrant Cascadian ] * Add external tool on GNU Guix for 7z. ## Download Version 254 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Reproducible builds in December 2023
o ⬋ ⬊ December 2023 in Reproducible Builds o o ⬊ ⬋ https://reproducible-builds.org/reports/2023-12/ o Welcome to the December 2023 report from the Reproducible Builds [0] project! In these reports we outline the most important things that we have been up to over the past month. As a rather rapid recap, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries (more info: [1]). [0] https://reproducible-builds.org [1] https://reproducible-builds.org/#why-does-it-matter § "Reproducible Builds: Increasing the Integrity of Software Supply Chains" awarded IEEE Software "Best Paper" award - In February 2022, we announced in these reports [2] that a paper written by Chris Lamb [3] and Stefano Zacchiroli [4] was now available in the March/April 2022 issue of IEEE Software [5]. Titled "Reproducible Builds: Increasing the Integrity of Software Supply Chains" [6] (PDF [7]). This month, however, IEEE Software [8] announced that this paper has won their Best Paper award [9] for 2022. [2] https://reproducible-builds.org/reports/2023-02/ [3] https://chris-lamb.co.uk [4] https://upsilon.cc/~zack/ [5] https://ieeexplore.ieee.org/abstract/document/9403390 [6] https://arxiv.org/abs/2104.06020 [7] https://arxiv.org/pdf/2104.06020 [8] https://www.computer.org/csdl/magazine/so [9] https://twitter.com/ieeesoftware/status/1736684911690436868 § Reproducibility to affect package migration policy in Debian In a post summarising the activities of the Debian Release Team [10] at a recent in-person Debian event in Cambridge, UK [11], Paul Gevers announced a change to the way packages are "migrated" into the staging area for the next stable Debian release based on its reproducibility status: > The folks from the Reproducibility Project have come a long way since they started working on it 10 years ago, and we believe it's time for the next step in Debian. Several weeks ago, we enabled a migration policy in our migration software that checks for regression in reproducibility. At this moment, that is presented as just for info, but we intend to change that to delays in the not so distant future. We eventually want all packages to be reproducible. To stimulate maintainers to make their packages reproducible now, we'll soon start to apply a bounty [speedup] for reproducible builds, like we've done with passing autopkgtests [12] for years. We'll reduce the bounty for successful autopkgtests at that moment in time. [10] https://wiki.debian.org/Teams/ReleaseTeam [11] https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge [12] https://people.debian.org/~eriberto/README.package-tests.html § Speranza: "Usable, privacy-friendly software signing" - Kelsey Merrill, Karen Sollins, Santiago Torres-Arias and Zachary Newman have developed a new system called Speranza, which is aimed at reassuring software consumers that the product they are getting has not been tampered with and is coming directly from a source they trust. A write-up on TechXplore.com [13] goes into some more details: > "What we have done," explains Sollins, "is to develop, prove correct, and demonstrate the viability of an approach that allows the [software] maintainers to remain anonymous." Preserving anonymity is obviously important, given that almost everyone—software developers included—value their confidentiality. This new approach, Sollins adds, "simultaneously allows [software] users to have confidence that the maintainers are, in fact, legitimate maintainers and, furthermore, that the code being downloaded is, in fact, the correct code of that maintainer." [14] The corresponding paper [15] is published on the arXiv [16] preprint server in various formats, and the announcement has also been covered in MIT News [17]. [13] https://techxplore.com/news/2023-12-boosting-faith-authenticity-source-software.html [14] https://techxplore.com/news/2023-12-boosting-faith-authenticity-source-software.html [15] https://arxiv.org/abs/2305.06463 [16] https://arxiv.org/ [17] https://news.mit.edu/2023/speranza-boosting-faith-authenticity-open-source-software-1211 § Nondeterministic Git bundles Paul Baecher [18] published an interesting blog post on "Reproducible git bundles" [19]. For those who are not familiar with them,
Re: Please review the draft for December's report
Chris Lamb wrote: > Please review the draft for December's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-12/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1745532388199764442 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for December's report
Hi all,
Please review the draft for December's Reproducible Builds report:
https://reproducible-builds.org/reports/2023-12/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-12.md
I intend to publish it no earlier than:
$ date -d 'Thu, 11 Jan 2024 19:00:00 +'
https://time.is/compare/1900_11_Jan_2024_in_GMT
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-12.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-12.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Updates on rb-general list moderation and Code of Conduct development
Hello, First off, the core R-B team would like to extend a genuine thank you to list members who have contacted us with suggestions, preferences and exemplars related to establishing a Code of Conduct (CoC) for the Reproducible Builds project We are writing with two updates today, one regarding list moderation, and the other on the CoC development process: 1) List moderation: We are deeply appreciative of those who have taken time to engage with us 1-1 to discuss your experiences of the list, offer your perspectives on norms & respectful behavior, and visions for how we can create the most productive and inclusive environment for collaborating on Reproducible Builds. We believe we have reached a point where we can turn off moderation on the rb-general list. Posts have returned to the constructive and respectful tone that we have taken for granted in the past. We thank everyone for your patience and understanding while the list was on moderation, and we ask everyone to maintain a constructive and respectful tone and content moving forward. In the spirit of transparency, we must note that one list member declined to engage in dialogue regarding posts of theirs which generated a number of complaints both on- as well as off-list. That individual is still subscribed to rb-general, albeit without posting privileges, with the understanding that we are happy to be in dialogue at any time about reaching a shared understanding regarding appropriate list behaviour. 2) Code of Conduct (CoC) development: The core RB team has had initial discussions about the process and timeline for CoC development. We also have a series of meetings scheduled from January onward to first articulate the CoC development process… and to be implemented after community feedback. We'll post more on this in January. In the meantime, we wish all of you a fine rest of 2023, and excellent (and reproducible…) 2024! Regards, — Chris, Gunner, Holger, Mattia & Vagrant Chris Lamb wrote: > Dear all, > > Thank you for your patience, understanding and support in the past > few days. > > We recently placed the rb-general mailing list under close moderation > after a number of community members expressed concerns about the tone > and content of some threads. Since that time, no new messages have > been permitted on the list. > > However, we've also been using this time to reach out to other members > of the community and have been listening to their feedback to ensure > we understood all perspectives involved. > > We still fully intend to formalise a Code of Conduct for our mailing > list and other communication channels. However, we do not wish to > continue blocking all discussions pending this announcement. > > --> We are therefore moving the rb-general mailing to a "rolling" > moderation model and appraising individual posts as they come in. <-- > > Because of this, may notice a slight delay in your messages before > they are distributed to others, as well as a small influx of messages > as we process the slight backlog. We will also be holding back > messages and threads that, to the best of our belief, are not up to > the standards we should be setting for our community or may be > unproductive during this particular time. > > As before, if you have any questions or have input on what you > consider to be exemplary policies, practices and norms for > communities such as ours, please email rb-c...@lists.reproducible-builds.org. > > Although we are making this change today, we want to reinforce that > a formal CoC is incoming and that list participants should treat > other members of the community with respect and always help foster a > welcoming environment for all. > > > Thanks, > > — Chris, Holger, Mattia, Vagrant and Gunner > > > > Chris Lamb wrote: > >> Hello, >> >> Thank you to those of you who have reached out to us with concerns >> regarding the tone and content of posts on this list in recent weeks. >> >> We realise and appreciate that the sometimes-confrontational >> interactions on the list have really not been what we expect or >> desire from our community. As maintainers of this list and community, >> we apologise for any role we have played in failing to de-escalate >> disagreements on various threads. >> >> The growth of both Reproducible Builds practices and the associated >> community places us at an important juncture. We want to enable growth >> and diversification of resources like this list without losing the >> positive and collaborative energy that has characterised our online >> communications from the outset of the project. >> >> For our part, we acknowledge some community governan
diffoscope 253 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 253 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 253 includes the following changes:
* Improve DOS/MBR extraction by adding support for 7z.
(Closes: reproducible-builds/diffoscope#333)
* Process objdump symbol comment filter inputs as the Python "bytes" type
(and not str). (Closes: reproducible-builds/diffoscope#358)
* Add a missing RequiredToolNotFound import.
* Update copyright years.
## Download
Version 253 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Reproducible builds in November 2023
ing language have recently released version 10.5.0, which introduces the inclusion of a composer.lock file, ensuring total reproducibility of the shipped binary file. Further details and the discussion that went into their particular implementation can be found on the associated GitHub pull request [39]. In addition, the presentation "Leveraging Nix in the PHP ecosystem" [40] has been given in late October at the PHP International Conference in Munichby Pol Dellaiera [41]. While the video replay is not yet available, the (reproducible) presentation slides and speaker notes [42] are available. [38] https://phpunit.de/ [39] https://github.com/sebastianbergmann/phpunit/pull/5576 [40] https://phpconference.com/web-development/leveraging-nix-php-ecosystem/ [41] https://github.com/drupol [42] https://github.com/drupol/ipc2023/releases/tag/v23-79efbb4c24ab0d42c73906d16233a79d9659c5ca §§§ ## diffoscope changes diffoscope [43] is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes, including: * Improving DOS/MBR extraction by adding support for 7z. [44] * Adding a missing RequiredToolNotFound import. [45] * As a UI/UX improvement, try and avoid printing an extended traceback if diffoscope runs out of memory. [46] * Mark diffoscope as 'stable' on PyPI.org [47]. [48] * Uploading version 252 to Debian unstable. [49] [43] https://diffoscope.org [44] https://salsa.debian.org/reproducible-builds/diffoscope/commit/59b86c1f [45] https://salsa.debian.org/reproducible-builds/diffoscope/commit/64ed5f38 [46] https://salsa.debian.org/reproducible-builds/diffoscope/commit/bb887ddb [47] https://pypi.org/ [48] https://salsa.debian.org/reproducible-builds/diffoscope/commit/e5e8d51e [49] https://tracker.debian.org/news/1479028/accepted-diffoscope-252-source-into-unstable/ §§§ ## Website updates A huge number of notes [50] were added to our website that were taken at our recent Reproducible Builds Summit [51] held between October 31st and November 2nd in Hamburg, Germany. In particular, a big thanks to Arnout Engelen, Bernhard M. Wiedemann, Daan De Meyer, Evangelos Ribeiro Tzaras, Holger Levsen and Orhun Parmaksız. In addition to this, a number of other changes were made, including: * Chris Lamb migrated the website's homepage [52] to a "hero" image [53] [54], improved the documentation related to SOURCE_DATE_EPOCH and CMake [55] [56], added iomart [57] (neé Bytemark) and DigitalOcean [58] to our sponsors page [59] [60] and dropped an unnecessary link on some horizontal navigation buttons [61]. * Holger Levsen also made a large number of notes pages [62] from our 2022 summit in Venice [63] [64], migrated the website's syntax highlighter from Pygments to Rouge [65][66], fixed some grammar on our donate page [67][68][69][70] and did a lot of updates to the Hamburg Summit's general information page [71][72]. [50] https://reproducible-builds.org/events/hamburg2023/agenda/ [51] https://reproducible-builds.org/events/hamburg2023/ [52] https://reproducible-builds.org/ [53] https://www.optimizely.com/optimization-glossary/hero-image/ [54] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/2f50ba8a [55] https://reproducible-builds.org/docs/source-date-epoch/#cmake [56] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/ee0d0e19 [57] https://www.iomart.com/ [58] https://www.digitalocean.com/ [59] https://reproducible-builds.org/who/sponsors/ [60] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/16b73a33 [61] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/25cd328b [62] https://reproducible-builds.org/events/venice2022/agenda/ [63] https://reproducible-builds.org/events/venice2022/ [64] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/65072a36 [65] https://rouge.jneen.net/ [66] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/5d46ea5d [67] https://reproducible-builds.org/donate/ [68] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/0343dfea [69] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/24bf9105 [70] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/31b26b15 [71] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/c8a86c6b [72] https://salsa.debian.org/reproducible-builds/reproducible-website/commit/66691658 §§§ ## Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: * Bernhard M. Wiedemann:
Re: Please review the draft for November's report
Chris Lamb wrote: > Please review the draft for November's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-11/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1732416478958506188 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for November's report
Hi folks,
Me again. Please review the draft for November's Reproducible Builds
report:
https://reproducible-builds.org/reports/2023-11/?draft
… or you can do this via the Git repository directly:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-11.md
I intend to publish it no earlier than:
$ date -d 'Wed, 06 Dec 2023 13:15:00 +'
https://time.is/compare/1315_06_Dec_2023_in_GMT
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-11.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-11.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: An update regarding communication on this list
Dear all, Thank you for your patience, understanding and support in the past few days. We recently placed the rb-general mailing list under close moderation after a number of community members expressed concerns about the tone and content of some threads. Since that time, no new messages have been permitted on the list. However, we've also been using this time to reach out to other members of the community and have been listening to their feedback to ensure we understood all perspectives involved. We still fully intend to formalise a Code of Conduct for our mailing list and other communication channels. However, we do not wish to continue blocking all discussions pending this announcement. --> We are therefore moving the rb-general mailing to a "rolling" moderation model and appraising individual posts as they come in. <-- Because of this, may notice a slight delay in your messages before they are distributed to others, as well as a small influx of messages as we process the slight backlog. We will also be holding back messages and threads that, to the best of our belief, are not up to the standards we should be setting for our community or may be unproductive during this particular time. As before, if you have any questions or have input on what you consider to be exemplary policies, practices and norms for communities such as ours, please email rb-c...@lists.reproducible-builds.org. Although we are making this change today, we want to reinforce that a formal CoC is incoming and that list participants should treat other members of the community with respect and always help foster a welcoming environment for all. Thanks, — Chris, Holger, Mattia, Vagrant and Gunner Chris Lamb wrote: > Hello, > > Thank you to those of you who have reached out to us with concerns > regarding the tone and content of posts on this list in recent weeks. > > We realise and appreciate that the sometimes-confrontational > interactions on the list have really not been what we expect or > desire from our community. As maintainers of this list and community, > we apologise for any role we have played in failing to de-escalate > disagreements on various threads. > > The growth of both Reproducible Builds practices and the associated > community places us at an important juncture. We want to enable growth > and diversification of resources like this list without losing the > positive and collaborative energy that has characterised our online > communications from the outset of the project. > > For our part, we acknowledge some community governance shortcomings > that we are working to address. While the Reproducible Builds Summits > have operated under the guidance of the DebConf Code of Conduct [1] > and the Debian Code of Conduct [2], we have never formally established > a Code of Conduct for the mailing list or participation in the RB > Community channels and platforms in general. > > We are in the process of addressing those issues, and we encourage > input from anyone who might want to provide feedback or point us at > what you consider to be exemplary policies, practices and norms for > communities such as ours. > > In the meantime, we have placed the list under moderation to ensure > that no further counter-productive exchanges occur. [3] Everyone on > this list is requested to treat all people with respect and help > create a welcoming environment. > > We are also reaching out to a number of individuals who have > participated in recent threads or offered feedback in order to make > sure that we are honoring and understanding the range of perspectives > represented in our community. > > We welcome comment and questions, and are happy to be in dialogue with > anyone who desires the same. > > > Thanks, > > — Chris, Holger, Mattia, Vagrant and Gunner > > [1] http://debconf.org/codeofconduct.shtml > [2] https://www.debian.org/code_of_conduct > [3] We had attempted to place the list into moderation on Friday, > but a snafu on our part prevented it from working as intended.
An update regarding communication on this list
Hello, Thank you to those of you who have reached out to us with concerns regarding the tone and content of posts on this list in recent weeks. We realise and appreciate that the sometimes-confrontational interactions on the list have really not been what we expect or desire from our community. As maintainers of this list and community, we apologise for any role we have played in failing to de-escalate disagreements on various threads. The growth of both Reproducible Builds practices and the associated community places us at an important juncture. We want to enable growth and diversification of resources like this list without losing the positive and collaborative energy that has characterised our online communications from the outset of the project. For our part, we acknowledge some community governance shortcomings that we are working to address. While the Reproducible Builds Summits have operated under the guidance of the DebConf Code of Conduct [1] and the Debian Code of Conduct [2], we have never formally established a Code of Conduct for the mailing list or participation in the RB Community channels and platforms in general. We are in the process of addressing those issues, and we encourage input from anyone who might want to provide feedback or point us at what you consider to be exemplary policies, practices and norms for communities such as ours. In the meantime, we have placed the list under moderation to ensure that no further counter-productive exchanges occur. [3] Everyone on this list is requested to treat all people with respect and help create a welcoming environment. We are also reaching out to a number of individuals who have participated in recent threads or offered feedback in order to make sure that we are honoring and understanding the range of perspectives represented in our community. We welcome comment and questions, and are happy to be in dialogue with anyone who desires the same. Thanks, — Chris, Holger, Mattia, Vagrant and Gunner [1] http://debconf.org/codeofconduct.shtml [2] https://www.debian.org/code_of_conduct [3] We had attempted to place the list into moderation on Friday, but a snafu on our part prevented it from working as intended. -- o ⬋ ⬊ o o reproducible-builds.org ⬊ ⬋ o
diffoscope 252 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 252 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 252 includes the following changes:
* As UI/UX improvement, try and avoid printing an extended traceback if
diffoscope runs out of memory. This may not always be possible to detect.
* Mark diffoscope as stable in setup.py (for PyPI.org). Whatever diffoscope
is, at least, not "alpha" anymore.
## Download
Version 252 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Reproducible Builds in October 2023
ble. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: * Bernhard M. Wiedemann: * edje_cc [53] (race condition) * elasticsearch [54] (build failure) * erlang-retest [55] (embedded .zip timestamp) * fdo-client [56] (embeds private keys) * fftw3 [57] (random ordering) * gsoap [58] (date issue) * gutenprint [59] (date) * hub/golang [60] (embeds random build path) * Hyprland [61] (filesystem issue) * kitty [62] (sort-related issue, .tar file embeds modification time) * libpinyin [63] (ASLR) * maildir-utils [64] (date embedded in copyright) * mame [65] (order-related issue) * mingw32-binutils [66] & mingw64-binutils [67] (date) * MooseX [68] (date from perl-MooseX-App) * occt [69] (sorting issue) * openblas [70] (embeds CPU count) * OpenRGB [71] (corruption-related issue [72]) * python-numpy [73] (random file names) * python-pandas [74] (FTBFS) * python-quantities [75] (date) * python3-pyside2 [76] (order) * qemu [77] (date and Sphinx issue) * qpid [78] (sorting problem) * rakudo [79] (filesystem ordering issue) * SLOF [80] (date-related issue) * spack [81] (CPU counting issue) * xemacs-packages [82] (date-related issue) * Chris Lamb: * #1053353 [83] filed against dacite [84]. * #1053356 [85] filed against rtpengine [86]. [53] https://git.enlightenment.org/enlightenment/efl/issues/41 [54] https://github.com/elastic/elasticsearch-py/issues/2320 [55] https://build.opensuse.org/request/show/1116208 [56] https://bugzilla.opensuse.org/show_bug.cgi?id=1216293 [57] https://github.com/FFTW/fftw3/issues/337 [58] https://sourceforge.net/p/gsoap2/patches/185/ [59] https://sourceforge.net/p/gimp-print/source/merge-requests/9/ [60] https://github.com/golang/go/issues/63851 [61] https://github.com/hyprwm/Hyprland/pull/3550 [62] https://github.com/kovidgoyal/kitty/pull/6685 [63] https://github.com/libpinyin/libpinyin/issues/162 [64] https://github.com/djcb/mu/pull/2569 [65] https://github.com/mamedev/mame/pull/11651 [66] https://build.opensuse.org/request/show/1116036 [67] https://build.opensuse.org/request/show/1116040 [68] https://github.com/maros/MooseX-App/pull/71 [69] https://build.opensuse.org/request/show/1119524 [70] https://build.opensuse.org/request/show/1118201 [71] https://gitlab.com/CalcProgrammer1/OpenRGB/-/issues/3675 [72] https://gitlab.com/CalcProgrammer1/OpenRGB/-/merge_requests/2103 [73] https://bugzilla.opensuse.org/show_bug.cgi?id=1216458 [74] https://build.opensuse.org/request/show/1117743 [75] https://build.opensuse.org/request/show/1117898 [76] https://bugreports.qt.io/browse/PYSIDE-2508 [77] https://build.opensuse.org/request/show/1121011 [78] https://github.com/apache/qpid-proton/pull/411 [79] https://github.com/rakudo/rakudo/pull/5426 [80] https://gitlab.com/qemu-project/SLOF/-/merge_requests/1 [81] https://build.opensuse.org/request/show/1118130 [82] https://build.opensuse.org/request/show/1119260 [83] https://bugs.debian.org/1053353 [84] https://tracker.debian.org/pkg/dacite [85] https://bugs.debian.org/1053356 In addition, Chris Lamb fixed an issue in diffoscope [87], where if the equivalent of "file -i" returns "text/plain", fallback to comparing as a text file. This was originally filed as Debian bug #1053668 [88]) by Niels Thykier. [89] This was then uploaded to Debian (and elsewhere) as version 251. [87] https://diffoscope.org [88] https://bugs.debian.org/1053668 [89] https://salsa.debian.org/reproducible-builds/diffoscope/commit/81c68d7b [86] https://tracker.debian.org/pkg/rtpengine §§§ ## Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org [90]) in order to check packages and other artifacts for reproducibility. In October, a number of changes were made by Holger Levsen: * Debian-related changes: * Refine the handling of package blacklisting, such as sending blacklisting notifications to the #debian-reproducible-changes IRC channel. [91][92][93] * Install systemd-oomd on all Debian bookworm nodes (re. Debian bug #1052257 [94]). [95] * Detect more cases of failures to delete schroots. [96] * Document various bugs in bookworm which are (currently) being manually worked around. [97] * Node-related changes: * Integrate the new arm64 machines from Codethink [98]. [99][100][101][102][103][104] * Improve various node cleanup routines. [105][106][107][108] * General node maintenance. [109][110][111][112] * Monitoring-related changes: * Remove unused Munin [113] monitoring plugins. [114] * Complain less visibly about "too many" installed kernels. [115] * Misc: * Enhance the firewall handling on Jenkins no
Re: Please review the draft for October's report
Hey,
> [..]
Thanks Pol for publishing the report. I'm not quite sure what
happened: I have a local, tagged commit here that publishes the report
by removing the "draft" flag.
But what is more surprising is that, until I just fixed it, my working
tree was claiming that this was what remote/origin/master was as well.
Weird.
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Please review the draft for October's report
Chris Lamb wrote: > Please review the draft for October's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-10/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1723268669940113521 Thanks! -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for October's report
Hi all,
Please review the draft for October's Reproducible Builds report:
https://reproducible-builds.org/reports/2023-10/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-10.md
I intend to publish it no earlier than:
$ date -d 'Fri, 10 Nov 2023 14:15:00 +'
https://time.is/compare/1415_10_Nov_2023_in_GMT
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-10.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-10.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Reproducible builds stickers and flyers
Fabian Keil wrote: > The summit is coming up, as you know I intend to attend and > I'm still interested to take a couple of hundred stickers of > your hand there to freely redistribute them in the near future. Somewhat relatedly, I was just tidying up around Chez Lamby and I discovered 20/25 Reproducible Builds badges that I'll bring to the Summit. :) Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 251 released
Hi, The diffoscope maintainers are pleased to announce the release of version 251 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 251 includes the following changes: * If the equivalent of `file -i` returns text/plain, fallback to comparing this file as a text file. This especially helps when file(1) miscategorises text files as some esoteric type. (Closes: Debian:#1053668) * Update copyright years. ## Download Version 251 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Re: Verification Builds and Snapshots For Debian
Dear Vagrant, > In the meantime, I worked on a naive implementation of this, using > debmirror and btrfs snapshots (zfs or xfs are other likely candidates > for filesystem-level snapshots). It is working better than I expected! […] > Currently weighing in at about 550GB, each snapshot of the archive for > amd64+all+source is weighing in under 330GB if I recall correctly... so > that is over a month worth of snapshots for the cost of about two full > snapshots. Obviously, adding more architectures would dramatically > increase the space used (Would probably add arm64, armhf, i386, ppc64el > and riscv64 if I were to do this again). This sounds like great progress. :) Do you have any updates since you posted your message? (Are you snapshotting after each dinstall and labelling them with some timestamp…? Or perhaps you have some other, cleverer, scheme?) Best wishes, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org chris-lamb.co.uk `-
Re: Please review the draft for September's report
Chris Lamb wrote: > Please review the draft for September's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-09/ .. and also consider re-tweeting: https://twitter.com/ReproBuilds/status/1712505932544950565 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for September's report
Hi all,
Please review the draft for September's Reproducible Builds report:
https://reproducible-builds.org/reports/2023-09/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-09.md
I intend to publish it no earlier than:
$ date -d 'Thu, 12 Oct 2023 17:00:00 +0100'
https://time.is/compare/1700_12_Oct_2023_in_BST
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-09.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-09.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Please review the draft for August's report
Chris Lamb wrote: > Please review the draft for August's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-08/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1700253293497536959 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 250 released
Hi, The diffoscope maintainers are pleased to announce the release of version 250 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 250 includes the following changes: [ Chris Lamb ] * Fix compatibility with file 5.45. (Closes: reproducible-builds/diffoscope#351) [ Vagrant Cascadian ] * Add external tool references for GNU Guix (for html2text and ttx). ## Download Version 250 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for August's report
Hi all,
Please review the draft for August's Reproducible Builds report:
https://reproducible-builds.org/reports/2023-08/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-08.md
I intend to publish it no earlier than:
$ date -d 'Fri, 08 Sep 2023 20:00:00 -'
https://time.is/compare/2000_08_Sep_2023_in_UTC
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-08.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-08.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 249 released
Hi, The diffoscope maintainers are pleased to announce the release of version 249 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 249 includes the following changes: [ FC Stegerman ] * Add specialize_as() method, and use it to speed up .smali comparison in APKs. (Closes: reproducible-builds/diffoscope!108) [ Chris Lamb ] * Add documentation for the new specialize_as, and expand the documentation of `specialize` too. (Re: reproducible-builds/diffoscope!108) * Update copyright years. [ Felix Yan ] * Correct typos in diffoscope/presenters/utils.py. ## Download Version 249 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 248 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 248 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 248 includes the following changes:
[ Greg Chabala ]
* Merge Docker "RUN" commands into single layer.
## Download
Version 248 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 247 released
Hi, The diffoscope maintainers are pleased to announce the release of version 247 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 247 includes the following changes: [ Chris Lamb ] * Fix compataibility with file(1) version 5.45. * Use assert_diff in test_uimage and test_cpio. [ Roland Clobus ] * xb-tool has moved in Debian bookworm. ## Download Version 247 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for July's report
Hi all, Please review the draft for July's Reproducible Builds report: https://reproducible-builds.org/reports/2023-07/?draft ⦠or, via the Git repository itself: https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-07.md I intend to publish it no earlier than: $ date -d 'Fri, 04 Aug 2023 14:15:00 +0100' https://time.is/compare/1415_04_Aug_2023_in_BST § Please feel free and commit/push to drafts directly without the overhead of sending patches or merge requests. You should make your changes to the "_reports/2023-07.md" file in the "reproducible-website" repository: $ git clone https://salsa.debian.org/reproducible-builds/reproducible-website $ cd reproducible-website $ sensible-editor _reports/2023-07.md I am happy to reword and/or rework additions prior to publishing. If you currently do not have access to the above repository, you can request access by following the instructions at: https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
diffoscope 246 released
Hi, The diffoscope maintainers are pleased to announce the release of version 246 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 246 includes the following changes: [ Gianfranco Costamagna ] * Add support for LLVM 16. ## Download Version 246 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
diffoscope 245 released
Hi, The diffoscope maintainers are pleased to announce the release of version 245 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 245 includes the following changes: [ Chris Lamb ] * Don't include file size in image metadata; it is, at best, distracting and it is already in the directory metadata. * Move to using assert_diff in ICO and JPEG tests. * Update copyright years. ## Download Version 245 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ ⦠but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Re: try.diffoscope.org stuck on comparison?
Hi Jaka,
> Got assigned a high queue number, that was still the same after an
> hour or so, so I went to sleep. It's still processing the same queue
> item now, so I think something is wrong; the process can't be that
> intensive for a single comparison.
Thanks for the heads up. It looks like it got wedged whilst purging
old files as part of its retention policy (which I don't quite
understand as deleting old data is the least intensive thing
try.diffoscope.org does).
I restarted the queue, and it is now: a) updating the version of
diffoscope (which should take 10 minutes or so); and then b) running
through all of the outstanding requests. I'll watch the queue logs
out of the corner of my eye until they are back to zero. Thanks again.
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 244 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 244 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 244 includes the following changes:
[ Chris Lamb ]
* Address compatibility with python-libarchive-c version 5.
(Closes: reproducible-builds/diffoscope#344)
* Testsuite changes:
- Mark that test_dex::test_javap_14_differences requires procyon.
- Fix "test skipped" textual reason generation in the case of a required
version being outside of the required range.
- Temporarily mark some Android-related as XFAIL due to Debian bugs
#1040941 and #1040916.
## Download
Version 244 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Please review the draft for June's report
Chris Lamb wrote: > Please review the draft for June's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-06/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1679119806820122627 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: breaking CI if build is not reproducible?
Martin Monperrus wrote: > Are you aware of any project where reproducibility is checked in a > continuous integration pipeline? Sorry for the delay here. Just to belatedly add that Tails [0] check for reproducibility in their CI. From what I recall, however, the check is configured as an advisory—if important—part of the build pipeline rather than failures "breaking the build". [0] https://tails.boum.org/ Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for June's report
Hi all,
Please review the draft for June's Reproducible Builds report:
https://reproducible-builds.org/reports/2023-06/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-06.md
I intend to publish it no earlier than:
$ date -d 'Wed, 12 Jul 2023 14:15:00 +0100'
https://time.is/compare/1415_12_Jul_2023_in_BST
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-06.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-06.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 243 released
Hi, The diffoscope maintainers are pleased to announce the release of version 243 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 243 includes the following changes: [ Chris Lamb ] * Drop Jenkins build reference in README.rst. [ Ed Maste ] * Update FreeBSD package names [ Mattia Rizzolo ] * Improve the documentation on to produce that binary blob that in the arsc comparator. ## Download Version 243 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Please review the draft for May's report
Chris Lamb wrote: > Please review the draft for May's Reproducible Builds report: This has now been published — thanks to all who contributed. If you have a moment, please share the following link: https://reproducible-builds.org/reports/2023-05/ … and also consider retweeting: https://twitter.com/ReproBuilds/status/1665775526466969600 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for May's report
Hi all,
Please review the draft for May's Reproducible Builds report:
https://reproducible-builds.org/reports/2023-05/?draft
… or, via the Git repository itself:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-05.md
I intend to publish it no earlier than:
$ date -d 'Mon, 05 Jun 2023 17:00 -'
https://time.is/compare/1700_05_Jun_2023_in_UTC
§
Please feel free and commit/push to drafts directly without the overhead of
sending patches or merge requests. You should make your changes to the
"_reports/2023-05.md" file in the "reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ sensible-editor _reports/2023-05.md
I am happy to reword and/or rework additions prior to publishing. If you
currently do not have access to the above repository, you can request access
by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Please review the draft for April's report
Chris Lamb wrote: > Please review the draft for April's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-04/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1654938704119726080 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 242 released
Hi, The diffoscope maintainers are pleased to announce the release of version 242 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 242 includes the following changes: * If the binwalk Python module is not available, ensure the user knows they may be missing more differences in, for example, concatenated .cpio archives. * Factor out routine to generate a human-readable comments when Python modules are missing. ## Download Version 242 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for April's report
Hey folks,
If you have a moment, please review the draft for the Reproducible
Builds report for April:
https://reproducible-builds.org/reports/2023-04/?draft
You can do this via the Git repository itself, too:
https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-04.md
I intend to publish it no earlier than:
$ date -d 'Sat, 06 May 2023 18:00:00 -'
https://time.is/compare/1800_06_May_2023_in_UTC
§
Please feel free and commit/push to drafts directly without the
overhead of sending patches or merge requests to me directly. You
should make your changes to the "_reports/2023-04.md" file in the
"reproducible-website" repository:
$ git clone https://salsa.debian.org/reproducible-builds/reproducible-website
$ cd reproducible-website
$ vim _reports/2023-04.md
I am happy to reword and/or rework additions prior to publishing,
though. If you currently do not have access to the above repository,
you can request access by following the instructions at:
https://reproducible-builds.org/contribute/salsa/
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 241 released
Hi, The diffoscope maintainers are pleased to announce the release of version 241 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 241 includes the following changes: [ Chris Lamb ] * Add a missing 'raise' statement dropped in 2d95ae41e. Thanks, Mattia! [ Mattia Rizzolo ] * document sending out an email upon release ## Download Version 241 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Debian and reproducible-builds.org incoherence?
Hi Alexis, > Have I compiled and packaged everything wrong? Given my skim reading of the SHA256sums, I don't think so. As in, you seem to be generating the same packages as tests.reproducible-builds.org, at least on amd64. If anything, "Debian", ie. the official binaries, are the "wrong" ones here… although I wouldn't quite use that term. :) > https://tracker.debian.org/pkg/fbreader indicates reproducibility > OK. This is, unfortunately, a little misleading. To clarify, this statement only means that *tests.reproducible-builds.org* believes that the fbreader source package is reproducible — it doesn't promise that the binary packages on the official Debian mirrors are bit-for-bit identical with anything. This is, of course, not ideal. Still, this is what folks on this list are getting at when they say they "want to make Debian 'really' reproducible". Regarding precisely why there is a difference, I can't write more at the moment, but have you tried comparing "your" fbreader_0.12.10dfsg2-4_amd64.deb with one shipped by Debian using diffoscope? Happy to run that for you if you can provide your file. Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Sphinx: localisation changes / reproducibility
Dear James,
Thanks for your recent emails. As the original bug filer (#9778), I'm
obviously invested in this being fixed… and I was enjoying watching
the recent flurry of activity hit my inbox.
> Probably nothing new to many of the folks on this mailing list and/or
> seasoned software engineers generally, but I figured I'd try to
> document my findings :)
Hah. This is much appreciated as well: as you imply, aborted
experiments usually don't end up getting documented at all, and so we
typically end up with the highly-misleading "here is the perfect
solution to this problem" PR. :)
Best wishes,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
Re: Please review the draft for March's report
Chris Lamb wrote: > Please review the draft for March's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-03/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1644283929598337024 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Three bytes in a zip file
Hi Larry, > TZ=UTC zip --latest-time "$zipfile" fab/* ^ Just as a quick glance, this may be expanding the '*' shell glob in a different order between your two systems due to the underlying filesystem order. Perhaps try "zip […] -r […] fab/"? > The diff is so small, it seems silly to post both files, but I'll > do that anyway. (Assuming I'm parsing this right, I think you forgot to attach or link them.) Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for March's report
Hi all, Please review the draft for March's Reproducible Builds report: https://reproducible-builds.org/reports/2023-03/?draft ⦠or, via the Git repository itself: https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-03.md I intend to publish it no earlier than: $ date -d 'Thu, 06 Apr 2023 14:15:00 +0100' https://time.is/compare/1415_06_Apr_2023_in_BST § Please feel free and commit/push to drafts directly without the overhead of sending patches or merge requests. You should make your changes to the "_reports/2023-03.md" file in the "reproducible-website" repository: $ git clone https://salsa.debian.org/reproducible-builds/reproducible-website $ cd reproducible-website $ sensible-editor _reports/2023-03.md I am happy to reword and/or rework additions prior to publishing. If you currently do not have access to the above repository, you can request access by following the instructions at: https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
diffoscope 240 released
Hi,
The diffoscope maintainers are pleased to announce the release of
version 240 of diffoscope.
diffoscope tries to get to the bottom of what makes files or
directories different. It will recursively unpack archives of many
kinds and transform various binary formats into more human-readable
form to compare them. It can compare two tarballs, ISO images, or PDF
just as easily.
Version 240 includes the following changes:
[ Holger Levsen ]
* Update Lintian override info format in debian/source/lintian-overrides.
* Add Lintian overrides for some "very long lines" in test cases.
* Update Lintian overrides for tests being tagged source-is-missing or
prebuilt.
* Add Lintian override for very long lines for debian/tests/control.
* Re-add two Lintian overrides about (well-known) source-is-missing
instances.
[ Mattia Rizzolo ]
* Drop the use of include_package_data=True in setup.py.
## Download
Version 240 is available from Debian unstable as well as PyPI, and
will shortly be available on other platforms surely. More details can
be found here:
https://diffoscope.org/
… but source tarballs may be located here:
https://diffoscope.org/archive/
The corresponding Docker image may be run via (for example):
$ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \
registry.salsa.debian.org/reproducible-builds/diffoscope a b
## Contribute
diffoscope is developed within the "Reproducible builds" effort.
- Git repository
https://salsa.debian.org/reproducible-builds/diffoscope
- Docker image, eg.
registry.salsa.debian.org/reproducible-builds/diffoscope
https://salsa.debian.org/reproducible-builds/diffoscope
- Issues and feature requests
https://salsa.debian.org/reproducible-builds/diffoscope/issues
- Contribution instructions (eg. to file an issue)
https://reproducible-builds.org/contribute/salsa/
Regards,
--
o
⬋ ⬊ Chris Lamb
o o reproducible-builds.org
⬊ ⬋
o
diffoscope 239 released
Hi, The diffoscope maintainers are pleased to announce the release of version 239 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 239 includes the following changes: [ Chris Lamb ] * Fix compatibility with pypdf 3.x, and correctly restore test data. (Closes: reproducible-builds/diffoscope#335) * Rework PDF annotations processing into a separate method. ## Download Version 239 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: Please review the draft for February's report
Chris Lamb wrote: > Please review the draft for February's Reproducible Builds report: This has now been published — thanks to all who contributed. If possible, please share the following link: https://reproducible-builds.org/reports/2023-02/ .. and also consider retweeting: https://twitter.com/ReproBuilds/status/1632305660657270786 Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Please review the draft for February's report
Hi all, Please review the draft for February's Reproducible Builds report: https://reproducible-builds.org/reports/2023-02/?draft ⦠or, via the Git repository itself: https://salsa.debian.org/reproducible-builds/reproducible-website/blob/master/_reports/2023-02.md I intend to publish it no earlier than: $ date -d 'Sat, 04 Mar 2023 14:15:00 +' https://time.is/compare/1415_04_Mar_2023_in_GMT § Please feel free and commit/push to drafts directly without the overhead of sending patches or merge requests. You should make your changes to the "_reports/2023-02.md" file in the "reproducible-website" repository: $ git clone https://salsa.debian.org/reproducible-builds/reproducible-website $ cd reproducible-website $ sensible-editor _reports/2023-02.md I am happy to reword and/or rework additions prior to publishing. If you currently do not have access to the above repository, you can request access by following the instructions at: https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬠⬠Chris Lamb o o reproducible-builds.org ð ⬠⬠o
Re: Does diffoscope compares disk partitions
Hey Venkata, > Does it support disk partitions or do I missing something? The short answer is that that diffoscope *should* support comparing your partition images properly, instead of falling back to a raw xxd(1) comparison. The reason diffoscope doesn't do that right now is either due to a bug, or we just need to extend support for this particular type of partition. Correctly detecting DOS/MBR files is somewhat more fiddly than one might think, but the pertinent part of the debug log is this: > image1.wic not identified by any comparator. Magic says: DOS/MBR boot > sector; partition 1 : ID=0xee, start-CHS (0x0,0,2), end-CHS > (0x3ff,255,63), startsector 1, 12546899 sectors, extended partition > table (last) Would it be possible for you to share the two .wic images somewhere? In fact, if you could re-file this issue in our bug tracker, that would be great: https://salsa.debian.org/reproducible-builds/diffoscope/-/issues (And just for clarity, the ".wic" files are files containing raw partitions, but the ".disk" files contain entire disk images including a partition table?) Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
Re: python datetime .. grrr
Larry Doolittle wrote: >> > [...] could >> > someone with access to r-b infrastructure check if the aforementioned >> > patch _actually_ results in reproducible verilator Debian packages? > >> Which aforementioned patch? My "utcfromtimestamp" one, or >> bc6a7787? Happy to try either for you, but would need a pointer >> to where to find bc6a7787. :) > > bc6a7787 as found at > > https://github.com/verilator/verilator/commit/bc6a7787ed271a8f52ed5b8f8a9e0e8cbba1ab38 > also attached. I can confirm this makes verilator build reproducibly in Debian. Hope this helps. Best wishes, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
diffoscope 236 released
Hi, The diffoscope maintainers are pleased to announce the release of version 236 of diffoscope. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily. Version 236 includes the following changes: [ FC Stegerman ] * Update code to match latest version of Black. (Closes: #1031433) [ Chris Lamb ] * Require at least Black version 23.1.0 to run the internal Black tests. * Update copyright years. ## Download Version 236 is available from Debian unstable as well as PyPI, and will shortly be available on other platforms surely. More details can be found here: https://diffoscope.org/ … but source tarballs may be located here: https://diffoscope.org/archive/ The corresponding Docker image may be run via (for example): $ docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope a b ## Contribute diffoscope is developed within the "Reproducible builds" effort. - Git repository https://salsa.debian.org/reproducible-builds/diffoscope - Docker image, eg. registry.salsa.debian.org/reproducible-builds/diffoscope https://salsa.debian.org/reproducible-builds/diffoscope - Issues and feature requests https://salsa.debian.org/reproducible-builds/diffoscope/issues - Contribution instructions (eg. to file an issue) https://reproducible-builds.org/contribute/salsa/ Regards, -- o ⬋ ⬊ Chris Lamb o o reproducible-builds.org ⬊ ⬋ o
