Bug#940234: debian-policy: add a section about source reproducibility
On Mon, Jun 20, 2022 at 07:43:45PM +0700, Teukumif tahulziran wrote: > > There is already a section about reproducibility in the debian-policy, > > but it only mentions the binary packages. It might be a good idea to > > add a new requirement that repeatedly building the source package in > > the same environment produces identical .dsc file modulo the GPG > > signature. as you say, it *might* be a good idea, but in our experience it's not practical because too many sources cannot be rebuild reproducibly. Also, and probably more importantly, it's quite unclear what the practical benefit is can you explain? > > I haven't checked how many packages do not fulfill this condition You should definitly do this before asking policy to be changed. It's also not really hard, just loop through all source packages, download them, rebuild them, compare. And you might want to start with just the essential set. and, TBH, I'm pretty sure very few source packages can be rebuild reproducible. Proove me wrong! :) -- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ⠈⠳⣄ The corona crisis is peanuts compared to the global climate disaster. signature.asc Description: PGP signature
Bug#940234: debian-policy: add a section about source reproducibility
On Mon, Jun 20, 2022 at 07:43:45PM +0700, Teukumif tahulziran wrote: > On Sat, 14 Sep 2019 13:34:49 +0200 Aurelien Jarno > wrote: > > Package: debian-policy > > Version: 4.4.0.1 > > Severity: wishlist > > > > There is already a section about reproducibility in the debian-policy, > > but it only mentions the binary packages. It might be a good idea to > > add a new requirement that repeatedly building the source package in > > the same environment produces identical .dsc file modulo the GPG > > signature. > > > > I haven't checked how many packages do not fulfill this condition, but > > there are for sure packages where the Build-Depends: entry in the dsc > > file does not match the debian/control file, as they have been added > > manually after the package build. TTBOMK there is nothing preventing > > that in the debian policy. What about the fact that .dsc include the hash of the .debian.tar.xz file that contains the debian/control, so changing debian/control invalidate the hash ? Cheers, Bill
Bug#940234: debian-policy: add a section about source reproducibility
On Sat, 14 Sep 2019 13:34:49 +0200 Aurelien Jarno wrote: > Package: debian-policy > Version: 4.4.0.1 > Severity: wishlist > > There is already a section about reproducibility in the debian-policy, > but it only mentions the binary packages. It might be a good idea to > add a new requirement that repeatedly building the source package in > the same environment produces identical .dsc file modulo the GPG > signature. > > I haven't checked how many packages do not fulfill this condition, but > there are for sure packages where the Build-Depends: entry in the dsc > file does not match the debian/control file, as they have been added > manually after the package build. TTBOMK there is nothing preventing > that in the debian policy. > > -- System Information: > Debian Release: bullseye/sid > APT prefers testing > APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental') > Architecture: amd64 (x86_64) > > Kernel: Linux 5.2.0-2-amd64 (SMP w/4 CPU cores) > Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE > Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr (charmap=UTF-8) > Shell: /bin/sh linked to /bin/dash > Init: systemd (via /run/systemd/system) > LSM: AppArmor: enabled > > debian-policy depends on no packages. > > Versions of packages debian-policy recommends: > ii libjs-sphinxdoc 1.8.5-3 > > Versions of packages debian-policy suggests: > pn doc-base > > -- no debconf information > >
Bug#940234: debian-policy: add a section about source reproducibility
On Sat, 14 Sep 2019 13:34:49 +0200 Aurelien Jarno wrote: > Package: debian-policy > Version: 4.4.0.1 > Severity: wishlist > > There is already a section about reproducibility in the debian-policy, > but it only mentions the binary packages. It might be a good idea to > add a new requirement that repeatedly building the source package in > the same environment produces identical .dsc file modulo the GPG > signature. > > I haven't checked how many packages do not fulfill this condition, but > there are for sure packages where the Build-Depends: entry in the dsc > file does not match the debian/control file, as they have been added > manually after the package build. TTBOMK there is nothing preventing > that in the debian policy. > > -- System Information: > Debian Release: bullseye/sid > APT prefers testing > APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental') > Architecture: amd64 (x86_64) > > Kernel: Linux 5.2.0-2-amd64 (SMP w/4 CPU cores) > Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE > Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr (charmap=UTF-8) > Shell: /bin/sh linked to /bin/dash > Init: systemd (via /run/systemd/system) > LSM: AppArmor: enabled > > debian-policy depends on no packages. > > Versions of packages debian-policy recommends: > ii libjs-sphinxdoc 1.8.5-3 > > Versions of packages debian-policy suggests: > pn doc-base > > -- no debconf information > >
Bug#940234: debian-policy: add a section about source reproducibility
On Sat, Sep 14, 2019 at 11:57:43PM +0200, Guillem Jover wrote: > > >> I haven't checked how many packages do not fulfill this condition > > > please do check. last (and only) time we (=r-b) looked, it wasn't > > > practical at all. this was around 5 years ago, but I don't remember any > > > work done on improving this. > Back when we were fixing the binary package reproducible problems > within dpkg, I also checked the source side, and fixed a few > problematic cases. Assuming the same tools installed as defined in > the .buildinfo file, and the same content in the unpacked source > tree, dpkg-source should be producing the same output source packages. oh, cool, thanks for this spreading this information! > If this does not hold, I'd consider it a bug to be fixed. great! so now someone just needs to do something^wa rebuild of say 1000 source packages and share the stats... -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Bug#940234: debian-policy: add a section about source reproducibility
On Sat, 2019-09-14 at 08:58:21 -0700, Sean Whitton wrote: > On Sat 14 Sep 2019 at 02:01PM +00, Holger Levsen wrote: > > On Sat, Sep 14, 2019 at 01:34:49PM +0200, Aurelien Jarno wrote: > >> There is already a section about reproducibility in the debian-policy, > >> but it only mentions the binary packages. It might be a good idea to > >> add a new requirement that repeatedly building the source package in > >> the same environment produces identical .dsc file modulo the GPG > >> signature. > >> > >> I haven't checked how many packages do not fulfill this condition > > > > please do check. last (and only) time we (=r-b) looked, it wasn't > > practical at all. this was around 5 years ago, but I don't remember any > > work done on improving this. > > Right. While we can all agree that it would be nice for source package > builds to reproducible, I think our current source package formats make > it quite a hard problem, so it would be good to have some data before we > spend any time discussing this further. Back when we were fixing the binary package reproducible problems within dpkg, I also checked the source side, and fixed a few problematic cases. Assuming the same tools installed as defined in the .buildinfo file, and the same content in the unpacked source tree, dpkg-source should be producing the same output source packages. If this does not hold, I'd consider it a bug to be fixed. Thanks, Guillem
Bug#940234: debian-policy: add a section about source reproducibility
Hello, On Sat 14 Sep 2019 at 02:01PM +00, Holger Levsen wrote: > On Sat, Sep 14, 2019 at 01:34:49PM +0200, Aurelien Jarno wrote: >> There is already a section about reproducibility in the debian-policy, >> but it only mentions the binary packages. It might be a good idea to >> add a new requirement that repeatedly building the source package in >> the same environment produces identical .dsc file modulo the GPG >> signature. >> >> I haven't checked how many packages do not fulfill this condition > > please do check. last (and only) time we (=r-b) looked, it wasn't > practical at all. this was around 5 years ago, but I don't remember any > work done on improving this. Right. While we can all agree that it would be nice for source package builds to reproducible, I think our current source package formats make it quite a hard problem, so it would be good to have some data before we spend any time discussing this further. -- Sean Whitton signature.asc Description: PGP signature
Bug#940234: debian-policy: add a section about source reproducibility
Aurelien Jarno writes: > Package: debian-policy > Version: 4.4.0.1 > Severity: wishlist > > There is already a section about reproducibility in the debian-policy, > but it only mentions the binary packages. It might be a good idea to > add a new requirement that repeatedly building the source package in > the same environment produces identical .dsc file modulo the GPG > signature. > > I haven't checked how many packages do not fulfill this condition, but > there are for sure packages where the Build-Depends: entry in the dsc > file does not match the debian/control file, as they have been added > manually after the package build. TTBOMK there is nothing preventing > that in the debian policy. I'm not sure if this is exactly the same issue, but I've recently been thinking about (and messing up) source package reproducibility from git repos. It is probably to early for policy language to be talking about git, but it might be worth keeping in mind the fact that there are various tools producing source packages, sometimes in non-obvious ways. d
Bug#940234: debian-policy: add a section about source reproducibility
On Sat, Sep 14, 2019 at 01:34:49PM +0200, Aurelien Jarno wrote: > There is already a section about reproducibility in the debian-policy, > but it only mentions the binary packages. It might be a good idea to > add a new requirement that repeatedly building the source package in > the same environment produces identical .dsc file modulo the GPG > signature. > > I haven't checked how many packages do not fulfill this condition please do check. last (and only) time we (=r-b) looked, it wasn't practical at all. this was around 5 years ago, but I don't remember any work done on improving this. -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Bug#940234: debian-policy: add a section about source reproducibility
Package: debian-policy Version: 4.4.0.1 Severity: wishlist There is already a section about reproducibility in the debian-policy, but it only mentions the binary packages. It might be a good idea to add a new requirement that repeatedly building the source package in the same environment produces identical .dsc file modulo the GPG signature. I haven't checked how many packages do not fulfill this condition, but there are for sure packages where the Build-Depends: entry in the dsc file does not match the debian/control file, as they have been added manually after the package build. TTBOMK there is nothing preventing that in the debian policy. -- System Information: Debian Release: bullseye/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 5.2.0-2-amd64 (SMP w/4 CPU cores) Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled debian-policy depends on no packages. Versions of packages debian-policy recommends: ii libjs-sphinxdoc 1.8.5-3 Versions of packages debian-policy suggests: pn doc-base -- no debconf information
