Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
René Mayrhofer wrote: On 2014-09-25 06:24, Hans-Christoph Steiner wrote: W. Martin Borgert wrote: On 2014-09-24 23:05, Hans-Christoph Steiner wrote: * the signature files sign the package contents, not the hash of whole .deb file (i.e. control.tar.gz and data.tar.gz). So preinst and friends would not be signed? Sounds dangerous to me. All package contents would be signed, except the signature itself. The signature would be a separate file in the ar archive of the .deb that signs control.tar.gz and data.tar.gz. See jar or apk format for an example of how this works. I know I'm late to the discussion, but for the record, I fully agree with this approach as the probably best compromise between usability (don't underestimate that, see the emergence of the various app shops for Linux applications), security, and flexibility. If anybody wants to work on that, I'm happy to support it in the University Linz context (i.e. as student work, thesis, etc.) and contribute to the process (although, depressingly but realistically, not the implementation). Rene Since you mention Austria, I'll be based in Vienna from Oct 30th until March 3rd, perhaps we could even arrange a dev meeting/sprint on this topic in Linz or Vienna. .hc -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/543fe36a.5070...@at.or.at
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 2014-09-25 06:24, Hans-Christoph Steiner wrote: W. Martin Borgert wrote: On 2014-09-24 23:05, Hans-Christoph Steiner wrote: * the signature files sign the package contents, not the hash of whole .deb file (i.e. control.tar.gz and data.tar.gz). So preinst and friends would not be signed? Sounds dangerous to me. All package contents would be signed, except the signature itself. The signature would be a separate file in the ar archive of the .deb that signs control.tar.gz and data.tar.gz. See jar or apk format for an example of how this works. I know I'm late to the discussion, but for the record, I fully agree with this approach as the probably best compromise between usability (don't underestimate that, see the emergence of the various app shops for Linux applications), security, and flexibility. If anybody wants to work on that, I'm happy to support it in the University Linz context (i.e. as student work, thesis, etc.) and contribute to the process (although, depressingly but realistically, not the implementation). Rene -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/543cde75.7050...@debian.org
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On Sunday 21 September 2014 21:13:50, Richard van den Berg wrote: Package formats like apk and jar avoid this chicken and egg problem by hashing the files inside a package, and storing those hashes in a manifest file. Signatures only sign the manifest file. The manifest itself and the signature files are not part of the manifest, but are part of the package. So a package including it's signature(s) is still a single file. This is bad design and will inevitably lead to security issues (as has been demonstrated by Android and apk). One must check the signature first, and only if the signature matches, start parsing complex file formats. And yes, zip is complex enough to be a problem. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/6823138.NpYddIaakV@k
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
Daniel Kahn Gillmor wrote: Thanks for the discussion, Hans. On 09/19/2014 02:47 PM, Hans-Christoph Steiner wrote: Packages should not be accepted into any official repo, sid included, without some verification builds. A .deb should remain unchanged once it is accepted into any official repo (maybe experimental could be an exception, but not sid). I think that is essential. But some repositories could have different rules for package inclusion than others, right? for example, say debian wanted to offer an unstable-reproducible suite, which only permitted packages that had been independently rebuilt reproducibly by multiple DDs and at least two different buildds. Ideally, the packages that are shared between this repository and other repositories would be identical. Note that if .deb files are internally signed, two developers *cannot* create the same exact .deb if they do not share their secret keys. You're missing one key detail here, let's see if I can suss it out: * the builds are _exactly_ the same, except the signatures * the embedded signature does not sign the signature files (see jar and apk formats, which are almost the same, for examples) * anyone can just copy other dev's signature into the package and it will validate because the package contents are exactly the same * the signature files sign the package contents, not the hash of whole .deb file (i.e. control.tar.gz and data.tar.gz). Therefore two developers can easily create the same .deb if that have access to the signature file since they can just copy it. No need to run the signing process again. If people create their own .deb files in a reproducible process, then copy in the same signature files, then the hash of the .deb will be the same. I see no reason for changing the .deb between sid and testing, except for perhaps how existing implementations are doing it. It is usually worth the work to do things right way, rather than the easy way. I agree with this sentiment, i think we're trying to sort out what is the right way. The build verification process needs to happen between the package upload and publishing to sid or security updates. Two builds is easy: the .deb that the uploader generates and the one the Debian process makes. That is probably enough. In Debian's case, it probably is too complicated to include multiple signatures. In that case, there should be only one canonical signature by dak once the build verification signature threshold has been passed. Then all of the other signatures could be added to .buildinfo or .changes or whatever other file. but the .buildinfo file is designed to say i generated the .deb that matches this digest exactly, which the corroborating builder cannot do, because they cannot produce the internal signature. No need to produce the signature, just copy it! Plus, we now have two different places to look for signatures. one canonical one and then some external ones, and the signatures themselves have different properties (one signs parts of the deb, the other signs the whole .deb; one signs the build environment, the other does not, etc) Definitely look at jar signing, it handles multiple signatures fine. I see no reason why you can't include an unlimited number of signatures in a .deb. Changing the number of signatures will change the hash of the .deb, that is why there needs to be a canonical set of signatures for each .deb. As for signing the hash of the entire .deb, that is what apt already gives us, that does not need to be reproduced in the dpkg-sig embedded signature. For people who want to verify the contents of a .deb with any kind of signature, then a tool will have to compare the hashes of control.tar.gz and data.tar.gz. Another option is to do it like f-droid.org does. F-droid.org generates a APK signing key for each app, then manages the signing on a specialized signing server. Or another option is just requiring all the signers to be from the debian-keyring, rather than an exact match for previous signers. Again, i think this is getting ahead of the discussion. i'm not proposing that we try to set debian (or other derived distro) archive policy here, i just think we want to think In any case, the .deb needs to remain unchanged. right. but it can't be unchanged if the archive distributor decides that a different signer is the canonical signer. So you're making the contents of the .deb dependent on archive policy, rather than the other way around. I *want* ubuntu and debian and mint to all ship the exact same .deb for any packages that are reproducible (and eventually, all packages!) that they share, and i also want those different distros to be able to produce the reproducible .deb independently of one another. If foo_1.2-3_mipsel.deb is built first on the ubuntu builders and ubuntu decides to include it in the archive, and then debian is able to reproduce that build
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 2014-09-24 23:05, Hans-Christoph Steiner wrote: * the signature files sign the package contents, not the hash of whole .deb file (i.e. control.tar.gz and data.tar.gz). So preinst and friends would not be signed? Sounds dangerous to me. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140925035052.GA20936@fama
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
W. Martin Borgert wrote: On 2014-09-24 23:05, Hans-Christoph Steiner wrote: * the signature files sign the package contents, not the hash of whole .deb file (i.e. control.tar.gz and data.tar.gz). So preinst and friends would not be signed? Sounds dangerous to me. All package contents would be signed, except the signature itself. The signature would be a separate file in the ar archive of the .deb that signs control.tar.gz and data.tar.gz. See jar or apk format for an example of how this works. .hc -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5423991b.5010...@at.or.at
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
Am 22.09.14 um 01:52 schrieb Paul Wise: On Mon, Sep 22, 2014 at 2:04 AM, Elmar Stellnberger wrote: A package with some new signatures added is no more the old package. That is exactly what we do *not* want for reproducible builds. It should have a different checksum and be made available again for update. The Debian archive does not allow files to change their checksum, so every signature addition requires a new version number. That sounds like a bad idea to me. Yes, that is something we definitely do not want. Nonetheless it would still be an issue to have the package and the signatures in one file because we usually need them together. My only idea to realize this in spite of the said objection would be another proposal: Put the .deb and the signatures into one .ar called .sdeb and make tools like dpkg work on .sdebs or on .deb + signatures respecively. Whenever someone offers some packages for download that will be in the form of .sdebs while official debian repositories may separate both kinds of files. User interfaces like http://debtags.debian.net/search/ could then generate .sdebs on the fly to satisfy petted users. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/541fd8a3.3030...@gmail.com
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 09/22/2014 04:07 AM, Elmar Stellnberger wrote: Am 22.09.14 um 01:52 schrieb Paul Wise: The Debian archive does not allow files to change their checksum, so every signature addition requires a new version number. That sounds like a bad idea to me. Yes, that is something we definitely do not want. Nonetheless it would still be an issue to have the package and the signatures in one file because we usually need them together. My only idea to realize this in spite of the said objection would be another proposal: Put the .deb and the signatures into one .ar called .sdeb and make tools like dpkg work on .sdebs or on .deb + signatures respecively. Whenever someone offers some packages for download that will be in the form of .sdebs while official debian repositories may separate both kinds of files. User interfaces like http://debtags.debian.net/search/ could then generate .sdebs on the fly to satisfy petted users. This is almost exactly what i proposed a couple days ago on the reproducible-builds mailing list [0], except that i used the extension .debs instead of .sdeb :) --dkg [0] http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20140915/000432.html signature.asc Description: OpenPGP digital signature
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 09/21/2014 02:04 PM, Elmar Stellnberger wrote: a well programmed dpkg-cmp. ... and as long as the tool should not be available simply un-ar and compare the data.tar.gz-s. fwiw, this suggestion fails to compare the contents of control.tar.gz, which includes the maintainer scripts (preinst, postinst, etc). If someone wanted to damage your system with a modified package, modified preinst and postinst scripts would be much more effective (they run as root, automatically upon package installation!) than just tweaking a given binary. i just wanted to point out that this theoretical dpkg-cmp is at least slightly more complex than the above suggestion makes it out to be. And of course there are many other tools already that use plain old cmp or digest comparisons against .deb packages already, and thinking about how to interoperate with existing infrastructure is important. --dkg signature.asc Description: OpenPGP digital signature
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
A package with some new signatures added is no more the old package. It should have a different checksum and be made available again for update. Perhaps someone wants to install the package not before certain signatures have been added. Your thought experiment would this way of course require an adjusted toolchain i.e. sth. like dpkg-cmp that outputs differences in the description tags, signatures and file content separately. I believe this would be the best way to do it because you seldomly need to compare two packages while you will need to install individually downloaded packages more often. Just think f.i. about printer driver, device firmware, 3rd party software or singleton packages which you do not want to add to apt. Downloading two files all the time would be far more enervating than a well programmed dpkg-cmp. ... and as long as the tool should not be available simply un-ar and compare the data.tar.gz-s. Am 19.09.14 um 15:16 schrieb Daniel Kahn Gillmor: On 09/19/2014 06:07 AM, Elmar Stellnberger wrote: Isn`t there really any way to include the signatures in the header of the .deb files? Why not simply add multiple signature files in the control.tar.gz of a .deb just next to the md5sums which should in deed be a sha256sums (otherwise there is no way to establish a 'chain of trust'). That would not add any non-determinism because if it is done right then we can have all the signers in the package! If we succeed in creating reproducible builds (we're farther along toward that goal than i had dared hope, it's exciting!) then one of the nice opportunites we have is for other people or organizations to corroborate the build after the package is first distributed. For example, an upload to sid might have one signature (by the original uploader), but maybe it waits to transition to testing until there are corroborations from multiple builders. (Note: this is not a concrete proposal or an expectation of exactly what will happen, just a thought experiment) In this scenario, how do you propose to add those signatures into the package? If we bundle them into the .deb, then the size and digest of the .deb itself changes after it is first distributed. This seems like it would violate all sorts of existing expectations -- i can't imagine how many different tools and pieces of infrastructure expect that foo_1.2-3_mipsel.deb should always have the same size and digest. It would be far better than detaching the signatures from the package because the general use case where you need package signatures is the manual download of packages. Detached signatures are completely useless for such a use case! I don't think this is the case. People who download a .deb and verify it could also download the associated .buildinfo file and whichever signatures they'd like (or all of them) and verify the package that way. --dkg -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/541f134a.3080...@gmail.com
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 2014-09-21 20:04, Elmar Stellnberger wrote: A package with some new signatures added is no more the old package. It should have a different checksum and be made available again for update. Perhaps someone wants to install the package not before certain signatures have been added. If a package would change by adding another signature, then this would invalidate previous signatures. Exactly because of your use case (waiting for a number of signatures or waiting for a specific signature before installation) the signature must be separated from the actual package. -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140921182918.GA27550@fama
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 21 sep. 2014, at 20:29, W. Martin Borgert deba...@debian.org wrote: If a package would change by adding another signature, then this would invalidate previous signatures. Package formats like apk and jar avoid this chicken and egg problem by hashing the files inside a package, and storing those hashes in a manifest file. Signatures only sign the manifest file. The manifest itself and the signature files are not part of the manifest, but are part of the package. So a package including it's signature(s) is still a single file. Richard -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/8ce64b3d-6269-47a6-8cf2-5ecaa631b...@vdberg.org
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 2014-09-21 21:13, Richard van den Berg wrote: Package formats like apk and jar avoid this chicken and egg problem by hashing the files inside a package, and storing those hashes in a manifest file. Is there a chicken and egg problem? Only if one insists on embedding the signatures in one file, I would say. Signatures only sign the manifest file. The manifest itself and the signature files are not part of the manifest, but are part of the package. So a package including it's signature(s) is still a single file. This is nice, indeed, but: The Debian repository is mirrored all over the world and distributed on DVSs/CDs. If package files change whenever a signature is added, this would lead to needless traffic and obliterate readonly media. (Well, rsync would mitigate the mirror problem by only transmitting the signature parts of a file, right?) -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140921205441.GA29763@fama
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On Mon, Sep 22, 2014 at 2:04 AM, Elmar Stellnberger wrote: A package with some new signatures added is no more the old package. That is exactly what we do *not* want for reproducible builds. It should have a different checksum and be made available again for update. The Debian archive does not allow files to change their checksum, so every signature addition requires a new version number. That sounds like a bad idea to me. Perhaps someone wants to install the package not before certain signatures have been added. Thats a good idea and it could certainly be implemented with the design behind reproducible builds as well. Your thought experiment would this way of course require an adjusted toolchain i.e. sth. like dpkg-cmp that outputs differences in the We definitely need a tool like this for reproducible builds and indeed it already exists: https://wiki.debian.org/ReproducibleBuilds#bash_script_to_compare_two_package_builds Reproducible builds and independent verification of those builds by multiple parties -- bye, pabs https://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAKTje6GJWbA7C66SashTWCFixJpdL=yhhk7nv8rkamcktbf...@mail.gmail.com
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On Mon, Sep 22, 2014 at 7:52 AM, Paul Wise wrote: Reproducible builds and independent verification of those builds by multiple parties... sigh... is an essential feature for any operating system in today's security climate. -- bye, pabs https://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-security-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caktje6fwp5vytrjx3yz67lenhmmqyjqkguzumys_td7wvyf...@mail.gmail.com
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 09/19/2014 06:07 AM, Elmar Stellnberger wrote: Isn`t there really any way to include the signatures in the header of the .deb files? Why not simply add multiple signature files in the control.tar.gz of a .deb just next to the md5sums which should in deed be a sha256sums (otherwise there is no way to establish a 'chain of trust'). That would not add any non-determinism because if it is done right then we can have all the signers in the package! If we succeed in creating reproducible builds (we're farther along toward that goal than i had dared hope, it's exciting!) then one of the nice opportunites we have is for other people or organizations to corroborate the build after the package is first distributed. For example, an upload to sid might have one signature (by the original uploader), but maybe it waits to transition to testing until there are corroborations from multiple builders. (Note: this is not a concrete proposal or an expectation of exactly what will happen, just a thought experiment) In this scenario, how do you propose to add those signatures into the package? If we bundle them into the .deb, then the size and digest of the .deb itself changes after it is first distributed. This seems like it would violate all sorts of existing expectations -- i can't imagine how many different tools and pieces of infrastructure expect that foo_1.2-3_mipsel.deb should always have the same size and digest. It would be far better than detaching the signatures from the package because the general use case where you need package signatures is the manual download of packages. Detached signatures are completely useless for such a use case! I don't think this is the case. People who download a .deb and verify it could also download the associated .buildinfo file and whichever signatures they'd like (or all of them) and verify the package that way. --dkg signature.asc Description: OpenPGP digital signature
Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy
On 09/19/2014 12:34 AM, Paul Wise wrote: On Fri, Sep 19, 2014 at 9:30 AM, Hans-Christoph Steiner wrote: Finally did this: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762153 Please note that you proposal to add signatures to .deb files will break reproducible builds because the hash of the .deb will differ depending on who signed it: https://wiki.debian.org/ReproducibleBuilds I think it would be far better to ship detached signatures in the archive since that allows for reproducible builds and also means there could be more than one signer (say one buildd, one Debian sponsor and one package maintainer). I agree with pabs on this. fwiw, i'm also hoping that we can ship at least one other signature for the upstream tarball (where such a thing exists): https://bugs.debian.org/759478 We also had a discussion in the reproducible-builds BoF at DC14 about how to deal with signatures on .buildinfo files, and came to the same conclusion: that a .buildinfo file should have detached signatures, to allow for multiple (corroborative) signers: https://wiki.debian.org/ReproducibleBuilds#A.buildinfo_signatures Note that a signature over a .buildinfo file should effectively cover the digest of the built .deb files, which should creates a strong cryptographic chain if you trust the hash function. Given that we would ultimately like one or more signed .buildinfo files shipped in the archive, and that they represent a way to have an builder's signature over a .deb, i think these make the idea of an internally-signed .deb redundant. Thanks to everyone who is thinking about and working on improving the cryptographic integrity of the archive! --dkg signature.asc Description: OpenPGP digital signature
