Re: Bug#540215: Introduce dh_checksums
On Fri, 16 Apr 2010, Harald Braumann wrote: On Thu, Apr 15, 2010 at 05:03:44PM +0200, Raphael Hertzog wrote: Even if it creates a checksum file, someone could always hand-edit the package to add files not listed in the checksum files and we need to decide whether that's something that needs to be catched and if yes by whom and at what point. Do you mean a maintainer, who hand-edits a package after it was built, or do you mean an adversery who has evil intentions? If the The latter. former, then this should just be forbidden. If the latter, than this can be solved by package signatures. Which one? We are discussing something that is a signature of the (content of the) package. And there's the signature on Release/Package which can authenticate the .deb in its entirety. I'm discussing the case where the signature of the checksums file is valid but that checksums file does not list all the files present in data.tar.gz or control.tar.gz. Cheers, -- Raphaël Hertzog Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/ My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100416060813.ga12...@rivendell
Re: Bug#540215: Introduce dh_checksums
On Fri, Apr 16, 2010 at 08:08:13AM +0200, Raphael Hertzog wrote: I'm discussing the case where the signature of the checksums file is valid but that checksums file does not list all the files present in data.tar.gz or control.tar.gz. Require that checksums exist for all files and let dpkg check that at installation time. But yes, I second your proposal for a DEP, instead of discussing details further in this thread, which has already become quite chaotic. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100416113022.gc25...@sbs288.lan
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Thu, Apr 15, 2010 at 04:04:51PM +0200, Goswin von Brederlow wrote: The checksum file could be attached as additional member in the .deb. And a signature could be a signed file containing the checksum size and name of all members of a .deb preceeding the signature. That way the signature can verify the deb itself or individual members, like the checksum file, in the .deb. Just a thought. I'm not sure, how you mean that exactly. But the signature must be over the checksum file, nothing more and nothing less. Otherwise you won't be able to verify the checksum file. A signature could look like this: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 90d462d27ac404ecabfc9ca7f306dec0b81d3576 3456 control.tar.gz ed43cc24b4f5472d25fc9c82a67daed317c8d415 3573458 data.tar.gz 90d462d27ac404ecab247a82a67daed317c8d415 971 checksum_control ed43cc24b4f5472d25fc9ca7f306dec0b81d3576 1234 checksum_data 9528348234958345473658358238452836482685 3536 signature_01 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iD8DBQFLyHvbH8SBz+0NfPoRAofQAJoDlO38O3UqfcSyN6xj92s/LQlAzwCgweC2 BiK6lI0aABtTwvXVIEiqXNg= =cOUY -END PGP SIGNATURE- Also I think it's really a very bad idea in general to mix multiple different things into one signature. The one thing is a signature over installed files (via the checksum file). The other is a signature over a package. The two are completely orthogonal and serve different purposes. It would be a signature over members of the .deb file. The meaning of each member doesn't matter. harry MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k4s7l9pj@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
Raphael Hertzog hert...@debian.org writes: On Fri, 16 Apr 2010, Harald Braumann wrote: On Thu, Apr 15, 2010 at 05:03:44PM +0200, Raphael Hertzog wrote: Even if it creates a checksum file, someone could always hand-edit the package to add files not listed in the checksum files and we need to decide whether that's something that needs to be catched and if yes by whom and at what point. Do you mean a maintainer, who hand-edits a package after it was built, or do you mean an adversery who has evil intentions? If the The latter. former, then this should just be forbidden. If the latter, than this can be solved by package signatures. Which one? We are discussing something that is a signature of the (content of the) package. And there's the signature on Release/Package which can authenticate the .deb in its entirety. I'm discussing the case where the signature of the checksums file is valid but that checksums file does not list all the files present in data.tar.gz or control.tar.gz. The checksum file can be altered prior to the signature being added. But so can any other part of the .deb file. We have to assume that no adversery with evil intentions has access to the .deb prior to it being signed. So it comes down to the maintainer not screwing up the package prior to uploading. The DAK can verify the validity of the signature and the completness of the checksum file during upload if that is considered neccessary. I do not think every user should have to do so during install. But it could be optional with default off. Cheers, MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87fx2vl99k@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
Hi Wouter, I followed somewhat the discussion and I'm sorry for the delay answering but package signatures are quite low in priority in our roadmap currently: http://wiki.debian.org/Teams/Dpkg/RoadMap We would be glad however if someone would lead this project. I think this project would benefit from a DEP so that we have a spec to refer to and so that we can get buy-in from all other involved parties before going too far in the implementation. Otherwise it will be one more thread without clear outcome... On Tue, 23 Mar 2010, Wouter Verhelst wrote: The idea would be to provide a path from a binary on disk to a GPG signature for installed packages of which the user no longer has the .deb file from which it was originally installed, nor the Packages and/or Release.gpg file that was used to download it. Ok, it looks like a good goal. The proposal as it currently stands would be that: - md5sums are phased out in favour of a more generic 'checksums' file that would use a more secure hash algorithm than MD5 (one of the SHA* family of hashes would probably do at the moment). - When asked to sign a .deb file, dpkg(-deb) would extract the checksums data member from control.tar.gz, create a detached signature for that file, and store it as an additional data member in that .deb. OK, but additional data members in .deb are ignored by dpkg like you noted, this also means that it doesn't end up on the disk after unpack. I assume your proposal is to modify dpkg to extract those and store them somewhere. Is that true? In that case, where and under which format? You have mentioned multiple signatures per package, would they be stored in multiple ar member or in a single one? My only wish at this point is to avoid exploding the number of small administrative files in /var/lib/dpkg/info/ due to this new feature. The biggest downside in your approach is that it's somewhat painful to ensure that all the content of the package is signed. If the checksums files is incomplete, what is supposed to happen? Is that something that dpkg should take care of or should that be outside the scope of dpkg? The benefits of this method as opposed to storing the signature in the control.tar.gz file would be: - Autobuilders would not need to be modified to support signed packages. - Adding a member to an ar file and removing it again later can be done in a way which is idempotent; the same is not true for modifying a tar file. As such, it is trivially possible to remove signatures from a .deb file to allow its verification against the original .changes file that was used for its upload, should this at any point be necessary - If adding multiple signatures is possible in a way which does not modify the contents of the .deb file itself, then the archive-wide key could in theory be used to sign individual .deb files. While a massive increase of CPU power on ftp-master.d.o would be needed to support this, it would allow for easier key management on the end-user end. - It would not break backwards compatibility. I tried this, by manually adding a file signature_01.asc to a .deb file; dpkg was still able to install this package, it just ignored the unknown file. The biggest concern I have with modifying the .deb multiple times along the path is the fact that we're changing the checksum of the whole file and we're not changing its name. Changing this invariant will lead to problems. On the other hand, adding a signature should not require changing the filename either. Did ftpmaster (and buildd maintainers) comment on this problem? the metadata and maintainer scripts in the control.tar.gz file. Doing this would require some way to clarify the difference between data in control.tar.gz and data in data.tar.gz; current suggestions are to either use a prefix (something like CONTROL:preinst, for instance) or to use the path of the binary-as-installed (wherein the above would become /var/lib/dpkg/info/package.preinst). There aren't any strong feelings towards one or the other, however. I prefer the prefix approach because the other one is hardcoding internal information that is not guaranteed to be stable between the dpkg that generates the .deb and the one that is used to unpack it. If any tool outside of dpkg needs to map this to a real file, it should use the appropriate interface (currently it's dpkg-query --control-path package controlfile). As you see there are quite some questions that still need to be cleared up and again I think the DEP process would allow us to answer them progressively and end up with a clear agreed-upon plan. Cheers, -- Raphaël Hertzog Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/ My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive:
Re: Bug#540215: Introduce dh_checksums
Raphael Hertzog hert...@debian.org writes: Hi Wouter, I followed somewhat the discussion and I'm sorry for the delay answering but package signatures are quite low in priority in our roadmap currently: http://wiki.debian.org/Teams/Dpkg/RoadMap We would be glad however if someone would lead this project. I think this project would benefit from a DEP so that we have a spec to refer to and so that we can get buy-in from all other involved parties before going too far in the implementation. Otherwise it will be one more thread without clear outcome... On Tue, 23 Mar 2010, Wouter Verhelst wrote: The idea would be to provide a path from a binary on disk to a GPG signature for installed packages of which the user no longer has the .deb file from which it was originally installed, nor the Packages and/or Release.gpg file that was used to download it. Ok, it looks like a good goal. The proposal as it currently stands would be that: - md5sums are phased out in favour of a more generic 'checksums' file that would use a more secure hash algorithm than MD5 (one of the SHA* family of hashes would probably do at the moment). - When asked to sign a .deb file, dpkg(-deb) would extract the checksums data member from control.tar.gz, create a detached signature for that file, and store it as an additional data member in that .deb. OK, but additional data members in .deb are ignored by dpkg like you noted, this also means that it doesn't end up on the disk after unpack. I assume your proposal is to modify dpkg to extract those and store them somewhere. Is that true? In that case, where and under which format? You have mentioned multiple signatures per package, would they be stored in multiple ar member or in a single one? My only wish at this point is to avoid exploding the number of small administrative files in /var/lib/dpkg/info/ due to this new feature. The introduction of multiarch will need to change the way metadata is stored there. Since some change is needed anyway it might be a good time to adapt to the increase in files stored there and use some subdirs. Maybe even have one dir per package. The biggest downside in your approach is that it's somewhat painful to ensure that all the content of the package is signed. If the checksums files is incomplete, what is supposed to happen? Is that something that dpkg should take care of or should that be outside the scope of dpkg? Yes, dpkg should create the checksum file. The checksum file could be attached as additional member in the .deb. And a signature could be a signed file containing the checksum size and name of all members of a .deb preceeding the signature. That way the signature can verify the deb itself or individual members, like the checksum file, in the .deb. Just a thought. The benefits of this method as opposed to storing the signature in the control.tar.gz file would be: - Autobuilders would not need to be modified to support signed packages. - Adding a member to an ar file and removing it again later can be done in a way which is idempotent; the same is not true for modifying a tar file. As such, it is trivially possible to remove signatures from a .deb file to allow its verification against the original .changes file that was used for its upload, should this at any point be necessary - If adding multiple signatures is possible in a way which does not modify the contents of the .deb file itself, then the archive-wide key could in theory be used to sign individual .deb files. While a massive increase of CPU power on ftp-master.d.o would be needed to support this, it would allow for easier key management on the end-user end. - It would not break backwards compatibility. I tried this, by manually adding a file signature_01.asc to a .deb file; dpkg was still able to install this package, it just ignored the unknown file. The biggest concern I have with modifying the .deb multiple times along the path is the fact that we're changing the checksum of the whole file and we're not changing its name. Changing this invariant will lead to problems. On the other hand, adding a signature should not require changing the filename either. Did ftpmaster (and buildd maintainers) comment on this problem? the metadata and maintainer scripts in the control.tar.gz file. Doing this would require some way to clarify the difference between data in control.tar.gz and data in data.tar.gz; current suggestions are to either use a prefix (something like CONTROL:preinst, for instance) or to use the path of the binary-as-installed (wherein the above would become /var/lib/dpkg/info/package.preinst). There aren't any strong feelings towards one or the other, however. I prefer the prefix approach because the other one is hardcoding internal information that is not guaranteed to be stable between the dpkg that generates the .deb and the one that is used to unpack it. If any tool
Re: Bug#540215: Introduce dh_checksums
On Thu, Apr 15, 2010 at 02:44:07PM +0200, Raphael Hertzog wrote:
On Tue, 23 Mar 2010, Wouter Verhelst wrote:
The idea would be to provide a path from a binary on disk to a GPG
signature for installed packages of which the user no longer has the
.deb file from which it was originally installed, nor the Packages
and/or Release.gpg file that was used to download it.
Ok, it looks like a good goal.
Now that snapshot.debian.org is officially deployed (and I can't stop
thanking DSA and the other involved parties for that), let me highlight
another potential advantage of reaching this goal.
snapshot.d.o now has a really nice lookup interface from (SHA1) checksum
to the actual file [1]. So having an easy tool to retrieve the (SHA1)
checksum of a given file installed on disk would make trivial
re-downloading the corresponding .deb even years later (which would be
*awesome*).
[1]
http://git.debian.org/?p=mirror/snapshot.debian.org.git;a=blob_plain;f=API;hb=HEAD
Keeping this goal in mind would mean either choosing SHA1 as hash format
or considering the possibility of extending the snapshot.d.o code to
support other hashes (I'm sure that Peter welcome patches :-)).
As you see there are quite some questions that still need to be cleared up
and again I think the DEP process would allow us to answer them
progressively and end up with a clear agreed-upon plan.
Ditto.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Thu, 15 Apr 2010, Goswin von Brederlow wrote: My only wish at this point is to avoid exploding the number of small administrative files in /var/lib/dpkg/info/ due to this new feature. The introduction of multiarch will need to change the way metadata is stored there. Since some change is needed anyway it might be a good time to adapt to the increase in files stored there and use some subdirs. Maybe even have one dir per package. My concern is not about the number of files per directory, my concern is the overall number of files eating 4 Kb of filesystem space for a few hundreds of bytes each in reality. The biggest downside in your approach is that it's somewhat painful to ensure that all the content of the package is signed. If the checksums files is incomplete, what is supposed to happen? Is that something that dpkg should take care of or should that be outside the scope of dpkg? Yes, dpkg should create the checksum file. Even if it creates a checksum file, someone could always hand-edit the package to add files not listed in the checksum files and we need to decide whether that's something that needs to be catched and if yes by whom and at what point. Cheers, -- Raphaël Hertzog Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/ My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100415150344.ga8...@rivendell
Re: Bug#540215: Introduce dh_checksums
On Thu, 15 Apr 2010, Stefano Zacchiroli wrote: On Thu, Apr 15, 2010 at 02:44:07PM +0200, Raphael Hertzog wrote: On Tue, 23 Mar 2010, Wouter Verhelst wrote: The idea would be to provide a path from a binary on disk to a GPG signature for installed packages of which the user no longer has the .deb file from which it was originally installed, nor the Packages and/or Release.gpg file that was used to download it. Ok, it looks like a good goal. Now that snapshot.debian.org is officially deployed (and I can't stop thanking DSA and the other involved parties for that), let me highlight another potential advantage of reaching this goal. snapshot.d.o now has a really nice lookup interface from (SHA1) checksum to the actual file [1]. So having an easy tool to retrieve the (SHA1) checksum of a given file installed on disk would make trivial re-downloading the corresponding .deb even years later (which would be *awesome*). Hu?! Retrieving the SHA1 checksum is done by running sha1sum /the/file... I don't see what dpkg would bring here. Furthermore, the content of a file might not change at each release which means it's not a one-to-one mapping but a one-to-many mapping. And snapshot stores the SHA1 of .deb, .dsc and related files but not the content of the .deb. I'm really confused at what you were trying to suggest. Cheers, -- Raphaël Hertzog Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/ My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100415151439.gb8...@rivendell
Re: Bug#540215: Introduce dh_checksums
On Thu, Apr 15, 2010 at 05:14:39PM +0200, Raphael Hertzog wrote:
On Tue, 23 Mar 2010, Wouter Verhelst wrote:
The idea would be to provide a path from a binary on disk to a GPG
signature for installed packages of which the user no longer has the
.deb file from which it was originally installed, nor the Packages
and/or Release.gpg file that was used to download it.
snip
Hu?! Retrieving the SHA1 checksum is done by running sha1sum
/the/file... I don't see what dpkg would bring here. Furthermore,
the content of a file might not change at each release which means it's
not a one-to-one mapping but a one-to-many mapping.
The scenario suggested by Wouter quote above is that the user has
deleted *part* of an installed package (e.g. a mistaken rm somewhere
under /usr/share/package/), but she no longer has the corresponding
.deb. Under that assumption, while the user can sha1sum /the/file, she
can no longer sha1sum /the/.deb; so there is no way to lookup
snapshot.d.o to retrieve the .deb.
It is my understanding that achieving the goal that you and Wouter
agreed upon would provide the step /the/file -- checksum of the
owning .deb. If this is the case, the circle is closed.
I'm really confused at what you were trying to suggest.
Any better? (even though it is not yet clear to me what was not clear in
my post, sorry about that)
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Thu, 15 Apr 2010, Stefano Zacchiroli wrote: On Thu, Apr 15, 2010 at 05:14:39PM +0200, Raphael Hertzog wrote: On Tue, 23 Mar 2010, Wouter Verhelst wrote: The idea would be to provide a path from a binary on disk to a GPG signature for installed packages of which the user no longer has the .deb file from which it was originally installed, nor the Packages and/or Release.gpg file that was used to download it. snip Hu?! Retrieving the SHA1 checksum is done by running sha1sum /the/file... I don't see what dpkg would bring here. Furthermore, the content of a file might not change at each release which means it's not a one-to-one mapping but a one-to-many mapping. The scenario suggested by Wouter quote above is that the user has deleted *part* of an installed package (e.g. a mistaken rm somewhere under /usr/share/package/), I did not read this in his words. I read that he wants to verify that the installed files correspond to files that were signed by Debian without having to keep around .deb files and/or Packages/Release files. It is my understanding that achieving the goal that you and Wouter agreed upon would provide the step /the/file -- checksum of the owning .deb. If this is the case, the circle is closed. Not in any straightforward way AFAIK. We get a checksum file for each package listing the SHA1 of all installed files but that's all. If you want the checksum of the owning .deb you have to record it at installation time, you can't reconstruct it from the installed files even with the new checksum file. Cheers, -- Raphaël Hertzog Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/ My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100415161202.gb8...@rivendell
Re: Bug#540215: Introduce dh_checksums
On Thu, Apr 15, 2010 at 05:03:44PM +0200, Raphael Hertzog wrote: Even if it creates a checksum file, someone could always hand-edit the package to add files not listed in the checksum files and we need to decide whether that's something that needs to be catched and if yes by whom and at what point. Do you mean a maintainer, who hand-edits a package after it was built, or do you mean an adversery who has evil intentions? If the former, then this should just be forbidden. If the latter, than this can be solved by package signatures. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100416010251.ga25...@sbs288.lan
Re: Bug#540215: Introduce dh_checksums
On Thu, Apr 15, 2010 at 04:04:51PM +0200, Goswin von Brederlow wrote: The checksum file could be attached as additional member in the .deb. And a signature could be a signed file containing the checksum size and name of all members of a .deb preceeding the signature. That way the signature can verify the deb itself or individual members, like the checksum file, in the .deb. Just a thought. I'm not sure, how you mean that exactly. But the signature must be over the checksum file, nothing more and nothing less. Otherwise you won't be able to verify the checksum file. Also I think it's really a very bad idea in general to mix multiple different things into one signature. The one thing is a signature over installed files (via the checksum file). The other is a signature over a package. The two are completely orthogonal and serve different purposes. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100416011001.gb25...@sbs288.lan
Re: Bug#540215: Introduce dh_checksums
On Sat, Mar 20, 2010 at 01:27:52PM +0100, Harald Braumann wrote: On Fri, Mar 19, 2010 at 05:56:40PM -0700, Russ Allbery wrote: Harald Braumann ha...@unheit.net writes: On Thu, Mar 18, 2010 at 04:52:07PM -0700, Russ Allbery wrote: You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. Wouldn't it be simpler to just extract *sums from control.tar.gz, create a detached signature for it and put it in the ar archive, instead of extracting data.tar.gz and generating the sums a second time? Or would this replace dh_*sums during package build time? I think it would replace dh_*sums during package build time and make obsolete including md5sums in the control.tar.gz. You don't really want the signature and checksums to be inside one of the other data members since that breaks, as Wouter points out, the ability to remove the signature and checksums and verify against the original *.changes file. And there's no need to include two copies of the checksums. There would only be one additional file, containing a detached signature for the checksum file. No duplication of checksums and it can easily be removed from the ar. But doing everything in one step, like you proposed, is better anyway. Yes, agreed; detached signatures are probably better, since they would allow for multiple signatures without the requirement of having several copies of the checksums file in the .deb. What's more, if the checksums are still in the control.tar.gz, this would provide some level of backwards compatibility with a version of dpkg that doesn't support the detached signatures yet (i.e., a package installed with that version would still have the checksums on disk, though not the signatures). Since I haven't seen any input from the dpkg developers to this thread (-dpkg Cc'ed for their benefit), and since we seem to have hammered out a proposal that would involve some change to dpkg, let me summarize the proposal and ask for their input. The idea would be to provide a path from a binary on disk to a GPG signature for installed packages of which the user no longer has the .deb file from which it was originally installed, nor the Packages and/or Release.gpg file that was used to download it. The proposal as it currently stands would be that: - md5sums are phased out in favour of a more generic 'checksums' file that would use a more secure hash algorithm than MD5 (one of the SHA* family of hashes would probably do at the moment). - When asked to sign a .deb file, dpkg(-deb) would extract the checksums data member from control.tar.gz, create a detached signature for that file, and store it as an additional data member in that .deb. The benefits of this method as opposed to storing the signature in the control.tar.gz file would be: - Autobuilders would not need to be modified to support signed packages. - Adding a member to an ar file and removing it again later can be done in a way which is idempotent; the same is not true for modifying a tar file. As such, it is trivially possible to remove signatures from a .deb file to allow its verification against the original .changes file that was used for its upload, should this at any point be necessary - If adding multiple signatures is possible in a way which does not modify the contents of the .deb file itself, then the archive-wide key could in theory be used to sign individual .deb files. While a massive increase of CPU power on ftp-master.d.o would be needed to support this, it would allow for easier key management on the end-user end. - It would not break backwards compatibility. I tried this, by manually adding a file signature_01.asc to a .deb file; dpkg was still able to install this package, it just ignored the unknown file. If we're going to sign .deb files, then it would make sense to also sign the metadata and maintainer scripts in the control.tar.gz file. Doing this would require some way to clarify the difference between data in control.tar.gz and data in data.tar.gz; current suggestions are to either use a prefix (something like CONTROL:preinst, for instance) or to use the path of the binary-as-installed (wherein the above would become /var/lib/dpkg/info/package.preinst). There aren't any strong feelings towards one or the other, however. What's the opinion of the dpkg maintainers on this? -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Fri, Mar 19, 2010 at 05:56:40PM -0700, Russ Allbery wrote: Harald Braumann ha...@unheit.net writes: On Thu, Mar 18, 2010 at 04:52:07PM -0700, Russ Allbery wrote: You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. Wouldn't it be simpler to just extract *sums from control.tar.gz, create a detached signature for it and put it in the ar archive, instead of extracting data.tar.gz and generating the sums a second time? Or would this replace dh_*sums during package build time? I think it would replace dh_*sums during package build time and make obsolete including md5sums in the control.tar.gz. You don't really want the signature and checksums to be inside one of the other data members since that breaks, as Wouter points out, the ability to remove the signature and checksums and verify against the original *.changes file. And there's no need to include two copies of the checksums. There would only be one additional file, containing a detached signature for the checksum file. No duplication of checksums and it can easily be removed from the ar. But doing everything in one step, like you proposed, is better anyway. And then create a second signature over all files in the ar archive directly. This one would be checked before extracting the containing tar.gz files, and the other one would be installed along with the *sums file. I think you want to checksum the underlying contents, not the *.tar.gz files in the ar archive. As Joey can attest to from writing pristine-tar, it's surprisingly difficult to reproduce a *.tar.gz file from its members. Misunderstanding. Forget what I said. To include checksums for control.tar.gz, just add them to the same checksum file, but with the paths, the files will have after package installation (/var/lib/dpkg/...). harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100320122752.gd1...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Fri, Mar 19, 2010 at 05:56:40PM -0700, Russ Allbery wrote: I think it would replace dh_*sums during package build time and make obsolete including md5sums in the control.tar.gz. You don't really want the signature and checksums to be inside one of the other data members since that breaks, as Wouter points out, the ability to remove the signature and checksums and verify against the original *.changes file. And there's no need to include two copies of the checksums. There would only be one additional file, containing a detached signature for the checksum file. No duplication of checksums and it can easily be removed from the ar. But doing everything in one step, like you proposed, is better anyway. Oh, I see what you're saying. Yeah, that could work too. To include checksums for control.tar.gz, just add them to the same checksum file, but with the paths, the files will have after package installation (/var/lib/dpkg/...). Yeah, that would be one such convention. I don't know if that's better or if adding a prefix of data: and control: to the path names would be better. My guess is that the latter may be a bit more flexible for possible long-term changes, like adding other deb members later for some reason that we want to sign. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/878w9ncf11@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Sat, Mar 20, 2010 at 06:13:14AM -0700, Russ Allbery wrote: Yeah, that would be one such convention. I don't know if that's better or if adding a prefix of data: and control: to the path names would be better. My guess is that the latter may be a bit more flexible for possible long-term changes, like adding other deb members later for some reason that we want to sign. But aren't we talking about checksums of installed files here? So after package installation I'd like to have the file as /var/lib/dpkg/info/packag.checksums, just like the md5sums now, only that it's signed (preferably with a detached signature). This file has to be included verbatim in the package. You can't strip the data:/control: prefix on installation, as this would invalidate the signature. And it shouldn't be installed containing these prefixes, because then you can't use standard-tools to verify the checksums. If other stuff should be added later, for instance debsigs like signatures, then additional files can be added to the deb. I don't think it's wise trying to define a catch-all format now and I don't see why arbitray additional files for further extensions couldn't be added to the deb later. All these files could be packed together in, say, security.tar.gz, so you can always remove this single member from the ar to get the classic deb. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100320154020.ge1...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Sat, Mar 20, 2010 at 06:13:14AM -0700, Russ Allbery wrote: Yeah, that would be one such convention. I don't know if that's better or if adding a prefix of data: and control: to the path names would be better. My guess is that the latter may be a bit more flexible for possible long-term changes, like adding other deb members later for some reason that we want to sign. But aren't we talking about checksums of installed files here? So after package installation I'd like to have the file as /var/lib/dpkg/info/packag.checksums, just like the md5sums now, only that it's signed (preferably with a detached signature). This file has to be included verbatim in the package. You can't strip the data:/control: prefix on installation, as this would invalidate the signature. And it shouldn't be installed containing these prefixes, because then you can't use standard-tools to verify the checksums. I agree with all of that; I'm just not sure the last bit actually matters. It's trivial to write a tiny tool that would verify the checksums using other tools. But I can see the appeal, and I wouldn't argue against using the installed path either. Note, though, that if only installed files can be listed in the signature, the signature doesn't cover DEBIAN/control file, which seems like a bad idea. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k4t6bz3q@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Thu, 2010-03-18 at 12:39 +0100, Harald Braumann wrote: On Thu, Mar 18, 2010 at 08:31:40AM +0100, Goswin von Brederlow wrote: Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Self-contained packages, where the signature is included and installed along with the checksum file, would have a lot of advantages. You wouldn't need access to a lot of infrastructure just to verify a signature. It would be very simple. It could be used for packages, that are not part of Debian. For instance, I could produce a package and send it to a friend and he could later use my key for verification. Oh please no. Don't advocate sending individual .deb files, ever. This practice should be strongly discouraged. One brilliant part of Debian packaging *is* the APT infrastructure, some key features: 1. Security updates 2. Bug fixes 4. Dependency resolution 5. Smoother dist-upgrades because: 5a. The APT repository provides newer version, with updated dependencies (libraries transitions...) 5b. The user don't have to visit each web site to dist-upgrade 6. Single GPG key to manage (revocation ; update...) 7. Single GPG key to trust (per repository) If people and ISV start publishing individual .deb, they (and we) will have to face the same problem as Windows/Mac/whatever had to solve: each application will need to embed a feature to Check for update, etc. I am spending about 2 hours every two month on my parents computer, just update all the damned Windows applications. Who really wants Debian to go down that say. I must say that if someone can't setup an APT repository to publish packages, you should reconsider the installation of any package from that person/ISV. (Reminder the Debian Policy has 135 pages, whom ever can read and use it to create a proper package can also read a few manpages to setup a repository). The same stand for RPM co. cat /home/fpiat/2¢ debian-devel Regards, Franklin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1268986453.3488.183.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Thu, Mar 18, 2010 at 08:31:40AM +0100, Goswin von Brederlow wrote: Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Further, the build part of a buildd runs inside a limited chroot running the target distribution, i.e. usually unstable (the real system runs stable with a backported version of sbuild), which doesn't have access to any key material in the real system. At the moment buildds don't have their own keys: a buildd maintainer inspects the build log and signs the .changes file for upload. Even for maintainer uploads, maintainers who build their packages in a minimal chroot with schroot, pbuilder, cowbuilder etc. (which is strongly recommended) don't necessarily have their signing key available inside the chroot (nor should they!). Signatures per buildd or per DD doing uploads are moderately interesting, but not nearly as interesting as a signature by a long-term stable key such as the archive key. Do we actually rely anywhere on packages not changing hashes between upload and publication in the repository, or is it just something we have as an invariant now because there's no reason for it not to be one? The path of least resistance here would be for DAK to add the package signature after verifying the signature of the uploader. This has the drawback that it modifies the *.deb and therefore breaks the hashes in the *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, do we actually care? The changes files are afaik archived somewhere and are needed to verify the archive integrity after a (feared) compromise. And what do you do when the archive key expires? Resign every deb in the archive and have every mirror download them all again? Why? A signature doesn't become invalid just because the key expires, as long as the signature was created before the expiration of the key. Same problem on releases. Releasing a new stable means a new stable key so every deb needs to be signed again and changes. I don't think this is a good idea. This would also cause users (apt/aptitude actually) to reinstall the packages needlessly creating even more mirror load. For this to work the signature for the checksum files should be detached so that it can be changed/extended without altering the deb files. I suggested this before but lets repeat it. Why not include a digest of the checksum file in DEBIAN/control, carry it into the changes file and into Packages.gz. That way all current debs automatically have a clear trust path. Further the changes files could be included in the pool alongside the debs and source files. That way everyone can verify the autenticity of the debs (or just the checksum file) independent of the archive key. Going one step further the changes files could be signed by the archive key(s) as well. Adding a new signature to them when keys change does mean they need to be mirrored again but they are relatively small. Self-contained packages, where the signature is included and installed along with the checksum file, would have a lot of advantages. You wouldn't need access to a lot of infrastructure just to verify a signature. It would be very simple. It could be used for packages, that are not part of Debian. For instance, I could produce a package and send it to a friend and he could later use my key for verification. The same holds for projects that publish deb files. With your proposal they would need a full fledged archive and keep a long history of all the files. You can always sign the deb. The tools to sign and verify are all present. Only ftp-master stands in the way of using that. And you could automatically download the changes files along with every deb and keep all changes files for installed package/version locally. Anyway, I don't consider a ftp/http client a lot of infrastructure. It would be trivial to write a tool that downloads the changes files for every installed package and verifies it. And what do you do with packages from testing/unstable/experimental? Would you keep all incarnations of the Release/Packages/changes files? And if I want to verify the signature of an installed file, from a package I once installed from, say, unstable, how would I find the right version of the changes file for the package? All changes files are already kept. And you would go directly to fetching the changes file for the package/version you have installed. All it would need is for the changes file archive to become public. Cheers, harry MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
Re: Bug#540215: Introduce dh_checksums
Russ Allbery r...@debian.org writes: Goswin von Brederlow goswin-...@web.de writes: And what do you do when the archive key expires? Why would you need to do anything at all when the archive key expires? Keys don't become magically compromised or worthless just because they've expired. All it means is that you can't trust the integrity of really old signatures as much as you can trust the integrity of new ones. s/expires/is compromised/ Same problem on releases. Releasing a new stable means a new stable key so every deb needs to be signed again and changes. I don't see why. The only *.debs that we might need to resign are ones where there have been no uploads for an entire release cycle and hence may be released with expired signatures, and while there are a few of those, that's a much smaller problem. You could even just do an all-arch binNMU to deal with resigning those. Packages are uploaded to unstable usualy and won't be signed by a stable key at all. At most they get signed by the automatic archive key, which is always in danger of being compromised. So on release every package should be signed by the new stable key. Or at least all those that where uploaded since the last release, which would be the majority. For this to work the signature for the checksum files should be detached so that it can be changed/extended without altering the deb files. We could do that as well, but it would require changing the binary package format and the archive software to track an additional file, and I think that's a much larger change. Changes files are alraedy archived. Just make them public. I suggested this before but lets repeat it. Why not include a digest of the checksum file in DEBIAN/control, carry it into the changes file and into Packages.gz. That way all current debs automatically have a clear trust path. Multiple people have explained to you why this doesn't solve the problem: it means that you lose your signature path as soon as the package is no longer listed in Packages. Which is why I kept writing the next paragraph. I'm not interested in new ways of authenticating packages that are still current and still listed in Packages. That's a solved problem, and while we can provide some moderate additional convenience, it's not really enough to justify the work. I'm interested in ways of authenticating packages that *aren't* listed in Packages, either because they're unofficial or because they're old and have been superseded. That would be a useful new feature. Further the changes files could be included in the pool alongside the debs and source files. That way everyone can verify the autenticity of the debs (or just the checksum file) independent of the archive key. Going one step further the changes files could be signed by the archive key(s) as well. Adding a new signature to them when keys change does mean they need to be mirrored again but they are relatively small. That's an interesting idea. Yes, we could add additional signatures to the *.changes file without any of this other impact. To solve the full problem for the Debian archive, we'd need to provide all the *.changes files for every *.deb that we've ever released, but thankfully they're small. And exist for past debs as well. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87sk7w4pdx@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
Russ Allbery r...@debian.org writes: Goswin von Brederlow goswin-...@web.de writes: The changes files are signed by a human and therefor have a strong trust level. The was XYZ is now UVW file would have to be automatically signed and much less trustworthy. This objection makes no sense to me. The archive key is *much* more trusted in practice than the individual DD keys. Haven't you been advocating using the Packages file for this purpose, which is signed by exactly the same key that would be doing this signature? There are different keys signing the Packages files. I trust a DDs key over the Debian Archive Automatic Signing Key, esspecially when it has expired or was potentially compromised. On the other hand the XYZ Stable Release Key is probably more trustworthy than $RANDOM DDs. They are kept offline, right? Also users would download the Packages when the key is still valid or get it freshly signed by the new key after a compromise. On the other hand the was XYZ is now UVW file would be essential in verifying that the master archive on ftp-master was not altered after a compromise. And if ftp-master was compomised then the automatic signing key is compromised and all the was XYZ is now UVW files are useless. The difference is that the was XYZ is now UVW file would be on the same system as the key. You might as well not sign them as the availability of the key make the signature pointless once the system is compromised. Esspecially if you suspect someone broke into ftp-master and modified some debs. Which they can do just as easily if you rely only on Packages. Even more easily, in fact. After a compromise the Packages files are only suspect untill the archive has been verified to be pristine and signed by a new key. So it is only a short term problem for users. The was XYZ is now UVW file on the other hand would permanently sever the trust chain in case of a compromise. The problems that you are citing here are problems that we already have; that, in fact, are much worse now than they would be under that proposed scheme. (However, I will note that your *.changes idea does offer some additional protection there over the scheme that I was considering.) Lets hope they will be made more public. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87ocik4osm@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
On Fri, Mar 19, 2010 at 09:14:13AM +0100, Frank Lin PIAT wrote: On Thu, 2010-03-18 at 12:39 +0100, Harald Braumann wrote: On Thu, Mar 18, 2010 at 08:31:40AM +0100, Goswin von Brederlow wrote: Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Self-contained packages, where the signature is included and installed along with the checksum file, would have a lot of advantages. You wouldn't need access to a lot of infrastructure just to verify a signature. It would be very simple. It could be used for packages, that are not part of Debian. For instance, I could produce a package and send it to a friend and he could later use my key for verification. Oh please no. Don't advocate sending individual .deb files, ever. This practice should be strongly discouraged. One brilliant part of Debian packaging *is* the APT infrastructure, some key features: It's local software that's relevant for me and maybe 3 other people. I don't think Debian would accept it in the archive. And I'm not going to set up an APT infrastructure for this either, because it's simply not needed. If people and ISV start publishing individual .deb, they (and we) will have to face the same problem as Windows/Mac/whatever had to solve: each application will need to embed a feature to Check for update, etc. These are exceptions, it's not like suddenly everyone starts publishing their own debs. But why shouldn't an implementation also support this? harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100319102357.ga1...@nn.nn
Re: Bug#540215: Introduce dh_checksums
On Fri, Mar 19, 2010 at 10:38:24AM +0100, Goswin von Brederlow wrote: You can always sign the deb. The tools to sign and verify are all present. Only ftp-master stands in the way of using that. I would love signed debs. But this is orthogonal to signed checksum files and should probably discussed separately. And you could automatically download the changes files along with every deb and keep all changes files for installed package/version locally. Anyway, I don't consider a ftp/http client a lot of infrastructure. It would be trivial to write a tool that downloads the changes files for every installed package and verifies it. The central repository is the infrastracture, not the http client. All changes files are already kept. And you would go directly to fetching the changes file for the package/version you have installed. All it would need is for the changes file archive to become public. If the signature was part of the package, this wasn't needed. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100319103042.gb1...@nn.nn
Re: Bug#540215: Introduce dh_checksums
On Thu, Mar 18, 2010 at 04:52:07PM -0700, Russ Allbery wrote: You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. That'd work pretty well, indeed. It would also have the advantage of making it theoretically possible to reverse the addition of the signatures again, should one want to re-verify against the original .changes file for some reason. That's of course assuming that the combination of ar a and ar d in whatever way dpkg does that is idempotent, but I don't see why it couldn't be. And as you say, this can be implemented in dak. That would have the advantage of not requiring keys on the buildds. So now that it's been reduced to a technical problem, who's going to do the implementation? I'm prepared to look at a dpkg patch, but Python just does not work for me. There are other implementation methods possible, but that's probably the most obvious one. Yes, agreed. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Fri, Mar 19, 2010 at 09:14:13AM +0100, Frank Lin PIAT wrote: On Thu, 2010-03-18 at 12:39 +0100, Harald Braumann wrote: On Thu, Mar 18, 2010 at 08:31:40AM +0100, Goswin von Brederlow wrote: Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Self-contained packages, where the signature is included and installed along with the checksum file, would have a lot of advantages. You wouldn't need access to a lot of infrastructure just to verify a signature. It would be very simple. It could be used for packages, that are not part of Debian. For instance, I could produce a package and send it to a friend and he could later use my key for verification. Oh please no. Don't advocate sending individual .deb files, ever. That already happens, will always happen, and it cannot be avoided. There are perfectly valid use cases for using individual .deb files. To give but one example that comes directly from my day job: I build an image for an embedded system that boots off a read-only /. Since we can't modify our filesystem after booting, the way an update is done is through regenerate the image and boot off a USB stick, rather than apt-get update. Since the latter doesn't work anyway, setting up a full repository with Packages file and whatnot is vast overkill, so the software that was written specifically for this system is packaged as a .deb file that is not in a repository, anywhere. This practice should be strongly discouraged. One brilliant part of Debian packaging *is* the APT infrastructure, some key features: In the above example: 1. Security updates Does not apply (when the devices are connected to a network, that means they're either undergoing maintenance or someone is trying to break into the system) 2. Bug fixes If it ain't broken, don't fix it applies even more to this kind of embedded systems than it does to servers. 4. Dependency resolution apt-get -f install 5. Smoother dist-upgrades because: 5a. The APT repository provides newer version, with updated dependencies (libraries transitions...) the custom software does not include libraries 5b. The user don't have to visit each web site to dist-upgrade hah. 6. Single GPG key to manage (revocation ; update...) I don't understand what you mean by that. 7. Single GPG key to trust (per repository) Well, signatures on .deb files would also have a single GPG key to manage, per source. There's no difference here, really. It's important to remember that whatever environment you're using Debian in, is not necessarily the *only* environment Debian is used in. Since a method that will work for individual .debs will also work for archive-wide installations, there really is no problem in doing it in such a way that it will work for both ways. I agree that for most cases, setting up a proper repository is the best way to distribute software. But there are exceptions, and even in those cases having a path from the binaries on disk to a GPG signature is a good thing. [...] -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
Wouter Verhelst wrote: On Fri, Mar 19, 2010 at 09:14:13AM +0100, Frank Lin PIAT wrote: On Thu, 2010-03-18 at 12:39 +0100, Harald Braumann wrote: On Thu, Mar 18, 2010 at 08:31:40AM +0100, Goswin von Brederlow wrote: Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Self-contained packages, where the signature is included and installed along with the checksum file, would have a lot of advantages. You wouldn't need access to a lot of infrastructure just to verify a signature. It would be very simple. It could be used for packages, that are not part of Debian. For instance, I could produce a package and send it to a friend and he could later use my key for verification. Oh please no. Don't advocate sending individual .deb files, ever. That already happens, will always happen, and it cannot be avoided. I wish we could avoid it for end-users consumer products. There are perfectly valid use cases for using individual .deb files. To give but one example that comes directly from my day job: I build an image for an embedded system that boots off a read-only /. You build, you deploy, you trust your self. Is is the best example out there ? I must admit it is still a valid example. Since we can't modify our filesystem after booting, the way an update is done is through regenerate the image and boot off a USB stick, rather than apt-get update. Since the latter doesn't work anyway, setting up a full repository with Packages file and whatnot is vast overkill, so the software that was written specifically for this system is packaged as a .deb file that is not in a repository, anywhere. This practice should be strongly discouraged. One brilliant part of Debian packaging *is* the APT infrastructure, some key features: In the above example: 1. Security updates Does not apply (when the devices are connected to a network, that means they're either undergoing maintenance or someone is trying to break into the system) In your scenario, the system adminitrator fetch the source from the distributor (That can/should be done through an APT mirror), then deploy the managed systems. 2. Bug fixes If it ain't broken, don't fix it applies even more to this kind of embedded systems than it does to servers. In my organisation, we do deploy service pack and point release, Even though the perceived state would suggest If it ain't broken, don't fix it 4. Dependency resolution apt-get -f install 5. Smoother dist-upgrades because: 5a. The APT repository provides newer version, with updated dependencies (libraries transitions...) the custom software does not include libraries Don't they include libraries? Don't they depend on Debian libraries? Wouldn't that prevent smooth upgrades? And more important, who the users are going to blame? The broken package or Debian (my conviction is that people will blame the OS vendor) 6. Single GPG key to manage (revocation ; update...) I don't understand what you mean by that. If each package is signed by a DD, a system admin have to manage multiple public keys. 7. Single GPG key to trust (per repository) Well, signatures on .deb files would also have a single GPG key to manage, per source. There's no difference here, really. It's important to remember that whatever environment you're using Debian in, is not necessarily the *only* environment Debian is used in. This is a very valid point. Since a method that will work for individual .debs will also work for archive-wide installations, there really is no problem in doing it in such a way that it will work for both ways. Yes, I am really concerned with the consumer products users. (and the usual do the easiet and faster way, long term consequences doesn't matter, see teh banner on [1]) BTW in 12 days, I will send a AUTORUN.DEB proposal: Any package named AUTORUN.DEB present on a removable media should be automatically installed when that media in inserted. ;-) Regards, Franklin [1] http://packages.debian.org/lenny/i386/dash/download -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/7bf34c766afd2f66985ae7f17b3028cc.squir...@www.klabs.be
Re: Bug#540215: Introduce dh_checksums
On Fri, 2010-03-19 at 17:40 +0100, Wouter Verhelst wrote: On Thu, Mar 18, 2010 at 04:52:07PM -0700, Russ Allbery wrote: You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. That'd work pretty well, indeed. It would also have the advantage of making it theoretically possible to reverse the addition of the signatures again, should one want to re-verify against the original .changes file for some reason. That's of course assuming that the combination of ar a and ar d in whatever way dpkg does that is idempotent, but I don't see why it couldn't be. And as you say, this can be implemented in dak. That would have the advantage of not requiring keys on the buildds. So now that it's been reduced to a technical problem, who's going to do the implementation? Yes, this solution is elegant. It shouldn't break anything, it is self-contained in the package. I'm prepared to look at a dpkg patch, but Python just does not work for me. My priority is the md5sum replacement, but I'll be happy to help if/when I can. Regards, Franklin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1269031779.4361.259.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
On Thu, Mar 18, 2010 at 04:52:07PM -0700, Russ Allbery wrote: Frank Lin PIAT fp...@klabs.be writes: I have no strong preferences between signed APT and SIGNED DEBs... it is just that the remaining of the thread showed that signed DEBs are quite tough to implement. (and I still wonder how we could preserve the current deb format with tar.gz in ar, while signing the debs) You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. Wouldn't it be simpler to just extract *sums from control.tar.gz, create a detached signature for it and put it in the ar archive, instead of extracting data.tar.gz and generating the sums a second time? Or would this replace dh_*sums during package build time? And then create a second signature over all files in the ar archive directly. This one would be checked before extracting the containing tar.gz files, and the other one would be installed along with the *sums file. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100320004007.gc1...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Thu, Mar 18, 2010 at 04:52:07PM -0700, Russ Allbery wrote: You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. Wouldn't it be simpler to just extract *sums from control.tar.gz, create a detached signature for it and put it in the ar archive, instead of extracting data.tar.gz and generating the sums a second time? Or would this replace dh_*sums during package build time? I think it would replace dh_*sums during package build time and make obsolete including md5sums in the control.tar.gz. You don't really want the signature and checksums to be inside one of the other data members since that breaks, as Wouter points out, the ability to remove the signature and checksums and verify against the original *.changes file. And there's no need to include two copies of the checksums. And then create a second signature over all files in the ar archive directly. This one would be checked before extracting the containing tar.gz files, and the other one would be installed along with the *sums file. I think you want to checksum the underlying contents, not the *.tar.gz files in the ar archive. As Joey can attest to from writing pristine-tar, it's surprisingly difficult to reproduce a *.tar.gz file from its members. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87eijfstdj@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Further, the build part of a buildd runs inside a limited chroot running the target distribution, i.e. usually unstable (the real system runs stable with a backported version of sbuild), which doesn't have access to any key material in the real system. At the moment buildds don't have their own keys: a buildd maintainer inspects the build log and signs the .changes file for upload. Even for maintainer uploads, maintainers who build their packages in a minimal chroot with schroot, pbuilder, cowbuilder etc. (which is strongly recommended) don't necessarily have their signing key available inside the chroot (nor should they!). Signatures per buildd or per DD doing uploads are moderately interesting, but not nearly as interesting as a signature by a long-term stable key such as the archive key. Do we actually rely anywhere on packages not changing hashes between upload and publication in the repository, or is it just something we have as an invariant now because there's no reason for it not to be one? The path of least resistance here would be for DAK to add the package signature after verifying the signature of the uploader. This has the drawback that it modifies the *.deb and therefore breaks the hashes in the *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, do we actually care? The changes files are afaik archived somewhere and are needed to verify the archive integrity after a (feared) compromise. And what do you do when the archive key expires? Resign every deb in the archive and have every mirror download them all again? Same problem on releases. Releasing a new stable means a new stable key so every deb needs to be signed again and changes. I don't think this is a good idea. This would also cause users (apt/aptitude actually) to reinstall the packages needlessly creating even more mirror load. For this to work the signature for the checksum files should be detached so that it can be changed/extended without altering the deb files. I suggested this before but lets repeat it. Why not include a digest of the checksum file in DEBIAN/control, carry it into the changes file and into Packages.gz. That way all current debs automatically have a clear trust path. Further the changes files could be included in the pool alongside the debs and source files. That way everyone can verify the autenticity of the debs (or just the checksum file) independent of the archive key. Going one step further the changes files could be signed by the archive key(s) as well. Adding a new signature to them when keys change does mean they need to be mirrored again but they are relatively small. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87iq8uf5lv@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
Wouter Verhelst wou...@debian.org writes: On Wed, Mar 17, 2010 at 04:12:46PM -0700, Russ Allbery wrote: Wouter Verhelst wou...@debian.org writes: This is not true. wou...@merkel:/org/ftp.debian.org/queue/done$ ls *ges|wc -l 28969 These are only the *active* changes files, though: wou...@merkel:/org/ftp.debian.org/queue/done$ find . -name 'nbd*ges'|wc -l 898 ... since no .changes file is ever thrown away: wou...@merkel:/org/ftp.debian.org/queue/done$ du -sh . 7.1G They may not be visible on the mirrors, but they are there. Ah, thank you. I didn't realize that we kept them at all. Note, though, that if the concern is a cryptographically strong audit trail, we could still retain a link from the original *.changes file to the final package with a second (possibly signed) document archived with the *.changes file listing the original and final checksums of the now-signed packages. True. False. The changes files are signed by a human and therefor have a strong trust level. The was XYZ is now UVW file would have to be automatically signed and much less trustworthy. Esspecially if you suspect someone broke into ftp-master and modified some debs. They would just recreate and resign the was XYZ is now UVW file with the automatic archive key. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87eijif5fs@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 17, 2010 at 02:36:31PM +, Simon McVittie wrote: On Wed, 17 Mar 2010 at 12:41:58 +0100, Harald Braumann wrote: It should be signed at build time, just after dh_shasums and then the sig file packaged together with all the other files. I don't see a problem with that. Or maybe I'm not getting something here? Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Further, the build part of a buildd runs inside a limited chroot running the target distribution, i.e. usually unstable (the real system runs stable with a backported version of sbuild), which doesn't have access to any key material in the real system. At the moment buildds don't have their own keys: a buildd maintainer inspects the build log and signs the .changes file for upload. Even for maintainer uploads, maintainers who build their packages in a minimal chroot with schroot, pbuilder, cowbuilder etc. (which is strongly recommended) don't necessarily have their signing key available inside the chroot (nor should they!). Thanks for explaining all these details. I build my packages with sbuild/schroot, and my GPG key isn't available inside the build system as a result of using gfcombinefs to split my key between my laptop and a USB stick (so that if either is stolen, my key isn't compromised). I'm told some developers take this further, and only store their key on a non-networked machine to which they transfer files for signing (the current package upload procedure makes this possible - they only really need to transfer the .changes file, in fact). I think it would be irresponsible to make it necessary for DDs to choose between weakening the security of their GPG keys, or producing less reproducible builds. Theoretically you could produce rather short-lived keys that are signed with your maintainer key, make those available in the build environment and use them for signing. The signing key along with the signature by the maintainer key would have to be included in the package as well. But I guess that this would be a tad to complex and I wouldn't propose it. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100318110955.gb17...@nn.nn
Re: Bug#540215: Introduce dh_checksums
On Thu, Mar 18, 2010 at 08:31:40AM +0100, Goswin von Brederlow wrote: Russ Allbery r...@debian.org writes: Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Further, the build part of a buildd runs inside a limited chroot running the target distribution, i.e. usually unstable (the real system runs stable with a backported version of sbuild), which doesn't have access to any key material in the real system. At the moment buildds don't have their own keys: a buildd maintainer inspects the build log and signs the .changes file for upload. Even for maintainer uploads, maintainers who build their packages in a minimal chroot with schroot, pbuilder, cowbuilder etc. (which is strongly recommended) don't necessarily have their signing key available inside the chroot (nor should they!). Signatures per buildd or per DD doing uploads are moderately interesting, but not nearly as interesting as a signature by a long-term stable key such as the archive key. Do we actually rely anywhere on packages not changing hashes between upload and publication in the repository, or is it just something we have as an invariant now because there's no reason for it not to be one? The path of least resistance here would be for DAK to add the package signature after verifying the signature of the uploader. This has the drawback that it modifies the *.deb and therefore breaks the hashes in the *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, do we actually care? The changes files are afaik archived somewhere and are needed to verify the archive integrity after a (feared) compromise. And what do you do when the archive key expires? Resign every deb in the archive and have every mirror download them all again? Why? A signature doesn't become invalid just because the key expires, as long as the signature was created before the expiration of the key. Same problem on releases. Releasing a new stable means a new stable key so every deb needs to be signed again and changes. I don't think this is a good idea. This would also cause users (apt/aptitude actually) to reinstall the packages needlessly creating even more mirror load. For this to work the signature for the checksum files should be detached so that it can be changed/extended without altering the deb files. I suggested this before but lets repeat it. Why not include a digest of the checksum file in DEBIAN/control, carry it into the changes file and into Packages.gz. That way all current debs automatically have a clear trust path. Further the changes files could be included in the pool alongside the debs and source files. That way everyone can verify the autenticity of the debs (or just the checksum file) independent of the archive key. Going one step further the changes files could be signed by the archive key(s) as well. Adding a new signature to them when keys change does mean they need to be mirrored again but they are relatively small. Self-contained packages, where the signature is included and installed along with the checksum file, would have a lot of advantages. You wouldn't need access to a lot of infrastructure just to verify a signature. It would be very simple. It could be used for packages, that are not part of Debian. For instance, I could produce a package and send it to a friend and he could later use my key for verification. The same holds for projects that publish deb files. With your proposal they would need a full fledged archive and keep a long history of all the files. And what do you do with packages from testing/unstable/experimental? Would you keep all incarnations of the Release/Packages/changes files? And if I want to verify the signature of an installed file, from a package I once installed from, say, unstable, how would I find the right version of the changes file for the package? Cheers, harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100318113959.gc17...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Goswin von Brederlow goswin-...@web.de writes: And what do you do when the archive key expires? Why would you need to do anything at all when the archive key expires? Keys don't become magically compromised or worthless just because they've expired. All it means is that you can't trust the integrity of really old signatures as much as you can trust the integrity of new ones. Same problem on releases. Releasing a new stable means a new stable key so every deb needs to be signed again and changes. I don't see why. The only *.debs that we might need to resign are ones where there have been no uploads for an entire release cycle and hence may be released with expired signatures, and while there are a few of those, that's a much smaller problem. You could even just do an all-arch binNMU to deal with resigning those. For this to work the signature for the checksum files should be detached so that it can be changed/extended without altering the deb files. We could do that as well, but it would require changing the binary package format and the archive software to track an additional file, and I think that's a much larger change. I suggested this before but lets repeat it. Why not include a digest of the checksum file in DEBIAN/control, carry it into the changes file and into Packages.gz. That way all current debs automatically have a clear trust path. Multiple people have explained to you why this doesn't solve the problem: it means that you lose your signature path as soon as the package is no longer listed in Packages. I'm not interested in new ways of authenticating packages that are still current and still listed in Packages. That's a solved problem, and while we can provide some moderate additional convenience, it's not really enough to justify the work. I'm interested in ways of authenticating packages that *aren't* listed in Packages, either because they're unofficial or because they're old and have been superseded. That would be a useful new feature. Further the changes files could be included in the pool alongside the debs and source files. That way everyone can verify the autenticity of the debs (or just the checksum file) independent of the archive key. Going one step further the changes files could be signed by the archive key(s) as well. Adding a new signature to them when keys change does mean they need to be mirrored again but they are relatively small. That's an interesting idea. Yes, we could add additional signatures to the *.changes file without any of this other impact. To solve the full problem for the Debian archive, we'd need to provide all the *.changes files for every *.deb that we've ever released, but thankfully they're small. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87d3z1h37v@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
Goswin von Brederlow goswin-...@web.de writes: The changes files are signed by a human and therefor have a strong trust level. The was XYZ is now UVW file would have to be automatically signed and much less trustworthy. This objection makes no sense to me. The archive key is *much* more trusted in practice than the individual DD keys. Haven't you been advocating using the Packages file for this purpose, which is signed by exactly the same key that would be doing this signature? Esspecially if you suspect someone broke into ftp-master and modified some debs. Which they can do just as easily if you rely only on Packages. Even more easily, in fact. The problems that you are citing here are problems that we already have; that, in fact, are much worse now than they would be under that proposed scheme. (However, I will note that your *.changes idea does offer some additional protection there over the scheme that I was considering.) -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/878w9ph328@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Wed, 2010-03-17 at 11:31 +0100, Wouter Verhelst wrote: On Wed, Mar 17, 2010 at 08:58:28AM +0100, Goswin von Brederlow wrote: Wouter Verhelst wou...@debian.org writes: On Fri, Mar 12, 2010 at 05:16:55AM +0100, Goswin von Brederlow wrote: Harald Braumann ha...@unheit.net writes: On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. That can be avoided by including a hash of the checksum file in the Packages files. That doesn't help for the problem we're trying to fix here: having a path to a GPG signature from an individual binary on the hard disk, months or years after the package was installed. With your proposal, you lose the signatures once the package is out of the archive and you run 'apt-get update'. Then don't do that. :) We can hardly say to our users if you want to be able to check signatures, never run run apt-get update... Not necessarily. I assume that it would be easy (and cheap) to preserve a copy of the previous: http://ftp.debian.org/debian/dists/testing/InRelease http://ftp.debian.org/debian/dists/testing/main/binary-i386/Packages.diff/* It would then be possible to reverse apply the diff, and validate the packages when needed. The disk-space cost would be quite low. Currently, that's 448K for 6 days (27MB/year), which is quite cheap compared to cached binaries that have to be preserved too. (And it comes to my mind that it might be possible to implement some roll-back feature... If we were supporting to downgrade packages to some extend;) I have no strong preferences between signed APT and SIGNED DEBs... it is just that the remaining of the thread showed that signed DEBs are quite tough to implement. (and I still wonder how we could preserve the current deb format with tar.gz in ar, while signing the debs) That's 2 more cents from me, Franklin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1268956013.11872.58.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
Frank Lin PIAT fp...@klabs.be writes: I have no strong preferences between signed APT and SIGNED DEBs... it is just that the remaining of the thread showed that signed DEBs are quite tough to implement. (and I still wonder how we could preserve the current deb format with tar.gz in ar, while signing the debs) You add an additional ar member that contains the signed checksums of all of the files in data.tar.gz, possibly another additional member that contains the signed checksums for control.tar.gz, or you document some convention so that you can combine both into the same signed checksum document. There are other implementation methods possible, but that's probably the most obvious one. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/871vfhchnc@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
Wouter Verhelst wou...@debian.org writes: On Fri, Mar 12, 2010 at 05:16:55AM +0100, Goswin von Brederlow wrote: Harald Braumann ha...@unheit.net writes: On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. That can be avoided by including a hash of the checksum file in the Packages files. That doesn't help for the problem we're trying to fix here: having a path to a GPG signature from an individual binary on the hard disk, months or years after the package was installed. With your proposal, you lose the signatures once the package is out of the archive and you run 'apt-get update'. Then don't do that. :) I don't think signing the checksum file itself will be feasable as that would alter the contents of the deb and change the checksums in the changes files autobuilders send the admin for signing. It would break the existing signing infrastructure for autobuilders. It would also require running dpkg-genchanges again during signing or otherwise adjust the checksums in the changes file. But for packages no longer in the archive there is snapshot.debian.net (or the official replacement). MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87eijj7523@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 17, 2010 at 08:58:28AM +0100, Goswin von Brederlow wrote: Wouter Verhelst wou...@debian.org writes: On Fri, Mar 12, 2010 at 05:16:55AM +0100, Goswin von Brederlow wrote: Harald Braumann ha...@unheit.net writes: On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. That can be avoided by including a hash of the checksum file in the Packages files. That doesn't help for the problem we're trying to fix here: having a path to a GPG signature from an individual binary on the hard disk, months or years after the package was installed. With your proposal, you lose the signatures once the package is out of the archive and you run 'apt-get update'. Then don't do that. :) We can hardly say to our users if you want to be able to check signatures, never run run apt-get update... I don't think signing the checksum file itself will be feasable as that would alter the contents of the deb and change the checksums in the changes files autobuilders send the admin for signing. Yes, it would be a problem for autobuilders. However, I don't think it's completely unfeasible. It would break the existing signing infrastructure for autobuilders. It would also require running dpkg-genchanges again during signing or otherwise adjust the checksums in the changes file. But for packages no longer in the archive there is snapshot.debian.net (or the official replacement). Which are both not very useful at the moment. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 17, 2010 at 5:31 PM, Wouter Verhelst w...@uter.be wrote: But for packages no longer in the archive there is snapshot.debian.net (or the official replacement). Which are both not very useful at the moment. I've found http://snapshot-dev.debian.org quite useful recently. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e13a36b31003170408q28ad567x9222db5c6eb4d...@mail.gmail.com
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 17, 2010 at 08:58:28AM +0100, Goswin von Brederlow wrote: I don't think signing the checksum file itself will be feasable as that would alter the contents of the deb and change the checksums in the changes files autobuilders send the admin for signing. It would break the existing signing infrastructure for autobuilders. It would also require running dpkg-genchanges again during signing or otherwise adjust the checksums in the changes file. It should be signed at build time, just after dh_shasums and then the sig file packaged together with all the other files. I don't see a problem with that. Or maybe I'm not getting something here? Cheers, harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100317114158.ga17...@nn.nn
Re: Bug#540215: Introduce dh_checksums
On Wed, 17 Mar 2010 at 12:41:58 +0100, Harald Braumann wrote: It should be signed at build time, just after dh_shasums and then the sig file packaged together with all the other files. I don't see a problem with that. Or maybe I'm not getting something here? Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Further, the build part of a buildd runs inside a limited chroot running the target distribution, i.e. usually unstable (the real system runs stable with a backported version of sbuild), which doesn't have access to any key material in the real system. At the moment buildds don't have their own keys: a buildd maintainer inspects the build log and signs the .changes file for upload. Even for maintainer uploads, maintainers who build their packages in a minimal chroot with schroot, pbuilder, cowbuilder etc. (which is strongly recommended) don't necessarily have their signing key available inside the chroot (nor should they!). I build my packages with sbuild/schroot, and my GPG key isn't available inside the build system as a result of using gfcombinefs to split my key between my laptop and a USB stick (so that if either is stolen, my key isn't compromised). I'm told some developers take this further, and only store their key on a non-networked machine to which they transfer files for signing (the current package upload procedure makes this possible - they only really need to transfer the .changes file, in fact). I think it would be irresponsible to make it necessary for DDs to choose between weakening the security of their GPG keys, or producing less reproducible builds. Another issue with signing automatically at build-time is that it gives preliminary versions of a package the same level of authentication (signature) as the uploaded version. It sometimes takes a few iterations to make a final build of a package, so the workflow I use is to build an unsigned package and test it. If it works well enough to be suitable for upload, I sign and upload it; if it doesn't, I discard it, amend the source and repeat. Simon -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100317143631.ga5...@reptile.pseudorandom.co.uk
Re: Bug#540215: Introduce dh_checksums
Simon McVittie s...@debian.org writes: Most packages (in terms of proportion of the archive, in particular for architectures other than i386 and amd64) are built by a buildd, so each buildd would have to have a signing key that could sign the checksums file during build. Further, the build part of a buildd runs inside a limited chroot running the target distribution, i.e. usually unstable (the real system runs stable with a backported version of sbuild), which doesn't have access to any key material in the real system. At the moment buildds don't have their own keys: a buildd maintainer inspects the build log and signs the .changes file for upload. Even for maintainer uploads, maintainers who build their packages in a minimal chroot with schroot, pbuilder, cowbuilder etc. (which is strongly recommended) don't necessarily have their signing key available inside the chroot (nor should they!). Signatures per buildd or per DD doing uploads are moderately interesting, but not nearly as interesting as a signature by a long-term stable key such as the archive key. Do we actually rely anywhere on packages not changing hashes between upload and publication in the repository, or is it just something we have as an invariant now because there's no reason for it not to be one? The path of least resistance here would be for DAK to add the package signature after verifying the signature of the uploader. This has the drawback that it modifies the *.deb and therefore breaks the hashes in the *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, do we actually care? -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87pr32zurw@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 17, 2010 at 11:07:47AM -0700, Russ Allbery wrote: *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, This is not true. wou...@merkel:/org/ftp.debian.org/queue/done$ ls *ges|wc -l 28969 These are only the *active* changes files, though: wou...@merkel:/org/ftp.debian.org/queue/done$ find . -name 'nbd*ges'|wc -l 898 ... since no .changes file is ever thrown away: wou...@merkel:/org/ftp.debian.org/queue/done$ du -sh . 7.1G They may not be visible on the mirrors, but they are there. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100317224016.gc15...@celtic.nixsys.be
Re: Bug#540215: Introduce dh_checksums
On 2010-03-17, Wouter Verhelst wou...@debian.org wrote: On Wed, Mar 17, 2010 at 11:07:47AM -0700, Russ Allbery wrote: *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, This is not true. They may not be visible on the mirrors, but they are there. But, as far as I know, there is no connection from a file in the archive to the corresponding .changes. It's true that they are somehow archived, but not tracked. Kind regards, Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrnhq2mtb.3t7.tr...@kelgar.0x539.de
Re: Bug#540215: Introduce dh_checksums
Wouter Verhelst wou...@debian.org writes: This is not true. wou...@merkel:/org/ftp.debian.org/queue/done$ ls *ges|wc -l 28969 These are only the *active* changes files, though: wou...@merkel:/org/ftp.debian.org/queue/done$ find . -name 'nbd*ges'|wc -l 898 ... since no .changes file is ever thrown away: wou...@merkel:/org/ftp.debian.org/queue/done$ du -sh . 7.1G They may not be visible on the mirrors, but they are there. Ah, thank you. I didn't realize that we kept them at all. Note, though, that if the concern is a cryptographically strong audit trail, we could still retain a link from the original *.changes file to the final package with a second (possibly signed) document archived with the *.changes file listing the original and final checksums of the now-signed packages. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87ljdqtudt@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 17, 2010 at 04:12:46PM -0700, Russ Allbery wrote: Wouter Verhelst wou...@debian.org writes: This is not true. wou...@merkel:/org/ftp.debian.org/queue/done$ ls *ges|wc -l 28969 These are only the *active* changes files, though: wou...@merkel:/org/ftp.debian.org/queue/done$ find . -name 'nbd*ges'|wc -l 898 ... since no .changes file is ever thrown away: wou...@merkel:/org/ftp.debian.org/queue/done$ du -sh . 7.1G They may not be visible on the mirrors, but they are there. Ah, thank you. I didn't realize that we kept them at all. Note, though, that if the concern is a cryptographically strong audit trail, we could still retain a link from the original *.changes file to the final package with a second (possibly signed) document archived with the *.changes file listing the original and final checksums of the now-signed packages. True. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100317231806.ge15...@celtic.nixsys.be
Re: Bug#540215: Introduce dh_checksums
On Wed, 2010-03-17 at 23:40 +0100, Wouter Verhelst wrote: On Wed, Mar 17, 2010 at 11:07:47AM -0700, Russ Allbery wrote: *.changes file and hence its original signature, but given that we throw out the *.changes file anyway, This is not true. wou...@merkel:/org/ftp.debian.org/queue/done$ ls *ges|wc -l 28969 These are only the *active* changes files, though: wou...@merkel:/org/ftp.debian.org/queue/done$ find . -name 'nbd*ges'|wc -l 898 ... since no .changes file is ever thrown away: wou...@merkel:/org/ftp.debian.org/queue/done$ du -sh . 7.1G They may not be visible on the mirrors, but they are there. Not that you'll be able to verify most of them, since the keyring only contains keys that are accepted for new uploads. Ben. -- Ben Hutchings One of the nice things about standards is that there are so many of them. signature.asc Description: This is a digitally signed message part
Re: Bug#540215: Introduce dh_checksums
On Fri, Mar 12, 2010 at 05:16:55AM +0100, Goswin von Brederlow wrote: Harald Braumann ha...@unheit.net writes: On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. That can be avoided by including a hash of the checksum file in the Packages files. That doesn't help for the problem we're trying to fix here: having a path to a GPG signature from an individual binary on the hard disk, months or years after the package was installed. With your proposal, you lose the signatures once the package is out of the archive and you run 'apt-get update'. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100312221356.gj3...@celtic.nixsys.be
Re: Bug#540215: Introduce dh_checksums, clear-signed checksum
On Thu, 2010-03-11 at 00:37 +, The Fungi wrote:
On Wed, Mar 10, 2010 at 11:22:00PM +0100, Frank Lin PIAT wrote:
I made some tests, and it seems that we could allow,but not require, GPG
signed checksum-file. sha256sum will ignore invalid lines by default
(unless you specify --warn option).
Similarly, the policy could state that GPG clear-signed shasum files are
allowed. Tools using shasum should still strip the signature, especially
when using the checksum for security purpose.
Is there any good reason not to use a detached signature in a
separate file instead? I know that doubles the number of files, but
it also reduces the raw size by around 47 bytes and simplifies
parsing of the checksum files themselves.
My real first question was to know if that can be useful. Plus, not
every one uses gpg-agent, and they may not like to sign each package
twice.
Regarding clearsign-versus-detached, I have no strong preference myself.
clearsigned are nice because they are self-contained, but... see your
rational.
That being said...
Stripping signature:
Stripping the gpg signature is not needed for sha256sum command line,
and it is trivial, for bash/perl...
sed -n -e '/^-\(BEGIN PGP SIGNED
MESSAGE\)-/,/^-[^\1]/s/^[[:xdigit:]]\{32,\}\s/\0/p' testfile.asc
On disk usage:
¨¨
echo testfile
gpg -b testfile
gpg --clearsign testfile
ls -l testfile*
-rw-r--r--. 1 fpiat fpiat 1 2010-03-11 09:55 testfile
-rw-r--r--. 1 fpiat fpiat 886 2010-03-11 09:55 testfile.asc
-rw-r--r--. 1 fpiat fpiat 543 2010-03-11 09:55 testfile.sig
but...
du testfile*
4 testfile
4 testfile.asc
4 testfile.sig
The actual on disk usage is increased, up to one disk block
Tarfile usage
¨
tar -zcvf detached.tar.gz testfile testfile.sig
testfile
testfile.sig
tar -zcvf clearsign.tar.gz testfile.asc
testfile.asc
ls -l *.gz
-rw-r--r--. 1 fpiat fpiat 815 2010-03-11 10:00 clearsign.tar.gz
-rw-r--r--. 1 fpiat fpiat 759 2010-03-11 10:00 detached.tar.gz
The archive file is increased by 47, which is marginal, compared to the
increase in size of sha256 md5 hash size :-(
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/1268298752.3959.161.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 10, 2010 at 11:13:31AM -0600, Peter Samuelson wrote: [Wouter Verhelst] At any rate, a PGP signature takes a lot of data; much more so than a checksum. It's therefore more economical to produce a signed package.checksums file than it is to produce a package.pgpsigs. Huh? Since asymmetric cryptography is so computationally expensive, PGP never signs the payload directly. I am aware of that. Instead, the payload is hashed and then the hash is signed. So it is not (noticeably) more economical to sign foo.md5sums than to sign the whole data.tar.gz. I was not suggesting to sign the data.tar.gz, because that is not useful anymore once the package is installed (you do not have the data.tar.gz anymore to verify). Instead, I was suggesting to sign individual files, which would require several signatures per package (one per file). Just check the length of any PGP signature, and compare it against the lenght of a random checksum. You'll agree that a signed file with checksums takes less data than a file with several signatures. [...] Or is this not what you meant? I'm confused. Hope that explains, [...] If you have a .deb on a different host and don't want to transfer the entire thing over the network, well, no reason you can't do your SHA16384 on both ends, and transfer only the hashes at that time. True. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. I was wondering about that. Unfortunately I'm quite ignorant of the details of the whole upload and build process. - Are all packages that end up in the archive built by the autobuilders, or can maintainers upload binary packages directly? - How are the Release files signed? Is it done automatically or manually? By whom? Cheers, harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100311173710.gc25...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. That can be avoided by including a hash of the checksum file in the Packages files. That would be a relatively minor change in apt-ftparchive. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87aaue898o@frosties.localdomain
Re: Bug#540215: Introduce dh_checksums
Goswin von Brederlow goswin-...@web.de writes: On Wed, Mar 10, 2010 at 03:32:14PM +0100, Wouter Verhelst wrote: Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. That can be avoided by including a hash of the checksum file in the Packages files. That would be a relatively minor change in apt-ftparchive. Having the signature only in the Packages file only solves a part of the problem. It's going to be very common to want to verify the integrity of a package that's no longer current and hence no longer listed in the Packages file. I'd really rather see us pursue solutions that solve the entire problem instead, including verification of the signature on isolated *.deb files. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87wrxi6tw4@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Mon, Mar 08, 2010 at 12:59:00PM -0800, Russ Allbery wrote: Frank Lin PIAT fp...@klabs.be writes: Find a patch attached, for a smooth transition from DEBIAN/md5sums to a recent checksum. The way it is implemented, is that the dh_md5sums is a symlink to the new dh_checksums. The new helper computes both md5sum (for compatibility/transition) and a new checksum (SHA256, which was already chosen by FTP-masters as a remplacement for md5sum for signed files) If there's a general consensus that we should go this route, I'm okay with it, but I'm personally not sure this is the right approach. I think there are two possible directions we can take with this package feature: 1. Strengthen the integrity check so that it could potentially be useful for security purposes as well as for simple integrity checking. 2. Declare such checksums only useful for corruption and local (benign) modification checksumming. If we take option 1, going to a stronger hash is strictly less useful in every respect than going to embedded PGP signatures, which apparently only require active maintenance and a plan to be acceptable in the archive and which would need a different format anyway. Maybe, but we're not there yet. At any rate, a PGP signature takes a lot of data; much more so than a checksum. It's therefore more economical to produce a signed package.checksums file than it is to produce a package.pgpsigs. Having package.checksums be GPG-signed will take a significant change in our infrastructure (buildd hosts, for instance, would need to have a way to sign checksums files as well), so it's not going to happen tomorrow. But changing to a more secure checksum algorithm could be done easily with current infrastructure. And since checksum files are generated at build time rather than at install time, it is possible to download a known-good copy of the .deb that is installed (using snapshot.debian.org, once it gets live, for instance, or from a not-compromised host's apt cache), and verify the files against that copy rather than the copy that is on the disk. Until signed packages or signed checksums are available, this would of course be a stopgap measure, but one that would actually work -- provided, of course, that the used checksum algorithm is cryptographically secure, which MD5 is no longer. If we take option 2, SHA256 offers no benefits over MD5 and just takes longer to compute. There is essentially zero chance that random file corruption or typical local modification will result in a successful MD5 preimage attack. Of course, that much is true. -- The biometric identification system at the gates of the CIA headquarters works because there's a guard with a large gun making sure no one is trying to fool the system. http://www.schneier.com/blog/archives/2009/01/biometrics.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100310143214.ge10...@celtic.nixsys.be
Re: Bug#540215: Introduce dh_checksums
[Wouter Verhelst] At any rate, a PGP signature takes a lot of data; much more so than a checksum. It's therefore more economical to produce a signed package.checksums file than it is to produce a package.pgpsigs. Huh? Since asymmetric cryptography is so computationally expensive, PGP never signs the payload directly. Instead, the payload is hashed and then the hash is signed. So it is not (noticeably) more economical to sign foo.md5sums than to sign the whole data.tar.gz. [Same goes for encrypting: PGP encrypts with a conventional block cipher like AES and a randomly generated key, then encrypts the _key_ using RSA. Again, this is for efficiency.] Or is this not what you meant? I'm confused. And since checksum files are generated at build time rather than at install time, it is possible to download a known-good copy of the .deb that is installed (using snapshot.debian.org, once it gets live, for instance, or from a not-compromised host's apt cache), and verify the files against that copy rather than the copy that is on the disk. If you're going to the trouble to download a .deb, why bother with signatures at all? Why not just compare the full text directly? If you have a .deb on a different host and don't want to transfer the entire thing over the network, well, no reason you can't do your SHA16384 on both ends, and transfer only the hashes at that time. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100310171331.gt18...@p12n.org
Re: Bug#540215: Introduce dh_checksums
Peter Samuelson pe...@p12n.org writes: [Wouter Verhelst] At any rate, a PGP signature takes a lot of data; much more so than a checksum. It's therefore more economical to produce a signed package.checksums file than it is to produce a package.pgpsigs. Huh? Since asymmetric cryptography is so computationally expensive, PGP never signs the payload directly. Instead, the payload is hashed and then the hash is signed. So it is not (noticeably) more economical to sign foo.md5sums than to sign the whole data.tar.gz. However, since the most common verification action is probably going to be to check whether a specific file installed on the system has been modified, Wouter's approach is probably the best implementation strategy. It's more efficient to compute the checksum of a file, check that it matches the checksum in the signed file, and verify the signature on that file than it is to verify the data.tar.gz signature and then extract the relevant file from it and compare it to the file on disk. Among other things, you can use the first algorithm with nothing but the signed checksum files, which are a lot smaller than keeping the whole package around. If you're going to the trouble to download a .deb, why bother with signatures at all? Why not just compare the full text directly? Indeed. It's an efficiency gain for much the same reasons as above, but not really a security gain. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87y6i0ggal@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
Hello,
On Mon, 2010-03-08 at 17:36 +0100, Frank Lin PIAT wrote:
On Thu, 2010-03-04 at 20:08 +0100, Tollef Fog Heen wrote:
Frank Lin PIAT wrote:
What about a transitional dh_md5sums that would produce md5sum AND
invoke dh_sha ?
Or call it dh_checksums or something so we don't have to change the tool
name each time we decide to change the algorithm.
Find a patch attached, for a smooth transition from DEBIAN/md5sums to a
recent checksum.
Since SHA algorithms is a family, tools and API usually implement
multiple variants. Wouter's initial email suggested to use the name
shasums. I must admit I find this quite sensible for future
improvements. People should be encourage to detect and support SHA-224
and better hash, even though we should only accept sha256 in the archive
for now.
I have still named the helper dh_checksums, because it we ever want to
ship a different algorithm, we would probably still use the same
(updated) helper to generate that file.
Find an updated patch attached.
Regards,
diff --git a/debian/copyright b/debian/copyright
index a9f950d..162bfc0 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -48,7 +48,7 @@ Copyright: Steve Robbins s...@debian.org
License: GPL-2+
Files: dh_md5sums
-Copyright: Charles Briscoe-Smith c...@ukc.ac.uk
+Copyright: Charles Briscoe-Smith c...@ukc.ac.uk, Frank Lin PIAT
fp...@klabs.be
License: GPL-2+
Files: dh_bugfiles
diff --git a/debian/links b/debian/links
new file mode 100644
index 000..3e7d603
--- /dev/null
+++ b/debian/links
@@ -0,0 +1,3 @@
+usr/share/man/man1/dh_md5sums.1.gz usr/share/man/man1/dh_checksums.1.gz
+usr/bin/dh_md5sums usr/bin/dh_checksums
+
diff --git a/dh b/dh
index bcac8da..0aa9bc3 100755
--- a/dh
+++ b/dh
@@ -322,7 +322,7 @@ $sequences{install} = [...@{$sequences{build}}, qw{
my @b=qw{
dh_installdeb
dh_gencontrol
- dh_md5sums
+ dh_checksums
dh_builddeb
};
$sequences{'binary-indep'} = [...@{$sequences{install}}, @b];
diff --git a/dh_md5sums b/dh_md5sums
index da00090..33bf561 100755
--- a/dh_md5sums
+++ b/dh_md5sums
@@ -2,7 +2,7 @@
=head1 NAME
-dh_md5sums - generate DEBIAN/md5sums file
+dh_checksums - generate DEBIAN/*sums files (md5, sha256)
=cut
@@ -12,18 +12,24 @@ use Debian::Debhelper::Dh_Lib;
=head1 SYNOPSIS
+Bdh_checksums [SIdebhelper options] [B-x] [B-XIitem]
[B--include-conffiles]
+
Bdh_md5sums [SIdebhelper options] [B-x] [B-XIitem]
[B--include-conffiles]
=head1 DESCRIPTION
-dh_md5sums is a debhelper program that is responsible for generating
-a DEBIAN/md5sums file, which lists the md5sums of each file in the package.
-These files are used by the debsums package.
+dh_checksums is a debhelper program that is responsible for generating
+a DEBIAN/md5sums and DEBIAN/sha256sums files, which respectively lists the
+md5sums and sha256sums of each file in the package. These files are used
+by the debsums package.
-All files in DEBIAN/ are omitted from the md5sums file, as are all
+All files in DEBIAN/ are omitted from the checksums files, as are all
conffiles (unless you use the --include-conffiles switch).
-The md5sums file is installed with proper permissions and ownerships.
+The checksums files are installed with proper permissions and ownerships.
+
+dh_md5sums is deprecated, you should use dh_checksums instead, which generates
the
+type of checksums files recommended by the Debian policy.
=head1 OPTIONS
@@ -37,7 +43,7 @@ redundant since it is included elsewhere in debian packages.
=item B-XIitem, B--exclude=Iitem
Exclude files that contain item anywhere in their filename from
-being listed in the md5sums file.
+being listed in the checkums file.
=back
@@ -48,15 +54,26 @@ init(options = {
include-conffiles = \$dh{INCLUDE_CONFFILES},
});
+my ($basename) = $0=~m:.*/(.+):;
+
foreach my $package (@{$dh{DOPACKAGES}}) {
next if is_udeb($package);
-
+
+ if (basename($0) == 'dh_md5sums') {
+ warning(This program should no longer be used. Please read the
dh_checksums(1) man page.);
+ }
+
my $tmp=tmpdir($package);
if (! -d $tmp/DEBIAN) {
doit(install,-d,$tmp/DEBIAN);
}
+ # Detect if this is run multiple times (calling both dh_md5sums and
dh_checksums?)
+ if (-f $tmp/DEBIAN/md5sums or -f $tmp/DEBIAN/sha256sums) {
+ warning(Re-computing checksum file (even though md5sums and/or
sha256sums exists));
+ }
+
# Check if we should exclude conffiles.
my $exclude=;
if (! $dh{INCLUDE_CONFFILES} -r $tmp/DEBIAN/conffiles) {
@@ -76,6 +93,7 @@ foreach my $package (@{$dh{DOPACKAGES}}) {
}
complex_doit((cd $tmp /dev/null ; find . -type f $exclude ! -regex
'.*/DEBIAN/.*' -printf '%P\\0' | xargs -r0 md5sum DEBIAN/md5sums)
/dev/null);
+ complex_doit((cd $tmp /dev/null ; find . -type f $exclude ! -regex
'.*/DEBIAN/.*' -printf '%P\\0' | xargs -r0 sha256sum
Re: Bug#540215: Introduce dh_checksums, clear-signed checksum
On Wed, 2010-03-10 at 10:52 -0800, Russ Allbery wrote: Peter Samuelson pe...@p12n.org writes: [Wouter Verhelst] At any rate, a PGP signature takes a lot of data; much more so than a checksum. It's therefore more economical to produce a signed package.checksums file than it is to produce a package.pgpsigs. Huh? Since asymmetric cryptography is so computationally expensive, PGP never signs the payload directly. Instead, the payload is hashed and then the hash is signed. So it is not (noticeably) more economical to sign foo.md5sums than to sign the whole data.tar.gz. However, since the most common verification action is probably going to be to check whether a specific file installed on the system has been modified, Wouter's approach is probably the best implementation strategy. It's more efficient to compute the checksum of a file, check that it matches the checksum in the signed file, and verify the signature on that file than it is to verify the data.tar.gz signature and then extract the relevant file from it and compare it to the file on disk. Among other things, you can use the first algorithm with nothing but the signed checksum files, which are a lot smaller than keeping the whole package around. GPG clear-signed messages ¨ I made some tests, and it seems that we could allow,but not require, GPG signed checksum-file. sha256sum will ignore invalid lines by default (unless you specify --warn option). Similarly, the policy could state that GPG clear-signed shasum files are allowed. Tools using shasum should still strip the signature, especially when using the checksum for security purpose. Let me know you find this feature useful (or over engineered). Regards, Franklin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1268259720.3291.3261.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums, clear-signed checksum
On Wed, Mar 10, 2010 at 11:22:00PM +0100, Frank Lin PIAT wrote:
I made some tests, and it seems that we could allow,but not require, GPG
signed checksum-file. sha256sum will ignore invalid lines by default
(unless you specify --warn option).
Similarly, the policy could state that GPG clear-signed shasum files are
allowed. Tools using shasum should still strip the signature, especially
when using the checksum for security purpose.
Is there any good reason not to use a detached signature in a
separate file instead? I know that doubles the number of files, but
it also reduces the raw size by around 47 bytes and simplifies
parsing of the checksum files themselves.
--
{ IRL(Jeremy_Stanley); PGP(9E8DFF2E4F5995F8FEADDC5829ABF7441FB84657);
SMTP(fu...@yuggoth.org); IRC(fu...@irc.yuggoth.org#ccl); ICQ(114362511);
AIM(dreadazathoth); YAHOO(crawlingchaoslabs); FINGER(fu...@yuggoth.org);
MUD(fu...@katarsis.mudpy.org:6669); WWW(http://fungi.yuggoth.org/); }
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100311003709.gi1...@yuggoth.org
Re: Bug#540215: Introduce dh_checksums
[despite having not yet replied to this thread, I am watching it...I just don't have the desire to add to yet another giant, silly thread on -devel. anyways...] On Mon, Mar 08, 2010 at 12:21:42PM -0500, Joey Hess wrote: Your comments on the patch are obviously welcome (feel free to hack it your self if you want) Any chance to merge it before squeeze Freeze? Is debsums ready to handle other checksums types? no. I will happily add support for it if there is consensus that a switch to sha256sums (or any other checksum algorithm, for that matter) should happen, and once packages begin to migrate to it. Cheers, Ryan -- _ Ryan Niebur ryanrya...@gmail.com signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Mon, 2010-03-08 at 19:57 -0800, Russ Allbery wrote: Joey Hess jo...@debian.org writes: Russ Allbery wrote: It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. True. Adding new binaries is, in my experience, more common than modifying binaries already on the system. I don't really mean to be arguing for debsums as a security mechanism, more just commenting on the general question. I'm on the side that thinks that debsums isn't a horribly useful direction to go for full-blown intrusion detection, and that for what it's really useful for right now MD5 remains entirely adequate. How do people know which binaries are still pristine, so they can keep looking for issues elsewhere? (like added binaries and modified and insecure configuration file...) Not everyone uses aide (popcon: 0.49%). What solution do we have for Joe user (whom did not install aide)? What's the agenda for squeeze and squeeze+1? Who is actually stepping up to do the Job? Please, let's do the easy move *now* for Squeeze, using shasums, and go ahead later with an even better solution. This transition can be quick, it will remain quite unnoticed by people that aren't interested, but it will be appreciated by people who want to use it. Regards, -- Franklin Piat | The better is the enemy of the good. (Voltaire) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1268123295.21347.12099.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
On 09/03/2010 14:24, Bernhard R. Link wrote: It it's that straight forward, please help with the cruft package. Last time I looked (several years ago) it was severly limited by that problem (there not being a way to know which files should be there and which not). I personally think without something in this direction, intrusion detection based on file lists is not really possible. I for one pledge the addition of a dpkg-register (or dpkg --register or anything), bound with dpkg, that would allow maintainers to specify that a file belongs to their package (it could be managed through postinsts, via ucf...) The number of files under /etc that belong to no package (in the sense of dpkg -S) makes it very hard to keep a clean system. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4b964e0b.3060...@free.fr
Re: Bug#540215: Introduce dh_checksums
On Mon, Mar 08, 2010 at 10:49:54PM -0500, Joey Hess wrote: Russ Allbery wrote: It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. And it's as straightforward to find files which don't belong to any package and have some other means in place to check locally generated files. If I understand you correctly, you argue that one would need some IDS anyway to cover all files, and that could then be used also to verify package files. Therefore making file signatures in packages superfluous. I think I could agree with that. On the other hand, I tend to keep /usr/local clean and create packages for for home-grown software. If you do this consistently, you'd get a system where you could verify all files without additional software (modulo the script that checks for surplus files). More important would be package signatures, anyway, because currently there is no way to verify a package. I work with testing/unstable a lot and often I have deb files lying around are not in any Release, so there is no way of verifying them. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309125920.gc15...@nn.nn
Re: Bug#540215: Introduce dh_checksums
* Harald Braumann ha...@unheit.net [100309 13:59]: On Mon, Mar 08, 2010 at 10:49:54PM -0500, Joey Hess wrote: Russ Allbery wrote: It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. And it's as straightforward to find files which don't belong to any package and have some other means in place to check locally generated files. It it's that straight forward, please help with the cruft package. Last time I looked (several years ago) it was severly limited by that problem (there not being a way to know which files should be there and which not). I personally think without something in this direction, intrusion detection based on file lists is not really possible. Hochachtungsvoll, Bernhard R. Link -- Never contain programs so few bugs, as when no debugging tools are available! Niklaus Wirth -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309132432.ga30...@pcpool00.mathematik.uni-freiburg.de
Re: Bug#540215: Introduce dh_checksums
* Peter Samuelson pe...@p12n.org, 2010-03-09, 08:21: [Frank Lin PIAT] Why is that everyone seems to move away from MD5? That's the $64000 question, isn't it? There seems to be this knee-jerk reaction to all things md5, OH NOES, MD5 is broken! Therefore it must be replaced in every possible use, never mind whether any particular use has anything to do with malicious attackers. Strange that rsync seems to have escaped this madness, nobody has been frantically calling for it to migrate to something more up to date than MD4. Which, IIRC, is just as broken. I guess the masses must have realized, in a way they usually do not, that sometimes an integrity check is just an integrity check. FYI: $ zgrep MD5 /usr/share/doc/rsync/changelog.gz - Protocol 30 now uses MD5 checksums instead of MD4. That said, I totally agree with you. -- Jakub Wilk signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Mon, 2010-03-08 at 12:59 -0800, Russ Allbery wrote: If we take option 2, SHA256 offers no benefits over MD5 and just takes longer to compute. [Frank Lin PIAT] Why is that everyone seems to move away from MD5? That's the $64000 question, isn't it? There seems to be this knee-jerk reaction to all things md5, OH NOES, MD5 is broken! Therefore it must be replaced in every possible use, never mind whether any particular use has anything to do with malicious attackers. Strange that rsync seems to have escaped this madness, nobody has been frantically calling for it to migrate to something more up to date than MD4. Which, IIRC, is just as broken. I guess the masses must have realized, in a way they usually do not, that sometimes an integrity check is just an integrity check. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309142149.gp18...@p12n.org
Re: Bug#540215: Introduce dh_checksums
[Frank Lin PIAT] Please, let's do the easy move *now* for Squeeze, using shasums, and go ahead later with an even better solution. Drawbacks: more CPU time on build daemons, slightly larger binary packages to download, and some disruption when we're trying to get a release out the door. Advantages: ... umm ... warm fuzzy feeling that we aren't relying on that old stupid broken MD5 thing that is so out of fashion these days among the cognoscenti? If you really want to use /var/lib/dpkg/info/pkg.*sums files for any purpose other than detecting non-malicious corruption, obviously you need _either_ some form of package signatures, _or_ a server akin to http://packages.debian.org/changelogs/ for serving checksums from a more trusted source. And of course if you have that sort of server support anyway - why not just calculate those sha16384 sums on the server, with no change to the debs at all? Peter -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309165059.gr18...@p12n.org
Re: Bug#540215: Introduce dh_checksums
Harald Braumann wrote: On Mon, Mar 08, 2010 at 10:49:54PM -0500, Joey Hess wrote: It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. And it's as straightforward to find files which don't belong to any package and have some other means in place to check locally generated files. I don't want to get dragged into continuing to provide counterexamples, but it's also fairly easy to modify a file in /etc to provide a backdoor, such that neither debsums nor cruft will notice it. -- see shy jo signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Tue, Mar 09 2010, Jean-Christophe Dubacq wrote: On 09/03/2010 14:24, Bernhard R. Link wrote: It it's that straight forward, please help with the cruft package. Last time I looked (several years ago) it was severly limited by that problem (there not being a way to know which files should be there and which not). I personally think without something in this direction, intrusion detection based on file lists is not really possible. I for one pledge the addition of a dpkg-register (or dpkg --register or anything), bound with dpkg, that would allow maintainers to specify that a file belongs to their package (it could be managed through postinsts, via ucf...) The number of files under /etc that belong to no package (in the sense of dpkg -S) makes it very hard to keep a clean system. For what it is worth, ucf can also tell dpkg about the files and hashes that it manages (it already stores the data, all that has to be added is calls to the hypothetical dpkg-register). manoj -- Oliver's Law: Experience is something you don't get until just after you need it. Manoj Srivastava sriva...@acm.org http://www.golden-gryphon.com/ 4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20 05B6 CF48 9438 C577 9A1C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87pr3dbib2@anzu.internal.golden-gryphon.com
Re: Bug#540215: Introduce dh_checksums
On Tue, Mar 09, 2010 at 10:50:59AM -0600, Peter Samuelson wrote: [Frank Lin PIAT] Please, let's do the easy move *now* for Squeeze, using shasums, and go ahead later with an even better solution. Drawbacks: more CPU time on build daemons, slightly larger binary packages to download, and some disruption when we're trying to get a release out the door. Advantages: ... umm ... warm fuzzy feeling that we aren't relying on that old stupid broken MD5 thing that is so out of fashion these days among the cognoscenti? If you really want to use /var/lib/dpkg/info/pkg.*sums files for any purpose other than detecting non-malicious corruption, obviously you need _either_ some form of package signatures, _or_ a server akin to http://packages.debian.org/changelogs/ for serving checksums from a more trusted source. And of course if you have that sort of server support anyway - why not just calculate those sha16384 sums on the server, with no change to the debs at all? See, you don't need a server. You just ship a signature over the hash files. Easy as that. Of course the hashes would neet to be something more secure than md5, for that warm fuzzy feeling that in two years time not every script kiddy can mount hash attacks on their home computer. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309183541.ga25...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Hey Russ, On Tue, Mar 9, 2010 at 13:57, Russ Allbery r...@debian.org wrote: Joey Hess jo...@debian.org writes: Russ Allbery wrote: It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. True. Adding new binaries is, in my experience, more common than modifying binaries already on the system. I don't really mean to be arguing for debsums as a security mechanism, more just commenting on the general question. So what's the goal here? Basic integrity checking by defending against random corruption due to occassional hardware, software or admin errors? Or an additional security layer verifying that the installed system software is still exactly what it was intended to be? (Or, I guess, a third option would be that it's just some security theatre to make people feel more comfortable about their system's integrity, without actually adding any technical guarantees.) For basic integrity checking, I'm finding the dpkg patch I posted the other day works fairly nicely -- I've reinstalled the handful of packages that don't provide md5sums files so dpkg would generate the hashes file for me, and now I've got hashes for every package on my system, without having to worry about policy getting changed or packages getting reuploaded, or having to worry about bugs like any random packages I might use not following the new policy. And now that I actually look at debsums, rather than just checking with md5sum -c directly, I see debsums already gets invoked by apt by default to ensure that there's an md5sums file for every package. And that's been there for a fair while, based on Bug#132767. If the idea is just to catch a wide swathe of accidental errors, doesn't it make sense to stick with md5 as being cheap and unlikely to have accidental collisions, do the md5sum generation in either apt or dpkg to guarantee it's done for all installed packages, and leave it at that? That means we could get rid of a bunch of policy and code from package build scripts, which seems like it could only be a good thing; and it also seems pretty easy to do for squeeze (since it's already done as far as apt is concerned, and there's a patch for dpkg if desired, and no other packages need to be changed). The only drawback seems to be that you end up verifying what was actually installed, and that might differ from verifying what was meant to be installed in the event that the deb you're installing gets corrupted while you're reading it, despite having previously passing its own secure hash check. OTOH, if the idea is to do more than just basic integrity checking of dpkg-installed files, to aid in detection of and recovery from malicious/deliberate changes, it seems like there's a lot of things that ought to be done differently to how debsums/pkg.md5sums work (secure hash, checking conffiles, stuff in /var, checking added files, checking additional diversions, preventing addition/deletion/modification of the md5sums files...). (Well, better handling of the md5sums files themselves could be useful for basic integrity checking too -- a disk/fs error that trashes files in /var/lib/dpkg/info/ can be inconvenient too) Cheers, aj -- Anthony Towns a...@erisian.com.au -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87b3a4191003091145p26467cag8d2d4cfb9dc20...@mail.gmail.com
Re: Bug#540215: Introduce dh_checksums
[Harald Braumann] See, you don't need a server. You just ship a signature over the hash files. Easy as that. And that signature - if you don't have a server - you probably want to store it in the .deb, right? So you are going to be editing the .deb after it is built. At which time, you could just as well compute your SHA16384 hashes, sign those, and store them. That way you can even use an attached (as opposed to detached) gpg signature, without confusing downstream tools. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100310051154.gs18...@p12n.org
Bug#540215: Introduce dh_checksums
retitle 540215 Introduce dh_checksums
tag 540215 +patch
thanks
On Thu, 2010-03-04 at 20:08 +0100, Tollef Fog Heen wrote:
Frank Lin PIAT wrote:
What about a transitional dh_md5sums that would produce md5sum AND
invoke dh_sha ?
Or call it dh_checksums or something so we don't have to change the tool
name each time we decide to change the algorithm.
Hello,
Find a patch attached, for a smooth transition from DEBIAN/md5sums to a
recent checksum.
The way it is implemented, is that the dh_md5sums is a symlink to the
new dh_checksums. The new helper computes both md5sum (for
compatibility/transition) and a new checksum (SHA256, which was already
chosen by FTP-masters as a remplacement for md5sum for signed files)
Note regarding the patch:
I have tried to make the patch so it isn't too intrusive (for
instance, dh_checksums is a symlink to dh_md5sums even though it
should be the other way around).
Your comments on the patch are obviously welcome (feel free to hack
it your self if you want)
Any chance to merge it before squeeze Freeze?
Franklin
From 69799a95b470c19cd395c532356eeaa64bc1bac8 Mon Sep 17 00:00:00 2001
From: Frank Lin PIAT fp...@klabs.be
Date: Mon, 8 Mar 2010 16:35:39 +0100
Subject: [PATCH] Implement dh_checksums.
---
debian/copyright |2 +-
debian/links |3 +++
dh |2 +-
dh_md5sums | 41 +
4 files changed, 38 insertions(+), 10 deletions(-)
create mode 100644 debian/links
diff --git a/debian/copyright b/debian/copyright
index a9f950d..162bfc0 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -48,7 +48,7 @@ Copyright: Steve Robbins s...@debian.org
License: GPL-2+
Files: dh_md5sums
-Copyright: Charles Briscoe-Smith c...@ukc.ac.uk
+Copyright: Charles Briscoe-Smith c...@ukc.ac.uk, Frank Lin PIAT
fp...@klabs.be
License: GPL-2+
Files: dh_bugfiles
diff --git a/debian/links b/debian/links
new file mode 100644
index 000..3e7d603
--- /dev/null
+++ b/debian/links
@@ -0,0 +1,3 @@
+usr/share/man/man1/dh_md5sums.1.gz usr/share/man/man1/dh_checksums.1.gz
+usr/bin/dh_md5sums usr/bin/dh_checksums
+
diff --git a/dh b/dh
index bcac8da..0aa9bc3 100755
--- a/dh
+++ b/dh
@@ -322,7 +322,7 @@ $sequences{install} = [...@{$sequences{build}}, qw{
my @b=qw{
dh_installdeb
dh_gencontrol
- dh_md5sums
+ dh_checksums
dh_builddeb
};
$sequences{'binary-indep'} = [...@{$sequences{install}}, @b];
diff --git a/dh_md5sums b/dh_md5sums
index da00090..33bf561 100755
--- a/dh_md5sums
+++ b/dh_md5sums
@@ -2,7 +2,7 @@
=head1 NAME
-dh_md5sums - generate DEBIAN/md5sums file
+dh_checksums - generate DEBIAN/*sums files (md5, sha256)
=cut
@@ -12,18 +12,24 @@ use Debian::Debhelper::Dh_Lib;
=head1 SYNOPSIS
+Bdh_checksums [SIdebhelper options] [B-x] [B-XIitem]
[B--include-conffiles]
+
Bdh_md5sums [SIdebhelper options] [B-x] [B-XIitem]
[B--include-conffiles]
=head1 DESCRIPTION
-dh_md5sums is a debhelper program that is responsible for generating
-a DEBIAN/md5sums file, which lists the md5sums of each file in the package.
-These files are used by the debsums package.
+dh_checksums is a debhelper program that is responsible for generating
+a DEBIAN/md5sums and DEBIAN/sha256sums files, which respectively lists the
+md5sums and sha256sums of each file in the package. These files are used
+by the debsums package.
-All files in DEBIAN/ are omitted from the md5sums file, as are all
+All files in DEBIAN/ are omitted from the checksums files, as are all
conffiles (unless you use the --include-conffiles switch).
-The md5sums file is installed with proper permissions and ownerships.
+The checksums files are installed with proper permissions and ownerships.
+
+dh_md5sums is deprecated, you should use dh_checksums instead, which generates
the
+type of checksums files recommended by the Debian policy.
=head1 OPTIONS
@@ -37,7 +43,7 @@ redundant since it is included elsewhere in debian packages.
=item B-XIitem, B--exclude=Iitem
Exclude files that contain item anywhere in their filename from
-being listed in the md5sums file.
+being listed in the checkums file.
=back
@@ -48,15 +54,26 @@ init(options = {
include-conffiles = \$dh{INCLUDE_CONFFILES},
});
+my ($basename) = $0=~m:.*/(.+):;
+
foreach my $package (@{$dh{DOPACKAGES}}) {
next if is_udeb($package);
-
+
+ if (basename($0) == 'dh_md5sums') {
+ warning(This program should no longer be used. Please read the
dh_checksums(1) man page.);
+ }
+
my $tmp=tmpdir($package);
if (! -d $tmp/DEBIAN) {
doit(install,-d,$tmp/DEBIAN);
}
+ # Detect if this is run multiple times (calling both dh_md5sums and
dh_checksums?)
+ if (-f $tmp/DEBIAN/md5sums or -f $tmp/DEBIAN/sha256sums) {
+ warning(Re-computing checksum file (even though md5sums and/or
sha256sums exists));
+ }
+
# Check if we should
Re: Bug#540215: Introduce dh_checksums
Frank Lin PIAT wrote:
Note regarding the patch:
I have tried to make the patch so it isn't too intrusive (for
instance, dh_checksums is a symlink to dh_md5sums even though it
should be the other way around).
Symlink direction seems irrelevant.
I'd probably just make dh_md5sums call dh_checksums, and later add
a deprecation warning message.
Your comments on the patch are obviously welcome (feel free to hack
it your self if you want)
Any chance to merge it before squeeze Freeze?
Is debsums ready to handle other checksums types?
+a DEBIAN/md5sums and DEBIAN/sha256sums files, which respectively lists the
So this doubles the amount of work that's done on build. Is there any
reason to generate md5sums files, aside from keeping old debsums
working?
+my ($basename) = $0=~m:.*/(.+):;
Dh_Lib has a basename()
+ if (basename($0) == 'dh_md5sums') {
+ warning(This program should no longer be used. Please read the
dh_checksums(1) man page.);
+ }
It's probably too early for this warning, I prefer to give people some
time before starting to nag.
--
see shy jo
signature.asc
Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Mon, 2010-03-08 at 12:21 -0500, Joey Hess wrote:
Frank Lin PIAT wrote:
Note regarding the patch:
I have tried to make the patch so it isn't too intrusive (for
instance, dh_checksums is a symlink to dh_md5sums even though it
should be the other way around).
Symlink direction seems irrelevant.
I'd probably just make dh_md5sums call dh_checksums, and later add
a deprecation warning message.
Your comments on the patch are obviously welcome (feel free to hack
it your self if you want)
Any chance to merge it before squeeze Freeze?
Is debsums ready to handle other checksums types?
Currently, debsums silently ignores sha256 checksums, so it won't break
if we start shipping those checksums.
I intend to submit a patch (see the TODO list[1])
+a DEBIAN/md5sums and DEBIAN/sha256sums files, which respectively lists the
So this doubles the amount of work that's done on build. Is there any
reason to generate md5sums files, aside from keeping old debsums
working?
Yes, this is for transition.
We still have to decide how long that transition would be.
+ if (basename($0) == 'dh_md5sums') {
+ warning(This program should no longer be used. Please read the
dh_checksums(1) man page.);
+ }
It's probably too early for this warning, I prefer to give people some
time before starting to nag.
I agree,
Thank you for your quick review. I'll keep you informed about
lintian/debsums patch.
Franklin
[1] http://wiki.debian.org/Sha256sumsInPackages
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/1268070281.21347.10033.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
Frank Lin PIAT fp...@klabs.be writes: Find a patch attached, for a smooth transition from DEBIAN/md5sums to a recent checksum. The way it is implemented, is that the dh_md5sums is a symlink to the new dh_checksums. The new helper computes both md5sum (for compatibility/transition) and a new checksum (SHA256, which was already chosen by FTP-masters as a remplacement for md5sum for signed files) If there's a general consensus that we should go this route, I'm okay with it, but I'm personally not sure this is the right approach. I think there are two possible directions we can take with this package feature: 1. Strengthen the integrity check so that it could potentially be useful for security purposes as well as for simple integrity checking. 2. Declare such checksums only useful for corruption and local (benign) modification checksumming. If we take option 1, going to a stronger hash is strictly less useful in every respect than going to embedded PGP signatures, which apparently only require active maintenance and a plan to be acceptable in the archive and which would need a different format anyway. If we take option 2, SHA256 offers no benefits over MD5 and just takes longer to compute. There is essentially zero chance that random file corruption or typical local modification will result in a successful MD5 preimage attack. I therefore have to question what utility there is in switching to SHA256. It doesn't seem to achieve either possible goal, unless I'm missing some way in which it's a step towards option 1? -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87wrxmbkdn@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
On Mon, 2010-03-08 at 12:59 -0800, Russ Allbery wrote: Frank Lin PIAT fp...@klabs.be writes: Find a patch attached, for a smooth transition from DEBIAN/md5sums to a recent checksum. The way it is implemented, is that the dh_md5sums is a symlink to the new dh_checksums. The new helper computes both md5sum (for compatibility/transition) and a new checksum (SHA256, which was already chosen by FTP-masters as a remplacement for md5sum for signed files) 1. Strengthen the integrity check so that it could potentially be useful for security purposes as well as for simple integrity checking. Yes, this is the intended goal. Imagine the following scenario: 1. You boot a trusted device with Debian Live or D-I 2. A helper mounts the root, usr and var partitions of the inspected system 3. It then fetches the list of installed package and version 4. It then fetches the list of signatures for the installed system, from your trusted mirror. 5. Then it validates the installed files on the inspected system. 6. You just have to trust one GPG key (per repository). All that relies on the current infrastructure and chain of trust (Repository's Releases.gpg-Packages.gpg-checksum). Alternatively, what I am currently doing is to periodically export my md5sums to a trusted system. It would be much easier if a signed list of check-sums for current packages in our archive were published (basically, when we create Packages.gpg, we would need to extract the checksums contained in those packages, then sign it. This list could also included the recently updated packages, which is useful for point-releases, and unstable). The benefit is that you can quickly audit if installed binaries are pristine AND current. If we take option 1, going to a stronger hash is strictly less useful in every respect than going to embedded PGP signatures Can you elaborate? checksums are currently signed, no? which apparently only require active maintenance and a plan to be acceptable in the archive and which would need a different format anyway. I see some problems with signed packages: 1. Software developers and vendors will start distributing standalone binary packages, and pretend they are safe because they are signed. This is not safe: only an APT-like mechanism provides security updates. 2. You still need an infrastructure to publish valid packages version. 3. What do we do when we want to change a GPG key? we re-sign all the packages, probably breaking existing packages checksums? Last but not least: 4. How long is it going to implement it? Does it seems realistic to implement your proposal before Squeeze +1 (or event Squeeze +2)? What do we do in-between? If we take option 2, SHA256 offers no benefits over MD5 and just takes longer to compute. Why is that everyone seems to move away from MD5? There is essentially zero chance that random file corruption or typical local modification will result in a successful MD5 preimage attack. An attacker has plenty of time to pre-compute a valid replacement for, let say, a library in Debian Stable. I therefore have to question what utility there is in switching to SHA256. It doesn't seem to achieve either possible goal, unless I'm missing some way in which it's a step towards option 1? As a bottom line: 1. A package is not safe because it is signed, but because it is up-to-date. 2. I am completely ok go for GPG packages, *if* it can be implemented by squeeze. Otherwise, let's move on to sha*** for squeeze, which can be quite transparent, and move to GPG post squeeze. Regards, Franklin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1268085864.21347.11498.ca...@solid.paris.klabs.be
Re: Bug#540215: Introduce dh_checksums
Frank Lin PIAT fp...@klabs.be writes: On Mon, 2010-03-08 at 12:59 -0800, Russ Allbery wrote: 1. Strengthen the integrity check so that it could potentially be useful for security purposes as well as for simple integrity checking. Yes, this is the intended goal. Imagine the following scenario: 1. You boot a trusted device with Debian Live or D-I 2. A helper mounts the root, usr and var partitions of the inspected system 3. It then fetches the list of installed package and version 4. It then fetches the list of signatures for the installed system, from your trusted mirror. 5. Then it validates the installed files on the inspected system. 6. You just have to trust one GPG key (per repository). All that relies on the current infrastructure and chain of trust (Repository's Releases.gpg-Packages.gpg-checksum). Alternatively, what I am currently doing is to periodically export my md5sums to a trusted system. The missing link, in this validation scenario, is how to get a signed copy of the MD5 checksums of the files in the package. If that package is still current, you can get this by downloading and extracting the copy of that package from the Debian archive and examining the MD5 checksums in it, but of course if the package is still current you don't even need MD5 checksums. You can just compare the files directly with the files you have. Hence, adding a stronger checksum doesn't really help, except possibly for some minor convenience. It would be much easier if a signed list of check-sums for current packages in our archive were published (basically, when we create Packages.gpg, we would need to extract the checksums contained in those packages, then sign it. This list could also included the recently updated packages, which is useful for point-releases, and unstable). Yes, this plus a repository for older checksums for packages that are no longer current would be required to let this work as an audit. (It's common in practice for not all packages on a system to be at the latest version.) But if you instead put a PGP signature directly in the package, then you can always verify the package regardless of whether it's current or not provided that you still have the original package (in /var/cache/apt, for instance), which is why this is superior to using a hash. Not to mention that the infrastructure you describe for publishing signed hashes doesn't really exist and you have to download the full package. If we take option 1, going to a stronger hash is strictly less useful in every respect than going to embedded PGP signatures Can you elaborate? checksums are currently signed, no? No, the MD5 checksums currently included in packages aren't signed. The package as a whole, which includes the checksum, is itself checksumed, and that checksum is included in Packages, which in turn is signed. So we have validation of current packages at time of installation. But as soon as that's no longer the current package (which happens all the time in unstable and testing, and happens in stable with new point releases), we lose that signature because it depends on the current Packages list. I see some problems with signed packages: 1. Software developers and vendors will start distributing standalone binary packages, and pretend they are safe because they are signed. This is not safe: only an APT-like mechanism provides security updates. Well, it depends on how you define safety. The signed package is now verified to come from the package signer, which is indeed more safe. Having ongoing security support is a separate problem. I see this as a feature. It lets one distribute and install packages with verified origins without needing to set up a full apt archive, which a lot of people find tricky to do properly. 2. You still need an infrastructure to publish valid packages version. I don't think we do. 3. What do we do when we want to change a GPG key? we re-sign all the packages, probably breaking existing packages checksums? Key change is a problem, yes, but note that an expired PGP signature is still stronger in all respects than a SHA256 hash, given that you can always ignore the signature part completely and just verify the hash. (Assuming the signature uses a reasonable hash, of course.) So while this is a problem, it doesn't pose any worse of a problem than just having the SHA256 hash does. Last but not least: 4. How long is it going to implement it? Does it seems realistic to implement your proposal before Squeeze +1 (or event Squeeze +2)? What do we do in-between? Use MD5. My point is mostly that it's not clear to me SHA256 is any better than what we have now for the problem that hashes are capable of solving. If we take option 2, SHA256 offers no benefits over MD5 and just takes longer to compute. Why is that everyone seems to move away from MD5? Because MD5 is not sufficient protection against an attacker. It's
Re: Bug#540215: Introduce dh_checksums
Russ Allbery wrote: The missing link, in this validation scenario, is how to get a signed copy of the MD5 checksums of the files in the package. That's one missing link. The other one is that there are innumerable ways for an attacker to inject bad behavior/backdoors onto a system without touching binaries originating from dpkg. Expecting debsums to protect against any form of attack is bound to result in a false sense of security; and AFAIK aide makes a credible[1] attempt at solving the same problem. -- see shy jo, who does not need to be CCed anymore on this thread [1] Though my SWAG is that it's still not complete when you consider the boodloader, permissions of files in /dev, or subtly corrupted partitions. signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
On Tue, 9 Mar 2010, Joey Hess jo...@debian.org wrote: Russ Allbery wrote: The missing link, in this validation scenario, is how to get a signed copy of the MD5 checksums of the files in the package. That's one missing link. The other one is that there are innumerable ways for an attacker to inject bad behavior/backdoors onto a system without touching binaries originating from dpkg. Expecting debsums to protect against any form of attack is bound to result in a false sense of security; and AFAIK aide makes a credible[1] attempt at solving the same problem. [1] Though my SWAG is that it's still not complete when you consider the boodloader, permissions of files in /dev, or subtly corrupted partitions. http://etbe.coker.com.au/2010/03/08/designing-secure-linux/ I blogged about some of these things yesterday. -- russ...@coker.com.au http://etbe.coker.com.au/ My Main Blog http://doc.coker.com.au/ My Documents Blog -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201003091114.23871.russ...@coker.com.au
Re: Bug#540215: Introduce dh_checksums
On Mon, Mar 08, 2010 at 11:04:24PM +0100, Frank Lin PIAT wrote: On Mon, 2010-03-08 at 12:59 -0800, Russ Allbery wrote: 1. Strengthen the integrity check so that it could potentially be useful for security purposes as well as for simple integrity checking. It would be much easier if a signed list of check-sums for current packages in our archive were published (basically, when we create Packages.gpg, we would need to extract the checksums contained in those packages, then sign it. This list could also included the recently updated packages, which is useful for point-releases, and unstable). I'm a bit confused here. I think that here is a mix of package signatures and file signatures. These serve different purposes and are completely separate. A package signature lets you verfy the package before it is decompressed and, more importantly, maintainer scripts are run as root. File signatures let you check the installed files. Both should be part of the shipped package. Package signature as files in the arch archive, file signatures should be installed along with the files so you can always directly check installed files, without the need of any repository or original packages lying around. The benefit is that you can quickly audit if installed binaries are pristine AND current. If we take option 1, going to a stronger hash is strictly less useful in every respect than going to embedded PGP signatures Assuming file hashes here: create stronger hashes and then sign the hash file. This has 2 advantages: - hashes can be used to check file integrity even without the key - it can be implemented incrementally (1st: stonger hashes, 2nd: signature) which apparently only require active maintenance and a plan to be acceptable in the archive and which would need a different format anyway. Assuming package hashes here: a view additional files in the arch archive. I see some problems with signed packages: 1. Software developers and vendors will start distributing standalone binary packages, and pretend they are safe because they are signed. This is not safe: only an APT-like mechanism provides security updates. Yes, there seems to be a common misconception about what a signature means. But that's no argument to deny informed people the advantages of signatures. 2. You still need an infrastructure to publish valid packages version. Isn't that the Release file? 3. What do we do when we want to change a GPG key? we re-sign all the packages, probably breaking existing packages checksums? No, the signature has a timestamp and is still valid, if the revocation date of key is later. Last but not least: 4. How long is it going to implement it? Does it seems realistic to implement your proposal before Squeeze +1 (or event Squeeze +2)? What do we do in-between? If we take option 2, SHA256 offers no benefits over MD5 and just takes longer to compute. Why is that everyone seems to move away from MD5? Because it's broken? There is essentially zero chance that random file corruption or typical local modification will result in a successful MD5 preimage attack. An attacker has plenty of time to pre-compute a valid replacement for, let say, a library in Debian Stable. I therefore have to question what utility there is in switching to SHA256. It doesn't seem to achieve either possible goal, unless I'm missing some way in which it's a step towards option 1? See above. As a bottom line: 1. A package is not safe because it is signed, but because it is up-to-date. A signature hasn't got anything to do with how safe a package is. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309033126.ga15...@nn.nn
Re: Bug#540215: Introduce dh_checksums
On Mon, Mar 08, 2010 at 05:59:13PM -0500, Joey Hess wrote: Russ Allbery wrote: The missing link, in this validation scenario, is how to get a signed copy of the MD5 checksums of the files in the package. That's one missing link. The other one is that there are innumerable ways for an attacker to inject bad behavior/backdoors onto a system without touching binaries originating from dpkg. Signatures don't prevent bugs, they don't prevent trojans, they don't prevent attacks on SSH. But they let you *detect* attacks. It's not that easy to install a root kit that hides all changes and you can always boot from a trusted medium to check your files. Without signatures, you can't, or at least it a lot harder. Expecting debsums to protect against any form of attack is bound to result in a false sense of security; I don't expect that. harry -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100309033842.gb15...@nn.nn
Re: Bug#540215: Introduce dh_checksums
Harald Braumann ha...@unheit.net writes: On Mon, Mar 08, 2010 at 05:59:13PM -0500, Joey Hess wrote: That's one missing link. The other one is that there are innumerable ways for an attacker to inject bad behavior/backdoors onto a system without touching binaries originating from dpkg. Signatures don't prevent bugs, they don't prevent trojans, they don't prevent attacks on SSH. But they let you *detect* attacks. It's not that easy to install a root kit that hides all changes and you can always boot from a trusted medium to check your files. Without signatures, you can't, or at least it a lot harder. It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. We should always be striving for the best possible security and improving security measures, but along the way security measures that have weaknesses against a determined attacker can still be practically useful. It's a constant balance to not sacrifice real security for expediency, but at the same time to not discard tools that are already available and deployable just because they can't catch the most determined attackers. *Some* checking is better than *no* checking as long as you clearly understand what the capabilities and tradeoffs of the system you have are and don't think they're more than what they are. There are very, very few absolutes in computer security. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k4tmupju@windlord.stanford.edu
Re: Bug#540215: Introduce dh_checksums
Russ Allbery wrote: It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. -- see shy jo signature.asc Description: Digital signature
Re: Bug#540215: Introduce dh_checksums
Joey Hess jo...@debian.org writes: Russ Allbery wrote: It's also always worth bearing in mind that while a really good attacker can do all sorts of complex things that make them very hard to find, most attackers are stupid and straightforward. It's stupid and straightforward to install /usr/local/bin/ls. debsums will not detect it. True. Adding new binaries is, in my experience, more common than modifying binaries already on the system. I don't really mean to be arguing for debsums as a security mechanism, more just commenting on the general question. I'm on the side that thinks that debsums isn't a horribly useful direction to go for full-blown intrusion detection, and that for what it's really useful for right now MD5 remains entirely adequate. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87d3zeuoxr@windlord.stanford.edu
