Re: debdelta, Re: proposed release goal: DEBIAN/md5sums for all packages
Jörg Sommer [EMAIL PROTECTED] wrote: Sorry, I can't remember the name of the package. That must be cm-super. -- Florent -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debdelta, Re: proposed release goal: DEBIAN/md5sums for all packages
On Die, 04 Sep 2007, Florent Rougon wrote: Sorry, I can't remember the name of the package. That must be cm-super. Yup, cm-super does this trick. I once wanted to undo this and ship the font files directly, but got quite a lot of requests why the packages has gotten soo big. From the rules file (with some additional comments): # create md5sums for all but the type1 and the container file # from which the fonts are created dh_md5sums -p cm-super -X usr/share/texmf/fonts/type1/public/cm-super -X usr/share/cm-super # create the correct md5sums for the files generated on postinst (cd pfb ; for pfb in *.pfb ; do \ bn=`basename $$pfb .pfb` ; \ if ! grep -q ^$$bn$$ ../debian/fonts.cm-super-minimal ; then \ cat $$pfb | md5sum - | sed -e s|-|usr/share/texmf/fonts/type1/public/cm-super/$$pfb| ; \ fi ; \ done) debian/cm-super/DEBIAN/md5sums # add the md5sum of the empty file (t1c will be emptied on # postinst) echo d41d8cd98f00b204e9800998ecf8427e usr/share/cm-super/cm-super.t1c debian/$(package)/DEBIAN/md5sums Best wishes Norbert --- Dr. Norbert Preining [EMAIL PROTECTED]Vienna University of Technology Debian Developer [EMAIL PROTECTED] Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 --- The Web site you seek Cannot be located, but Countless more exist. --- Windows Error Haiku -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debdelta, Re: proposed release goal: DEBIAN/md5sums for all packages
Norbert Preining [EMAIL PROTECTED] writes: On Die, 04 Sep 2007, Florent Rougon wrote: Sorry, I can't remember the name of the package. That must be cm-super. Yup, cm-super does this trick. I once wanted to undo this and ship the font files directly, but got quite a lot of requests why the packages has gotten soo big. Ah, there are overrides. That explains it. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debdelta, Re: proposed release goal: DEBIAN/md5sums for all packages
Hi Russ, Russ Allbery [EMAIL PROTECTED] wrote: A Mennucc [EMAIL PROTECTED] writes: BTW, I also encountered a strange bug : sometimes the md5sums file contains MD5 of files that are not shipped. This is printed as a warning in my server. If MD5 will become a release goal, this should be corrected as well : in case, I will send bug reports. This should trigger the lintian tag md5sums-lists-nonexisting-file, but lintian.debian.org doesn't see any instances of that tag in the archives. I'd be very interested in example packages, since it sounds like the lintian test may be broken. I know of a package of the TeX team where they build font files or such, create the md5sums for them and remove these files from the package, but leave the md5sums and the entries in lists. In postint they rebuild these files from those shipped in the package. The intention is that the package with the files is 60MB and without the files it is 10MB, IIRC, and it's easy to rebuild them in postinst. Sorry, I can't remember the name of the package. Bye, Jörg. -- Damit das Mögliche entsteht, muß immer wieder das Unmögliche versucht werden. (Hermann Hesse) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Mon, Aug 27, 2007 at 12:04:51PM +0200, A Mennucc wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Stefano Zacchiroli ha scritto: In an attempt to prevent drift to a well-known counter argument: DEBIAN/md5sums (used by debsums) are *not* intended as a mean to counter security attacks, since they can be easily altered. If md5sums become part of the policy, then this brings me to an old idea of mine. (... idea related to forensic use of md5sums ...) This we could do already. We don't need md5sums in files, a script could just generate this for a stable release and publish that file (signed). Even better, that file could ship whatever hashes we believed were good enough for forensics (MD5? SHA-1? SHA-256?). I think I already pointed people interested in this to #268658. If ftpmasters where given the tools to implement this seamlessly then you could have aside tools that downloaded that file from the FTP site, and locally checked the md5sums. Regards Javier PS: BTW, if you do this with a searchable web interface you also have to ensure that you have a trust path to it, that means using SSL with a good certificate.. signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Javier Fernández-Sanguino Peña ha scritto:
On Mon, Aug 27, 2007 at 12:04:51PM +0200, A Mennucc wrote:
I think I already pointed people interested in this to #268658.
If ftpmasters where given the tools to implement this seamlessly then you
could have aside tools that downloaded that file from the FTP site, and
locally checked the md5sums.
AFAICS in bug 268658 you propose to ship a signed 'Checksums-${ARCH}.gz'
with releases.
What I had in mind was slightly broader, though.
What I have in mind is a database containing all checksums of all binary
packages passing trough unstable, with records such as
package / arch / version / file / permissions / md5 / sha1
The 'Checksums-${ARCH}.gz' that you mention in 268658 may be generated
from this database at release time; but also the database would be
useful for people using tracking testing and unstable. The database may
have web interface, and/or a LDAP interface (with cryptographic
protection), so it may be searched. When doing forensic, it would be
useful to search it using the hash as a key.
Again, following your reasoning in 268658, I would then add a link to
the web interface in packages pages such as
http://packages.debian.org/testing/base/procps
But you are definitely right on one point: records should be added by a
script inside the incoming queue.
a.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFG1I0R9B/tjjP8QKQRAr2BAJ4/dRWnUX8W6SRF+Uy9QqTd127uQACePtGH
1gprvSqm26Z7t5zepFpEkYI=
=1IVv
-END PGP SIGNATURE-
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Tue, Aug 28, 2007 at 11:01:06PM +0200, A Mennucc wrote:
Javier Fernández-Sanguino Peña ha scritto:
On Mon, Aug 27, 2007 at 12:04:51PM +0200, A Mennucc wrote:
I think I already pointed people interested in this to #268658.
If ftpmasters where given the tools to implement this seamlessly then you
could have aside tools that downloaded that file from the FTP site, and
locally checked the md5sums.
AFAICS in bug 268658 you propose to ship a signed 'Checksums-${ARCH}.gz'
with releases.
What I had in mind was slightly broader, though.
What I have in mind is a database containing all checksums of all binary
packages passing trough unstable, with records such as
package / arch / version / file / permissions / md5 / sha1
It seems obvious to me by this point that this thread is not about something
that should be a candidate release goal; a task should be presented as a
release goal *after* the technical details have been sorted out and all the
proposer is looking for is for the release team to support NMUing to
implement the existing consensus.
So please don't cc: this discussion to debian-release, which is not a
discussion list. :)
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
[EMAIL PROTECTED] http://www.debian.org/
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
debdelta, Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 hi just for the record : debdelta uses md5sums (when available) as a way to speed up delta creation, to rapidly detect if there are any identical files in the archives. So , yes, I (*) would be happy if md5sums where always available. BTW, I also encountered a strange bug : sometimes the md5sums file contains MD5 of files that are not shipped. This is printed as a warning in my server. If MD5 will become a release goal, this should be corrected as well : in case, I will send bug reports. a. * that is, my server that generates the patches for 'debdelta-upgrade' would be faster, and that would make me happier -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG0plu9B/tjjP8QKQRAm+iAKCLbo9dUeQtA3fR9FV9rIcLp8mCkgCfcWoj g1Rw/3sW8rPgaI3/laq/yn0= =jcX9 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Marc 'HE' Brockschmidt ha scritto: Yes, that sounds like a good idea. It might also be interesting to not put those into the control.tar.gz, but directly into the deb, so that it can easily be extracted. I do not agree, for two reasons: 1) it is quite easy, by piping 'ar' and 'tar' , to extract files such md5sums from control.tar.gz 2) if md5sums is inside control.tar.gz, then it is compressed better, since the list of files is also inside control.tar.gz, and gzip is quite good at compressing repeated stuff a. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG0p4a9B/tjjP8QKQRAjv3AJ4jSAZei167CemUvLu1LZ8KDjAE/QCggHm4 nsnr8dj2Abjo9mvFFgdyElE= =K2wf -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Lars Wirzenius ha scritto: It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. This way we wouldn't have to change packages to do it, and if we ever want to change the format (sha1 as well as md5?) there's only one place to change it. this would guarantee that the list will really reflect what is inside the data.tar.gz as I said in another post, my debdelta-generating-server often complains that packages ship a md5sum that does not correspond to what is in data.tar.gz a. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG0p6a9B/tjjP8QKQRAhP4AJ9MfotrnlskSd+TsArbYQP0kyvffgCfe2nR GMHUwKKTHNONe8V1j3o9g9s= =RRx8 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Stefano Zacchiroli ha scritto: In an attempt to prevent drift to a well-known counter argument: DEBIAN/md5sums (used by debsums) are *not* intended as a mean to counter security attacks, since they can be easily altered. If md5sums become part of the policy, then this brings me to an old idea of mine. Idea: we set up a database containing those md5sums , for all packages/versions that pass thru the archive, and add a web interface to that. This database then may be really used in forensic. Example usage. Suppose that I find out that my PC has been hacked. I then shut it down immediatly, and grab a live CD. I boot my PC using the live CD, and have it connect to Internet. I then start a simple utility (think of 'debsums --web --root'), that, for any package that is installed in the OS in the PC, downloads the md5sum for that version from the web interface, and goes checking; eventually leaving a list of all files that did not check OK or that were found in /etc /usr /bin ... and have no md5sum. Of course this would give many false positives, (such as the aspell hashes, as is discussed in a subthread ; and a lot of stuff in /etc); but it would be useful to prune the majority of OK files out, and leave a small subset for human forensic analysis. a. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG0qHD9B/tjjP8QKQRAmJnAJ9oUWwME6Q8g6JrRt6bF4nk6HYIawCdG1hP GRyBERL04/5Nz2/YmM16uts= =M3m4 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Peter Samuelson ha scritto: [Lars Wirzenius] It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. we already have debsums to do that also, in that case, it would damage my debdelta server :-( a. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG0qJt9B/tjjP8QKQRAmiBAJ9vC9AVUOZMJvyK1yHnG+X6IhcN6gCgn24X rbGdgmcvfWdshScmb7UtIg4= =44br -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Goswin von Brederlow ha scritto: So why waste all the mirror space and bandwith for something rather useless? I did not do statistics; but, knowing how compression works, I would estimate that the cost of shipping md5sums is ~ 20 bytes for each file that is in data.tar.gz IMHO this is not wasting a lot of bandwidth... a. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG0qTl9B/tjjP8QKQRAnL2AJ9JLRPRmm/nmK3U66jzbedkyq6wywCZAfGS eUXy6V36rTWN60/hSFjh8cc= =Ihrc -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: debdelta, Re: proposed release goal: DEBIAN/md5sums for all packages
A Mennucc [EMAIL PROTECTED] writes: BTW, I also encountered a strange bug : sometimes the md5sums file contains MD5 of files that are not shipped. This is printed as a warning in my server. If MD5 will become a release goal, this should be corrected as well : in case, I will send bug reports. This should trigger the lintian tag md5sums-lists-nonexisting-file, but lintian.debian.org doesn't see any instances of that tag in the archives. I'd be very interested in example packages, since it sounds like the lintian test may be broken. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Goswin von Brederlow wrote: So why waste all the mirror space and bandwith for something rather useless? Naïve approximation follows: Repacking my local apt cache (227 packages, although some are different versions of the same one) without md5sums files yields a gain of 980102 bytes = 957.13 KB 1 MB (apparently 13 do not already have a md5sum file, or weight the same with or without md5sums file). Given that my local cache weights about 352 MB, then the gain is less than 1/352*100 = 0.28%. Doesn't seem like a huge waste to me, but then I'm not hosting a mirror. I would like to note that my local apt cache has a diversity of packages, ranging from small dummy packages like amarok-engines to relatively large ones like texlive-latex-base or openoffice.org-core. Thus the approximation is not that bad I think. -- Felipe Sateler -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
(size savings +) Re: proposed release goal: DEBIAN/md5sums for all packages
* Pierre Habouzit * Date: Fri, 17 Aug 2007 15:22:05 +0200 [] Yes, that sounds like a good idea. It might also be interesting to not put those into the control.tar.gz, but directly into the deb, so that it can easily be extracted. OTOH that sucks because it would mean that we have to rebuild the whole archive that uses currently dh_md5sums, whereas we could just be backward compatible. After playing with size reduction, i came up with stripping configs and regenerating md5sums (if any). Yes, some packages aren't have them, but after removing much of the stuff all must be regenerated anyway. # dpkg-deb wrapper for geloiwa: cd $WDIR # here WDIR=WDIR/data GACONF=`du -s $CFGDIR` md5sum `find . -type f | sort` $CFGDIR/md5sums /dev/null cd $CFGDIR cleanup_debconf cleanup_scripts GAdtCONF=$((`ff $GACONF` - `ff $(du -s)`)) cd $WDIR/../ printf _ $PKG\t\t$GAdtSTAT+$GAdtCONF\t$GASTAT+$GACONF stats tar c -C data -f $DATAFILE . # shend How much size was reduced in each case is stored in shell-syntax stats file for ease of totals generation. Another design problem in this dpkg-deb-dpkg dance is this re-taring of data for dpkg. After another package database format, hopefully more faster, this is issue to solve. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Romain Francoise [EMAIL PROTECTED] writes: Stefano Zacchiroli [EMAIL PROTECTED] writes: [ fully quoting my original request, for the sake of context preservation ] Thanks for initiating the discussion! :-) On Fri, Aug 17, 2007 at 09:04:13AM +0200, Luk Claes wrote: With more than 600 issues, it's a bit early to make it a release goal IMHO. Though making maintainers aware by upgrading the lintian check to a warning and discussion on debian-devel about which exceptions are warranted (and possible mass bug filing) will probably be a good idea to get the amount reduced rather fast... One thing I've been pondering about is: are there any good reasons *not* to have an md5sums control file? It seems to me that the time spent to generate it on the buildds is probably insignificant compared to the total time needed to build the package... And since generating it can be done with a trivial shell command, it's not a complexity issue either. I fail to see any reason to HAVE a md5sums file. The package is signed (via Release.gpg, Release, Packages, md5sum+size) and thereby protected against tampering. The md5sum file in / var/lib/dpkg/info/ (or wherever you put it on the users system) is not protected and therefore useless as a security device. Fetching a md5sum file you can trust means fetching (at least partially) the deb and then you can just compare the files one by one. Also the md5sum file can be generated at install time if wanted. The cost of computing the md5sum on the fly is quite insignificant on most systems. So why waste all the mirror space and bandwith for something rather useless? MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 24, 2007 at 03:16:28PM +0200, Goswin von Brederlow wrote: I fail to see any reason to HAVE a md5sums file. It looks like you have not read all the thread, other's have made some good points as to why it's good. Just in case I'm going to voice my opinion here again and see if I can convicen you (and other's listening) :) The md5sum file in / var/lib/dpkg/info/ (or wherever you put it on the users system) is not protected and therefore useless as a security device. Fetching a md5sum file you can trust means fetching (at least partially) the deb and then you can just compare the files one by one. Useless sercurity device is an overstatemente here. One of security's fundamental corner stones is 'integrity'. System integrity can be lost due to: - a person without a malicious intent accidentally or on purpose changes the system, e.g. a novice admin that modified a script at /usr/bin installed by a package or redirected his stderr to a file he shouldn't have after mistyping a command. - uncontrolled accidents or disasters, e.g. hard disk / memory corruption in a system which makes it incorrectly write to disk a binary unpackaged from a package. - somebody with malicious intent, e.g. an unautorised user that elevates privileges and installs a trojan I do agree that the last case is probably only handled well with a signed hash database in a read only media (the Debian Security Manual gives some examples) but the two others can be served quite well just by including md5sum files in packages. So, md5sums *do* serve a security purpose even if they are not able to stop a determined cracker from violating the system's integrity in a way we cannot detect it. FWIW, IMHO the forst type of integrity losses are more common than the last. Also the md5sum file can be generated at install time if wanted. The cost of computing the md5sum on the fly is quite insignificant on most systems. Some computing systems cannot incur the cost (please read the thread). What do you think is (globally) more CPU-concious: compute once (in the buildds) and put in a file for everybody to use or compute once in every system the package is installed on. There might be hundreds of build systems (including the developer's Debian systems where packages are built) but there are thousands of users. It is quite obvious to me that we are saving energy if we just distribute them instead of forcing our end-users to recompute them. Energy is a scarse resources, save the planet! :) So why waste all the mirror space and bandwith for something rather useless? Waste all seems like a very bad phrase. The impact in our archive of mandating md5sums or sha1sums in packages when most *already* do that is neligible. From http://blog.orebokech.com/2007/08/debian-packages-without-md5sums.html: Random testing of my local Debian mirror shows that 644 binary packages out of 20774 (3.1%) are missing the DEBIAN/md5sums control file. If you want to go through the space and bandwidth road please provide some data to back it up. How much space do we munge if we *add* md5sums to packages that don't have that information already? Also: How much space do we save if we *remove* md5sums from all packages? Just my 2c, Javier signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Javier Fernández-Sanguino Peña [EMAIL PROTECTED] writes: On Fri, Aug 24, 2007 at 03:16:28PM +0200, Goswin von Brederlow wrote: I fail to see any reason to HAVE a md5sums file. It looks like you have not read all the thread, other's have made some good points as to why it's good. Just in case I'm going to voice my opinion here again and see if I can convicen you (and other's listening) :) Which nearly all can be satisfied by generating the md5sum on install. The md5sum file in / var/lib/dpkg/info/ (or wherever you put it on the users system) is not protected and therefore useless as a security device. Fetching a md5sum file you can trust means fetching (at least partially) the deb and then you can just compare the files one by one. Useless sercurity device is an overstatemente here. One of security's fundamental corner stones is 'integrity'. System integrity can be lost due to: - a person without a malicious intent accidentally or on purpose changes the system, e.g. a novice admin that modified a script at /usr/bin installed by a package or redirected his stderr to a file he shouldn't have after mistyping a command. Covered by generated md5sum files. - uncontrolled accidents or disasters, e.g. hard disk / memory corruption in a system which makes it incorrectly write to disk a binary unpackaged from a package. Memory corruption between unpacking the files and md5suming them could cause bad binaries with bad matching md5sums to be written with generated md5sum fields. But bad memory will have tons of effects and cause failures when matching md5sums too. Md5sums in debs aren't a memory tester so I don't quite care. Other corruptions won't affect md5sum generation on install so they are covered there too. - somebody with malicious intent, e.g. an unautorised user that elevates privileges and installs a trojan A malicious attacker would modify the md5sum files too making them useless as detection method. Unless he/she is stupid. I do agree that the last case is probably only handled well with a signed hash database in a read only media (the Debian Security Manual gives some examples) but the two others can be served quite well just by including md5sum files in packages. So, md5sums *do* serve a security purpose even if they are not able to stop a determined cracker from violating the system's integrity in a way we cannot detect it. Signed they would truely help. And they should also contain checksums for the other files in control.tar.gz. An intelligent hacker would just modify the pre/postrm script of a package to open some backdoor the next time the package is updated. Also a md5sum service online would be usefull for this too. E.g. packages.d.o could have a link to the md5sum file for each package so you could fetch them on a clean machine and then compare the files on the filesystem. Those md5sum files could be generated. This time not during the users install time but during DAK install time. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Javier Fernández-Sanguino Peña [EMAIL PROTECTED] writes: From http://blog.orebokech.com/2007/08/debian-packages-without-md5sums.html: Random testing of my local Debian mirror shows that 644 binary packages out of 20774 (3.1%) are missing the DEBIAN/md5sums control file. As of today my counter is down to 514 packages (2.47%). -- ,''`. : :' :Romain Francoise [EMAIL PROTECTED] `. `' http://people.debian.org/~rfrancoise/ `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 24, 2007 at 05:15:51PM +0200, Goswin von Brederlow wrote: It looks like you have not read all the thread, other's have made some good points as to why it's good. Just in case I'm going to voice my opinion here again and see if I can convicen you (and other's listening) :) Which nearly all can be satisfied by generating the md5sum on install. Sure, show us the code or do something to make the available code flow into dpkg. The code for doing that via debhelper + debsums is already there. It is even already wasting mirror bandwidth and disk usage since the percentage of packages without md5sums is quite low (seem Romain's figures). The current situation can only be described as some buggy packages which need to be fixed. Better solutions can of course be found, but they all seem, to me, more far away from the goal which can be obtained fixing the buggy package. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Bug report template for missing md5sums file (was: proposed release goal: DEBIAN/md5sums for all packages)
Luk Claes [EMAIL PROTECTED] writes: With more than 600 issues, it's a bit early to make it a release goal IMHO. Though making maintainers aware by upgrading the lintian check to a warning and discussion on debian-devel about which exceptions are warranted (and possible mass bug filing) will probably be a good idea to get the amount reduced rather fast... I've filed bug #439428 against strace, with a patch to add the 'dh_md5sums' commands. The following is presented for use by anyone who wants to file similar bugs on other packages. = Package: strace Version: 4.5.15-1 Severity: normal Tags: patch Debian policy recommends that all binary packages contain an 'md5sums' control file. The following command shows that a package built from this source package does not contain this file. $ lintian --info --display-info --check-part md5sums ./strace_4.5.15-1_powerpc.deb I: strace: no-md5sums-control-file N: N: This package does not contain an md5sums control file. This control N: file listing the MD5 checksums of the contents of the package is not N: required, but if present debsums can use it to verify that no files N: shipped with your package have been modified. Providing it is N: recommended. N: N: If you are using debhelper to create your package, just add a call to N: dh_md5sums at the end of your binary-indep or binary-arch target, N: right before dh_builddeb. N: The attached patch changes the 'debian/rules' file to generate the 'md5sums' file for this package, and causes the above command to run cleanly (with no output). If there is a specific reason for avoiding the recommended 'md5sums' file and not applying this patch or something similar, could you please explain in this bug entry. = -- \ Those are my principles. If you don't like them I have | `\ others. -- Groucho Marx | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Hello Javier, Am 2007-08-20 23:30:26, schrieb Javier Fernández-Sanguino Peña: BTW, NIST provides a very handy information called the National Software Reference Library (NSRL, http://www.nsrl.nist.gov/) which comes also very handy for either forensic analysis or setting up a baseline of known files (when using an integrity checking tool such as tripwire or samhain) for a large number of servers. If we provided such information they could possibly easily include it there too which would be an improvement, since they currently only carry information on ancient versions of Linux distributions (and Debian is not one of them) - END OF REPLIED MESSAGE - I use Debian since 1999/04 and have a local ARCHIVE mirror of several TByte of binaries and sources and I have tried to get the md5sum for all files but unfortunatly, for more then 30% of the files (deb, dsc, diff.gz and tar.gz) there are no md5sum. I use my PostgreSQL where I have stored all md5sum/sha1 + package names I ever downloaded from Debian server. I can easyly access it using a PHP webinterface or wget. Is there a source, where I can get ALL md5sums from Packages? (maybe since Bo?) Thanks, Greetings and nice Day Michelle Konzack Systemadministrator Tamay Dogan Network Debian GNU/Linux Consultant -- Linux-User #280138 with the Linux Counter, http://counter.li.org/ # Debian GNU/Linux Consultant # Michelle Konzack Apt. 917 ICQ #328449886 50, rue de Soultz MSN LinuxMichi 0033/6/6192519367100 Strasbourg/France IRC #Debian (irc.icq.com) signature.pgp Description: Digital signature
Md5/sha1sums for all the Release (was Re: proposed release goal: DEBIAN/md5sums for all packages)
On Fri, Aug 17, 2007 at 07:04:39PM -0500, Peter Samuelson wrote: [Russ Allbery] While it's not the be-all and end-all of security, other OS vendors (Sun in particular) have found it useful to make available a central database of MD5 checksums of known-good versions of various binaries. H. As far as being authoritative (and cryptographically secure), we already have $MIRROR/dists/stable/main/binary-i386/Packages.bz2. I actually would like to have a file similar to the Contents that provided the MD5/SHA1/whatever_hash of all the files distributed in Debian and have that file included in the Release file (so that it's GPG signed and we have a chain of trust). This has been discussed at #268658 but so far FTP maintainers have remain silent on this issue. Such a per Release file with all the MD5sums could be really useful to do forensic analysis of a system to detected corrupted (or locally modified) contents. It could also complement the md5sums provided by packages and serve as an additional reference to validate them if they are believed to be tampered with. Regards Javier signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 04:47:38PM -0700, Russ Allbery wrote: Peter Samuelson [EMAIL PROTECTED] writes: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. We already claim that the md5sums file isn't supposed to be any kind of security thing. Why bother to ship it? It is redundant information which can easily be regenerated on the user's system. While it's not the be-all and end-all of security, other OS vendors (Sun in particular) have found it useful to make available a central database of MD5 checksums of known-good versions of various binaries. This has proven invaluable in not a few breakins and compromises when doing forensics. Since we have such a database essentially for free now in the form of the md5sums control files, I'd rather not take an approach that gets rid of it, even if it isn't a horribly effective security measure. Actually, we should have this information as part of the information for a Release (as asked for in #268658), alongside the Contents and Packages files. Local Md5sums can be useful to detect hardware breakage but not so much for forensic analysis (unless taken from an external trusted sourced, not the system which was compromised) BTW, NIST provides a very handy information called the National Software Reference Library (NSRL, http://www.nsrl.nist.gov/) which comes also very handy for either forensic analysis or setting up a baseline of known files (when using an integrity checking tool such as tripwire or samhain) for a large number of servers. If we provided such information they could possibly easily include it there too which would be an improvement, since they currently only carry information on ancient versions of Linux distributions (and Debian is not one of them) Regards Javier signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Stefano Zacchiroli wrote: And even in this case, I still see as not harmful proceeding to fix the packages which are not using dh_md5sums atm. I agree. One of the reason is that no one yet showed code implementing this in dpkg #155676 actually -- see shy jo signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Stefano Zacchiroli [EMAIL PROTECTED] writes: Can you please upload this to people.debian.org or somewhere, and maybe keep it periodically updated? Updated daily at http://people.debian.org/~rfrancoise/md5sums-check/ -- ,''`. : :' :Romain Francoise [EMAIL PROTECTED] `. `' http://people.debian.org/~rfrancoise/ `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Adeodato Simó [EMAIL PROTECTED] writes: Adeodato Simó [EMAIL PROTECTED] amarok-engines This is a false positive. The package only ships /usr/share/doc/amarok-engines, which is a symlink. Thanks, the script now checks that the package has at least one regular file. -- ,''`. : :' :Romain Francoise [EMAIL PROTECTED] `. `' http://people.debian.org/~rfrancoise/ `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sun, Aug 19, 2007 at 05:25:17PM +0200, Romain Francoise wrote: Updated daily at http://people.debian.org/~rfrancoise/md5sums-check/ Wonderful, thanks! Small feature request, can you please invoke dd-list passing -u ? -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Stefano Zacchiroli [EMAIL PROTECTED] writes: Small feature request, can you please invoke dd-list passing -u ? -u is the default but I don't like it much since it makes the list longer than it really is. But I've now dropped -nou on the assumption that you know better than me. :) Cheers, -- ,''`. : :' :Romain Francoise [EMAIL PROTECTED] `. `' http://people.debian.org/~rfrancoise/ `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Hi, On Sat, 2007-08-18 at 09:43:06 +1000, Anthony Towns wrote: On Fri, Aug 17, 2007 at 05:05:28PM -0500, Peter Samuelson wrote: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. [...] Where's the code for that? Changing write_filelist_except to update a new .md5 control file ought to be possible. You'd probably want to add a *newhash to struct filenamenode, though, and fill it out when unpacking, but working out the hash while unpacking (rather than running over every file to be unpacked twice) would mean hax0ring into the fd_fd_copy() invocation in tarobject() (archives.c), which seems tricky. There's a patch in 155676 doing more or less this, which adds sha1sum verification support and generation at extract time. But I agree with Joey Hess that it's better done at package creation time as it seems wasteful to do those computations on all target systems instead of the single one building the binary. The only problem with generating the checksums in dpkg-deb (if they do not yet exist) is that it might take some time to transition those packages not having them, as it needs a rebuild. Although given the current few packages lacking them, it would not seem to be the case for md5sums right now, it might be if we would add sha1sums as well for example, but then it's just a question of time until all of them get the new metadata, and binNMUs are easy if desired. regards, guillem -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, 17 Aug 2007 12:35:30 +0200, Romain Francoise [EMAIL PROTECTED] said: Manoj Srivastava [EMAIL PROTECTED] angband angband-doc c2man calc flex-old flex-old-doc libgraphics-colordeficiency-perl libgraphics-colornames-perl libgraphics-colorobject-perl libmodule-load-perl make-doc polgen polgen-doc selinux-doc sepolgen tla-tools wm-icons Feh. That just goes to show how long these packages have not been uploaded; since now ./debian/common defaults to generating the md5sum for my packages. So, the next upload for any of these packages will automatically contain a md5sum file. manoj ps: I initially thought that the most efficient way of generating the md3sums would be in dpkg; but since I have been meaning to add this into dpkg since '99 (according to my mail logs), but haven't gotten around to doing it, it pretty much means I am not gonna be the one to hack dpkg. -- Darth Vader sleeps with a Teddywookie. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sat, Aug 18, 2007 at 02:15:31AM +0300, Lars Wirzenius wrote: dpkg could do its own checksum generation only if there isn't one in the package already, or something like that. These special cases can surely be worked around. Yes, probably the right solution. And even in this case, I still see as not harmful proceeding to fix the packages which are not using dh_md5sums atm. One of the reason is that no one yet showed code implementing this in dpkg and we don't know a timeframe for this, while we know how to get it working right now with dh_md5sums. The other reasons is that once we have the support in dpkg it would be easy to change dh_md5sums to nop, and just changing a bit of code in one place will delegate the task to dpkg from there on. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
la, 2007-08-18 kello 10:16 +0200, Stefano Zacchiroli kirjoitti: One of the reason is that no one yet showed code implementing this in dpkg and we don't know a timeframe for this, while we know how to get it working right now with dh_md5sums. The other reasons is that once we have the support in dpkg it would be easy to change dh_md5sums to nop, and just changing a bit of code in one place will delegate the task to dpkg from there on. Oh, certainly, I quite agree. There's no point in waiting for dpkg changes to fix things. -- Every time I say /quit I die a little. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
[Sven Mueller] He doesn't give any information _why_ this complicates packaging Because you then have to handle removal explicitly in postrm, rather than just letting dpkg take care of it. However, I don't agree that this complicates things enough to justify doing it. Especially when you end up with lintian getting a special case just to handle your needs. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sat, Aug 18, 2007 at 03:13:32AM +0200, Sven Mueller wrote: He doesn't give any information _why_ this complicates packaging that much, while his decision imposes additional work and complexity on others (be it the exception in lintian and probably linda or the difference between dpkg -L and the contents of the md5sums file, which makes integrity checking a bit harder). IMHO, packages (.deb) should only include files which are either listed in conffiles or in md5sums. The hash files in aspell/ispell/wordlist packages (for example*: aspell-en, idutch) are neither conffiles nor in md5sums. They are said to be arch-dependend and if I understand the aspell-en debian/rules correctly, they are shipped as empty files. I don't see why they couldn't just be created empty by the postinst before building the hash tables. I especially don't see how that complicates packaging. The aspell-autobuildhash / ispell-autobuildhash manpage says create an empty .compat, or one with 0 in it. I guess most people just create the empty one. This file is then used to decide if the hash file needs to be (re-)created or not. Reading the manpage again I see: This empty file will be overwritten when the real hash is created, but will make the hash be removed at package removal without any magic being done in the postrm and will also help to keep track about which package owns that file. I guess that's the more complicated part. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
* Peter Samuelson [EMAIL PROTECTED] [070818 00:06]: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. We already claim that the md5sums file isn't supposed to be any kind of security thing. Why bother to ship it? It is redundant information which can easily be regenerated on the user's system. When it is shipped, one is sure that the information in them is what is supposed in the package. When they are only generated uppon extracting the package, all it protects are corruptions after installing. But when you have bad RAM or a faulty disk or DMA settings for your disk, corruption might more easily occour when something actually happens, i.e. when the package is installed. Apt (hopefully, I've not checked this, but all is there, so I guess it does) checks the file's md5sum uppon retrieval. I'd guess if the apt-method retrieving it is doing the check, the file written to disk will not be checked again (and if it is, it is likely that not the file is checked, but what is in the kernel's cache at that moment). Later when the package is actually installed (which might be some dpkg runs which each load its database needing quite some memory which is very likely to reclaim some memory used for caches before) the image from the disk might be used which got corrupted by some problem. There is nothing I know of that would at this point check the md5sum of the package again and even if it is read two times, one to generate md5sums of the files and one to extract them, it is likely they produce the same result due to caching, even if it is only a in memory corruption or a wrong DMA setting corrupting the read data. And given that packages that quite corrupted packages got through the whole archive infrastructure and instalations before (I remember some package that only apt-listchanges(?) had its problems with as unpacking it manualy and thus noticing the .gz CRC did not match, while nothing else did notice that). If you want to be sure, that the files you have installed have not been modified by any accident or anything else short of an explicit attacker (or anything else that updates checksums when it modifies something.), there is a very simple (and tested solution): generate them at build time. (And generating them early makes it also easier to use them in a security context, as to generate somethine like a live-CD with a trusted kernel and userspace that lists all files not having default content (though that would still quite a bit, due to all that automatic generated stuff, spool files, lock files, data, logs,.) extracting just the .md5sum files from a mirror is much easier than to generate them. And when they are created at build time, there is less chance different versions may produce different files at install time, so one might actually only store checksums of the .md5sums files to use those available on the disk after verification). Hochachtungsvoll, Bernhard R. Link -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sat, Aug 18, 2007 at 01:27:45AM +0200, Sven Mueller wrote: Kurt Roeckx schrieb: On Fri, Aug 17, 2007 at 11:25:38AM -0700, Russ Allbery wrote: Some packages (aspell and ispell packages in particular) ship files that they then modify in maintainer scripts and intentionally exclude them from the md5sums file for that reason. The hash file, which is architecture dependend, is created on install. This is the only file in the package that is architecture dependend. If it is created on install, why is it in the packages filelist in the first place? Other packages also generate (supposedly architecture dependend) files during postinst, without shipping a placeholder in the .deb - so what is the reason why [ia]spell does that? It is modified on install, not created on install, although he final effect is similar. Uhm, also: I couldn't find any such example in the [ia]spell packages aspell does not provide any dict. ispell source package does not use autobuildhash, so we have different iamerican and ibritish packages for every arch. themselves nor in wamerican, myspell-de-de, These do not create hashes of any kind. ispell-de-de so perhaps ispell-de-de (some of) those packages used to do that sort of stuff, but refrain from doing so now? -- Agustin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sat, Aug 18, 2007 at 11:05:37AM +0200, Kurt Roeckx wrote:
On Sat, Aug 18, 2007 at 03:13:32AM +0200, Sven Mueller wrote:
He doesn't give any information _why_ this complicates packaging that
much, while his decision imposes additional work and complexity on
others (be it the exception in lintian and probably linda or the
difference between dpkg -L and the contents of the md5sums file, which
makes integrity checking a bit harder).
But on the other hand lets you know which package owns that file. If you
worry about an attacker removing a line from .md5sums file, the same could
have been done from the .list file. From a security POV, is not more
difficult removing a line from the md5sums file that removing it from the
.list file, ending in both cases with a non-listed file (either in the
.list or in the .md5sums file). As a mater of fact, there are a lot of
non-listed files under /var because they are created/removed from
maintainer scripts.
I think that if dpkg is to create .md5sums or .sha1 files it should do that
for the files listed in the .md5sums file (or for all files in case it
does not exists), or should at least have a mechanism to explicitely skip
files from that if needed.
IMHO, packages (.deb) should only include files which are either listed
in conffiles or in md5sums.
The hash files in aspell/ispell/wordlist packages (for example*:
aspell-en, idutch) are neither conffiles nor in md5sums. They are said
to be arch-dependend and if I understand the aspell-en debian/rules
correctly, they are shipped as empty files. I don't see why they
couldn't just be created empty by the postinst before building the hash
tables. I especially don't see how that complicates packaging.
The aspell-autobuildhash / ispell-autobuildhash manpage says create an
empty .compat, or one with 0 in it. I guess most people just create the
empty one. This file is then used to decide if the hash file needs to be
(re-)created or not.
Reading the manpage again I see:
This
empty file will be overwritten when the real hash is created, but will
make the hash be removed at package removal without any magic being
done in the postrm and will also help to keep track about which package
owns that file.
I guess that's the more complicated part.
This only affects the .compat files and the hash (ispell or aspell) files
when using {i,a}spell-autobuildhash, that is why not all ispell dicts are
affected. For some packages (e.g., aspell-en) this means around 13 files.
Of course you can take the list and create those files from postinst and
make sure they are removed from postrm, but we found current system simpler
and robust. And, being for something under /var, not that problematic, very
few things under /var should worry about md5 sums.
--
Agustin
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sat, Aug 18, 2007 at 06:33:40PM +0200, Agustin Martin wrote:
On Sat, Aug 18, 2007 at 11:05:37AM +0200, Kurt Roeckx wrote:
The aspell-autobuildhash / ispell-autobuildhash manpage says create an
empty .compat, or one with 0 in it. I guess most people just create the
empty one. This file is then used to decide if the hash file needs to be
(re-)created or not.
Reading the manpage again I see:
This
empty file will be overwritten when the real hash is created, but
will
make the hash be removed at package removal without any magic being
done in the postrm and will also help to keep track about which
package
owns that file.
I guess that's the more complicated part.
This only affects the .compat files and the hash (ispell or aspell) files
when using {i,a}spell-autobuildhash, that is why not all ispell dicts are
affected. For some packages (e.g., aspell-en) this means around 13 files.
Of course you can take the list and create those files from postinst and
make sure they are removed from postrm, but we found current system simpler
and robust. And, being for something under /var, not that problematic, very
few things under /var should worry about md5 sums.
Forgot to mention another important thing (It is some time since the tools
were written),
With the current setup once {a,i}spell-autobuildhash is run it looks at all
the .compat files and rebuild hashes if needed, for all dicts of the given
class. This is more efficient and in case of problems a message is written
about the .compat file to blame.
If compat files are created during postinst all the current system needs
to be rewritten so *-autobuildhash is run in a per-package basis, having
the .compat file as argument. While this does not at all mean very
difficult, is also a section of the more complicated part.
--
Agustin
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
[ fully quoting my original request, for the sake of context preservation ] On Fri, Aug 17, 2007 at 09:04:13AM +0200, Luk Claes wrote: Stefano Zacchiroli wrote: [ Assuming is not too late to propose release goals of course ] Hi, a long time ago we were wondering to have DEBIAN/md5sums generated for all packages in the archive ... and we are still wondering! Can we make it a release goal for lenny? Cheers PS thanks to Romain Francoise which reminded me of this with his blog entry (http://blog.orebokech.com/2007/08/debian-packages-without-md5sums.html) With more than 600 issues, it's a bit early to make it a release goal IMHO. Though making maintainers aware by upgrading the lintian check to a warning and discussion on debian-devel about which exceptions are warranted (and possible mass bug filing) will probably be a good idea to get the amount reduced rather fast... Ok, moving the discussion to -devel then. Please reply there, people. In an attempt to prevent drift to a well-known counter argument: DEBIAN/md5sums (used by debsums) are *not* intended as a mean to counter security attacks, since they can be easily altered. Rather, they are useful as a general mechanism to check if something got corrupted due to hardware/software failures and can be used to spot which packages need to be reinstalled. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Stefano Zacchiroli [EMAIL PROTECTED] writes: [ fully quoting my original request, for the sake of context preservation ] Thanks for initiating the discussion! :-) On Fri, Aug 17, 2007 at 09:04:13AM +0200, Luk Claes wrote: With more than 600 issues, it's a bit early to make it a release goal IMHO. Though making maintainers aware by upgrading the lintian check to a warning and discussion on debian-devel about which exceptions are warranted (and possible mass bug filing) will probably be a good idea to get the amount reduced rather fast... One thing I've been pondering about is: are there any good reasons *not* to have an md5sums control file? It seems to me that the time spent to generate it on the buildds is probably insignificant compared to the total time needed to build the package... And since generating it can be done with a trivial shell command, it's not a complexity issue either. -- ,''`. : :' :Romain Francoise [EMAIL PROTECTED] `. `' http://people.debian.org/~rfrancoise/ `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 10:07:36AM +0200, Romain Francoise wrote: Thanks for initiating the discussion! :-) Well, no, thank you, it's actually you who initiated the discussion :) One thing I've been pondering about is: are there any good reasons *not* to have an md5sums control file? I fail to see any of those. I think that most of the packages without the md5sums just happen to have been packaged before dh_md5sums was available, and later on did not add its invocation to debian/rules. Similarly all packages which uses cdbs for sure have the md5sums (since cdbs invokes it). I consider it to be one of the new packaging best practice which fails to distribute back in time to old packages. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
pe, 2007-08-17 kello 10:07 +0200, Romain Francoise kirjoitti: It seems to me that the time spent to generate it on the buildds is probably insignificant compared to the total time needed to build the package... And since generating it can be done with a trivial shell command, it's not a complexity issue either. It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. This way we wouldn't have to change packages to do it, and if we ever want to change the format (sha1 as well as md5?) there's only one place to change it. -- Latest nerd movie: Once were hackers -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
pe, 2007-08-17 kello 10:58 +0200, Stefano Zacchiroli kirjoitti: I fail to see any of those. I think that most of the packages without the md5sums just happen to have been packaged before dh_md5sums was available, There's also a number of packages packaged without using debhelper. (Mine is, until I finally give up soon and re-package it with cdbs, since anything else is too much work to conform to the Python policy.) -- Policy is your friend. Trust the Policy. Love the Policy. Obey the Policy. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 12:35:30PM +0200, Romain Francoise wrote: For the record, the list of binary packages without md5sums Can you please upload this to people.debian.org or somewhere, and maybe keep it periodically updated? I guess it would be useful for the sake of deciding what to do. (give or take a few that python-debian doesn't know how to parse): Are you using the debian_bundle.debfile module for that? I would be happy to receive in a bug report about what it fails to parse. More generally I'm also interested in some feedback if you tried it on the whole archive, since we haven't yet had a lot of large use case report about it. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Stefano Zacchiroli [EMAIL PROTECTED] writes: Can you please upload this to people.debian.org or somewhere, and maybe keep it periodically updated? I guess it would be useful for the sake of deciding what to do. No problem, will do. Are you using the debian_bundle.debfile module for that? I would be happy to receive in a bug report about what it fails to parse. Yep, it was sitting in my outbox and I've just sent it: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=438486 More generally I'm also interested in some feedback if you tried it on the whole archive, since we haven't yet had a lot of large use case report about it. It's great, and surprisingly fast. It scans the mirror (sid, all sections, amd64+all) in about 25 seconds on my desktop machine. -- ,''`. : :' :Romain Francoise [EMAIL PROTECTED] `. `' http://people.debian.org/~rfrancoise/ `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Lars Wirzenius [EMAIL PROTECTED] writes: There's also a number of packages packaged without using debhelper. Yep, that's what prompted me to look into this, I recently added md5sums to rcs which doesn't use debhelper. For the record, the list of binary packages without md5sums (give or take a few that python-debian doesn't know how to parse): Dale Scheetz (Dwarf #1) [EMAIL PROTECTED] spider Adam Cécile (Le_Vert) [EMAIL PROTECTED] audacious-plugins-dev Danai SAE-HAN [EMAIL PROTECTED] cjk-latex latex-cjk-all Clint Adams [EMAIL PROTECTED] bogofilter zsh-dbg zsh-dev Cosimo Alfarano [EMAIL PROTECTED] pyg Bill Allombert [EMAIL PROTECTED] flwm Hakan Ardo [EMAIL PROTECTED] xfaces Ben Armstrong [EMAIL PROTECTED] junior-arcade junior-art junior-doc junior-games-card junior-games-gl junior-games-net junior-games-sim junior-games-text junior-gnome junior-internet junior-kde junior-math junior-programming junior-puzzle junior-sound junior-system junior-toys junior-typing junior-writing subproject-howto xpilot-client-common xpilot-client-nas xpilot-client-nosound xpilot-client-rplay xpilot-server Ernesto Nadir Crespo Avila [EMAIL PROTECTED] flowscan nitpic xwhois Alan Bain [EMAIL PROTECTED] f2c libf2c2 libf2c2-dev nec ratfor rbootd xnecview Mark Baker [EMAIL PROTECTED] inform-docs Mark Baker [EMAIL PROTECTED] chimera2 Roland Bauerschmidt [EMAIL PROTECTED] colormake Christoph Baumann [EMAIL PROTECTED] pop3browser Daniel Baumann [EMAIL PROTECTED] lib32ncurses5 lib32ncurses5-dev libncurses5 libncurses5-dbg libncurses5-dev libncursesw5 libncursesw5-dbg libncursesw5-dev ncurses-base ncurses-bin ncurses-term Edelhard Becker [EMAIL PROTECTED] vifm Christoph Berg [EMAIL PROTECTED] sdate Ed Boraas [EMAIL PROTECTED] anarchism Cyril Bouthors [EMAIL PROTECTED] gnuplot John Bovey [EMAIL PROTECTED] libnjb-doc Goswin von Brederlow [EMAIL PROTECTED] debmirror Jeff Breidenbach [EMAIL PROTECTED] mhonarc Ludovic Brenta [EMAIL PROTECTED] adacontrol Adrian Bridgett [EMAIL PROTECTED] xbill Mark Brown [EMAIL PROTECTED] nis tua Rob Browning [EMAIL PROTECTED] emacsen-common lockfile-progs stalin Thomas Bushnell, BSG [EMAIL PROTECTED] miscfiles slib Eric Van Buggenhaut [EMAIL PROTECTED] udhcpc udhcpd Bruno Barrera C. [EMAIL PROTECTED] xmame-gl Juan Cespedes [EMAIL PROTECTED] bcc bin86 genromfs Christopher Cramer [EMAIL PROTECTED] usermode Marco d'Itri [EMAIL PROTECTED] binkd br2684ctl eciadsl gup ifcico ifgate ifmail inn innfeed libberkeleydb-perl libnet-whois-ripe-perl libvolume-id-dev libvolume-id0 module-init-tools netbase ninpaths openbsd-inetd ppp-dev purity-off rblcheck tin udev update-inetd vacation whois Debconf Developers [EMAIL PROTECTED] debconf-english Debian Berkeley DB Maintainers [EMAIL PROTECTED] db4.2-util db4.3-doc db4.3-util db4.4-doc db4.4-util db4.5-doc db4.5-util db4.6-doc db4.6-util libdb-dev libdb4.2 libdb4.2++-dev libdb4.2++c2 libdb4.2-dev libdb4.2-java libdb4.2-java-dev libdb4.2-tcl libdb4.3 libdb4.3++-dev libdb4.3++c2 libdb4.3-dev libdb4.3-java libdb4.3-java-dev libdb4.3-tcl libdb4.4 libdb4.4++ libdb4.4++-dev libdb4.4-dev libdb4.4-java libdb4.4-java-dev libdb4.4-tcl libdb4.5 libdb4.5++ libdb4.5++-dev libdb4.5-dev libdb4.5-java libdb4.5-java-dev libdb4.5-java-gcj libdb4.5-tcl libdb4.6 libdb4.6++ libdb4.6++-dev libdb4.6-dbg libdb4.6-java libdb4.6-java-dev libdb4.6-java-gcj libdb4.6-tcl Debian Edu Developers [EMAIL PROTECTED] debian-edu-archive-keyring Debian Games Team [EMAIL PROTECTED] xtux Debian GCC Maintainers [EMAIL PROTECTED] g++ g++-multilib g77 gcc-multilib gcj gfortran gfortran-multilib gij gnat gobjc gobjc++ gobjc++-4.1-multilib gobjc++-4.2-multilib gobjc++-multilib gobjc-multilib gpc gpc-doc Debian GIS Project [EMAIL PROTECTED] hdf4-tools libhdf4g libhdf4g-dev libhdf4g-doc Debian Install System Team [EMAIL PROTECTED] cdebconf debian-installer installation-guide-alpha installation-guide-amd64 installation-guide-arm installation-guide-hppa installation-guide-i386 installation-guide-ia64 installation-guide-mips installation-guide-mipsel installation-guide-powerpc installation-guide-s390 installation-guide-sparc installation-report libdebconfclient0 libdebconfclient0-dev os-prober user-setup Debian Kernel Team [EMAIL PROTECTED] ipw3945d Debian Mono Group [EMAIL PROTECTED] mono Debian multimedia packages maintainers [EMAIL PROTECTED] vlc-plugin-alsa wxvlc Debian Octave
Re: proposed release goal: DEBIAN/md5sums for all packages
* Romain Francoise [Fri, 17 Aug 2007 12:35:30 +0200]: Adeodato Simó [EMAIL PROTECTED] amarok-engines This is a false positive. The package only ships /usr/share/doc/amarok-engines, which is a symlink. Cheers, -- Adeodato Simó dato at net.com.org.es Debian Developer adeodato at debian.org If you want this to happen, do not use the word bottom with me again, Kirk. -- Luke Danes -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, 17 Aug 2007 12:35:30 +0200, Romain Francoise wrote: Debian Perl Group [EMAIL PROTECTED] libchemistry-elements-perl libdbd-odbc-perl libdigest-hmac-perl libmath-combinatorics-perl libmath-derivative-perl libmath-numbercruncher-perl libmath-spline-perl libole-storage-lite-perl libstar-parser-perl libtk-gbarr-perl libtk-histentry-perl libtk-splashscreen-perl Thanks for the list, I've fixed them all in svn and Damyan has started to review and upload them. Cheers, gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `-NP: Rigmor Gustafsson: The More I See signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Lars Wirzenius [EMAIL PROTECTED] writes: pe, 2007-08-17 kello 10:07 +0200, Romain Francoise kirjoitti: It seems to me that the time spent to generate it on the buildds is probably insignificant compared to the total time needed to build the package... And since generating it can be done with a trivial shell command, it's not a complexity issue either. It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. This way we wouldn't have to change packages to do it, and if we ever want to change the format (sha1 as well as md5?) there's only one place to change it. Yes, that sounds like a good idea. It might also be interesting to not put those into the control.tar.gz, but directly into the deb, so that it can easily be extracted. Marc -- BOFH #73: Daemons did it pgpbxQEAtKlMa.pgp Description: PGP signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Stefano Zacchiroli, 2007-08-17 12:43:55 +0200 : On Fri, Aug 17, 2007 at 12:35:30PM +0200, Romain Francoise wrote: For the record, the list of binary packages without md5sums Can you please upload this to people.debian.org or somewhere, and maybe keep it periodically updated? I guess it would be useful for the sake of deciding what to do. Maybe add a lintian/linda test? Maybe add that to Lina (http://asdfasdf.debian.net/~tar/lina/)? Roland. -- Roland Mas Fate always wins... At least, when people stick to the rules. -- in Interesting Times (Terry Pratchett) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 01:58:14PM +0200, Marc 'HE' Brockschmidt wrote: Yes, that sounds like a good idea. Agreed. But needs someone willing to patch dpkg for that: volunteers? It might also be interesting to not put those into the control.tar.gz, but directly into the deb, so that it can easily be extracted. I don't see the point here. dpkg-deb would probably be responsible of creating the md5sums and of extracting them on demand. Hence, having them inside control.tar.gz or outside it is just an implementation detail of dpkg-deb. The only thing you gain having them outside control.tar.gz is that you can extract such information via plain ar ... what's for? Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 12:56:14PM +0200, Romain Francoise wrote: I would be happy to receive in a bug report about what it fails to parse. Yep, it was sitting in my outbox and I've just sent it: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=438486 Thanks, this is now fixed in the python-debian repository. In the meantime you can grab the new debfile.py from http://people.debian.org/~zack/stalla/debfile.py if you want to re-run your tests to see what happen with that 34 packages (SPAM: or you can debcheckout python-debian of course, SCNR :-)) It's great, and surprisingly fast. It scans the mirror (sid, all sections, amd64+all) in about 25 seconds on my desktop machine. Wow, I'm impressed by those numbers as well! Thanks for your feedback. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 11:58:14AM +, Marc 'HE' Brockschmidt wrote: Lars Wirzenius [EMAIL PROTECTED] writes: pe, 2007-08-17 kello 10:07 +0200, Romain Francoise kirjoitti: It seems to me that the time spent to generate it on the buildds is probably insignificant compared to the total time needed to build the package... And since generating it can be done with a trivial shell command, it's not a complexity issue either. It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. This way we wouldn't have to change packages to do it, and if we ever want to change the format (sha1 as well as md5?) there's only one place to change it. Yes, that sounds like a good idea. It might also be interesting to not put those into the control.tar.gz, but directly into the deb, so that it can easily be extracted. OTOH that sucks because it would mean that we have to rebuild the whole archive that uses currently dh_md5sums, whereas we could just be backward compatible. Other issue, md5sums files are used in /var/lib/dpkg/info/ where dpkg puts the control.tar.gz's, and packages like debsums use it there. So we would have to patch dpkg to also extract files that are in the .deb into that directory as well, whereas it's already done for files in control.tar.gz automatically. All in all, I'd say it's a pretty bad idea for a minimal gain (control.tar.gz content is very small after all). But I agree that dpkg(-deb) is the place to plug this. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpwpvvjRGNjI.pgp Description: PGP signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Romain Francoise wrote: Daniel Baumann [EMAIL PROTECTED] lib32ncurses5 lib32ncurses5-dev libncurses5 libncurses5-dbg libncurses5-dev libncursesw5 libncursesw5-dbg libncursesw5-dev ncurses-base ncurses-bin ncurses-term fixed, thanks. -- Address:Daniel Baumann, Burgunderstrasse 3, CH-4562 Biberist Email: [EMAIL PROTECTED] Internet: http://people.panthera-systems.net/~daniel-baumann/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Roland Mas [EMAIL PROTECTED] writes: Maybe add a lintian/linda test? Maybe add that to Lina (http://asdfasdf.debian.net/~tar/lina/)? There's already a lintian test. It's just only info-level because last time I had checked there wasn't project consensus that md5sums should be required. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Kurt Roeckx [EMAIL PROTECTED] writes: On Fri, Aug 17, 2007 at 10:12:07AM -0700, Russ Allbery wrote: Lars Wirzenius [EMAIL PROTECTED] writes: It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. This way we wouldn't have to change packages to do it, and if we ever want to change the format (sha1 as well as md5?) there's only one place to change it. Some packages (aspell and ispell packages in particular) ship files that they then modify in maintainer scripts and intentionally exclude them from the md5sums file for that reason. lintian has special code to deal with this case. A replacement in dpkg would either need to cope with this or decide that those packages can no longer take that approach and have to solve whatever problem they're solving in some other way. (I don't know what problem this is solving.) As I understand it, it's so that debsums doesn't complain about that files changes. Right... I meant more what problem they had that they were solving by installing a file in /var as part of the package and then modifying it later. (It always struck me as odd, but I presume they have good reasons for doing things that way.) -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 10:12:07AM -0700, Russ Allbery wrote: Lars Wirzenius [EMAIL PROTECTED] writes: It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. This way we wouldn't have to change packages to do it, and if we ever want to change the format (sha1 as well as md5?) there's only one place to change it. Some packages (aspell and ispell packages in particular) ship files that they then modify in maintainer scripts and intentionally exclude them from the md5sums file for that reason. lintian has special code to deal with this case. A replacement in dpkg would either need to cope with this or decide that those packages can no longer take that approach and have to solve whatever problem they're solving in some other way. (I don't know what problem this is solving.) As I understand it, it's so that debsums doesn't complain about that files changes. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 11:25:38AM -0700, Russ Allbery wrote: Some packages (aspell and ispell packages in particular) ship files that they then modify in maintainer scripts and intentionally exclude them from the md5sums file for that reason. lintian has special code to deal with this case. A replacement in dpkg would either need to cope with this or decide that those packages can no longer take that approach and have to solve whatever problem they're solving in some other way. (I don't know what problem this is solving.) As I understand it, it's so that debsums doesn't complain about that files changes. Right... I meant more what problem they had that they were solving by installing a file in /var as part of the package and then modifying it later. (It always struck me as odd, but I presume they have good reasons for doing things that way.) The hash file, which is architecture dependend, is created on install. This is the only file in the package that is architecture dependend. Anyway, there is a policy file about this in: /usr/share/doc/dictionaries-common-dev/dsdt-policy.txt.gz Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
[Lars Wirzenius] It strikes me that if we want to make it policy, having dpkg generate the checksums upon creating the .deb would be the simplest and best way to do it. I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. We already claim that the md5sums file isn't supposed to be any kind of security thing. Why bother to ship it? It is redundant information which can easily be regenerated on the user's system. Russ's mention of packages that alter their installed files in the postinst just seems not worth supporting. Copy a template file instead. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Kurt Roeckx schrieb: On Fri, Aug 17, 2007 at 11:25:38AM -0700, Russ Allbery wrote: Some packages (aspell and ispell packages in particular) ship files that they then modify in maintainer scripts and intentionally exclude them from the md5sums file for that reason. The hash file, which is architecture dependend, is created on install. This is the only file in the package that is architecture dependend. If it is created on install, why is it in the packages filelist in the first place? Other packages also generate (supposedly architecture dependend) files during postinst, without shipping a placeholder in the .deb - so what is the reason why [ia]spell does that? Uhm, also: I couldn't find any such example in the [ia]spell packages themselves nor in wamerican, myspell-de-de, ispell-de-de so perhaps (some of) those packages used to do that sort of stuff, but refrain from doing so now? Regards, Sven signature.asc Description: OpenPGP digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
pe, 2007-08-17 kello 17:05 -0500, Peter Samuelson kirjoitti: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. We already claim that the md5sums file isn't supposed to be any kind of security thing. Why bother to ship it? It is redundant information which can easily be regenerated on the user's system. That would work too, I guess. It would give more flexibility in the choice of checksum, too. Still, it'd be dpkg doing it, not each package separately. -- When in doubt, use brute force. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
pe, 2007-08-17 kello 10:12 -0700, Russ Allbery kirjoitti: Some packages (aspell and ispell packages in particular) ship files that they then modify in maintainer scripts and intentionally exclude them from the md5sums file for that reason. lintian has special code to deal with this case. A replacement in dpkg would either need to cope with this or decide that those packages can no longer take that approach and have to solve whatever problem they're solving in some other way. (I don't know what problem this is solving.) dpkg could do its own checksum generation only if there isn't one in the package already, or something like that. These special cases can surely be worked around. -- The difference between appealing and appalling is very small. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 05:05:28PM -0500, Peter Samuelson wrote: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. [...] Where's the code for that? Changing write_filelist_except to update a new .md5 control file ought to be possible. You'd probably want to add a *newhash to struct filenamenode, though, and fill it out when unpacking, but working out the hash while unpacking (rather than running over every file to be unpacked twice) would mean hax0ring into the fd_fd_copy() invocation in tarobject() (archives.c), which seems tricky. Cheers, aj signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Sven Mueller [EMAIL PROTECTED] writes: If it is created on install, why is it in the packages filelist in the first place? Other packages also generate (supposedly architecture dependend) files during postinst, without shipping a placeholder in the .deb - so what is the reason why [ia]spell does that? Uhm, also: I couldn't find any such example in the [ia]spell packages themselves nor in wamerican, myspell-de-de, ispell-de-de so perhaps (some of) those packages used to do that sort of stuff, but refrain from doing so now? All I know about this topic is at: http://lists.debian.org/debian-mentors/2006/10/msg00075.html http://bugs.debian.org/324111 http://bugs.debian.org/346410 http://bugs.debian.org/374949 http://bugs.debian.org/401070 I'm happy to remove the exception again if this has changed. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Peter Samuelson [EMAIL PROTECTED] writes: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. We already claim that the md5sums file isn't supposed to be any kind of security thing. Why bother to ship it? It is redundant information which can easily be regenerated on the user's system. While it's not the be-all and end-all of security, other OS vendors (Sun in particular) have found it useful to make available a central database of MD5 checksums of known-good versions of various binaries. This has proven invaluable in not a few breakins and compromises when doing forensics. Since we have such a database essentially for free now in the form of the md5sums control files, I'd rather not take an approach that gets rid of it, even if it isn't a horribly effective security measure. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
[Russ Allbery] While it's not the be-all and end-all of security, other OS vendors (Sun in particular) have found it useful to make available a central database of MD5 checksums of known-good versions of various binaries. H. As far as being authoritative (and cryptographically secure), we already have $MIRROR/dists/stable/main/binary-i386/Packages.bz2. The thing is, if you're checking your system, you have to have something to check it against. If this is the md5sums file in /var/lib/dpkg/info, it doesn't matter whether it's included in the package. But if you're using the copy from the .deb (because, say, you don't trust your /var), it wouldn't be much harder to do 'dpkg-deb --extract' and then md5sum the extracted directory, than to do 'dpkg-deb --control' and read DEBIAN/md5sums. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
On Sat, Aug 18, 2007 at 01:27:45AM +0200, Sven Mueller wrote: The hash file, which is architecture dependend, is created on install. This is the only file in the package that is architecture dependend. If it is created on install, why is it in the packages filelist in the first place? Other packages also generate (supposedly architecture dependend) files during postinst, without shipping a placeholder in the .deb - so what is the reason why [ia]spell does that? I wasn't at all involved with writing this policy. So, I'll have to guess. Uhm, also: I couldn't find any such example in the [ia]spell packages themselves nor in wamerican, myspell-de-de, ispell-de-de so perhaps (some of) those packages used to do that sort of stuff, but refrain from doing so now? Policy doesn't say you have to do this. It's just one of the options. examples are things like: aspell-en ipolish aspell-pl idutch apsell-nl Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
Peter Samuelson wrote: The thing is, if you're checking your system, you have to have something to check it against. If this is the md5sums file in /var/lib/dpkg/info, it doesn't matter whether it's included in the package. But if you're using the copy from the .deb (because, say, you don't trust your /var), it wouldn't be much harder to do 'dpkg-deb --extract' and then md5sum the extracted directory, than to do 'dpkg-deb --control' and read DEBIAN/md5sums. It's even easier to do: dpkg --fsys-tarfile $deb | tar -C / -d However, not all machines have the luxury of being able to store the orignal .debs in /var, or of being able to redownload the same debs. -- see shy jo signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Peter Samuelson wrote: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. Not all debian systems have fast CPU and fast disk. -- see shy jo signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
On Fri, Aug 17, 2007 at 08:23:38PM -0400, Joey Hess wrote: I'd opt for dpkg generating the checksums upon _extracting_ the .deb file. Not all debian systems have fast CPU and fast disk. I could understand the fast CPU argument, but there's no good reason why MD5ing at extraction time wouldn't be feasible, costing you zero extra disk activity. If nothing else, the data is likely to be in the cache if you do extract, md5, extract, md5 for each file or even for each package (depending on the amount of RAM). /* Steinar */ -- Homepage: http://www.sesse.net/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: proposed release goal: DEBIAN/md5sums for all packages
[Joey Hess] It's even easier to do: dpkg --fsys-tarfile $deb | tar -C / -d Ha. I didn't know about tar -d. Yes, that is even better. However, not all machines have the luxury of being able to store the orignal .debs in /var, or of being able to redownload the same debs. Indeed, but that is unrelated to the discussion of storing a md5sums file in the .deb. (: -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ signature.asc Description: Digital signature
Re: proposed release goal: DEBIAN/md5sums for all packages
Russ Allbery schrieb: Sven Mueller [EMAIL PROTECTED] writes: If it is created on install, why is it in the packages filelist in the first place? Other packages also generate (supposedly architecture dependend) files during postinst, without shipping a placeholder in the .deb - so what is the reason why [ia]spell does that? Uhm, also: I couldn't find any such example in the [ia]spell packages themselves nor in wamerican, myspell-de-de, ispell-de-de so perhaps (some of) those packages used to do that sort of stuff, but refrain from doing so now? All I know about this topic is at: http://lists.debian.org/debian-mentors/2006/10/msg00075.html http://bugs.debian.org/324111 http://bugs.debian.org/346410 http://bugs.debian.org/374949 http://bugs.debian.org/401070 I'm happy to remove the exception again if this has changed. In all those mails, the only justification for shipping these files in the package - though they are changed (rebuilt?) during postinst - is the following sentence from Brian Nelson (pyro): Also, not including these files in the .deb packages significantly complicates the packaging. I really don't want to change to manangement of the files in maintainer scripts without a very good reason to do so. He doesn't give any information _why_ this complicates packaging that much, while his decision imposes additional work and complexity on others (be it the exception in lintian and probably linda or the difference between dpkg -L and the contents of the md5sums file, which makes integrity checking a bit harder). IMHO, packages (.deb) should only include files which are either listed in conffiles or in md5sums. The hash files in aspell/ispell/wordlist packages (for example*: aspell-en, idutch) are neither conffiles nor in md5sums. They are said to be arch-dependend and if I understand the aspell-en debian/rules correctly, they are shipped as empty files. I don't see why they couldn't just be created empty by the postinst before building the hash tables. I especially don't see how that complicates packaging. cu, Sven * Thanks to Kurt Roeckx for the examples PS: I just verified that the files in question are indeed zero-length files at least in aspell-en -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
