Re: md5sums files

[Anthony Towns]

On Sun, Mar 7, 2010 at 10:28, Goswin von Brederlow  wrote:
> Anthony Towns  writes:
>> Advantages of doing it when uploading:
>>  - provides some sort of double check of what's being uploaded
>>  - saves CPU time on users' machines
>   - avoids having bad checksums due to the user having bad hardware
>     (which is one big use case of the files)

"Big"? It only makes a difference if:
  a) the corruption happens as soon as it's written, not after some time
  b) the file is too big/the system is too loaded to keep the file in
the page cache
  c) the system memory is corrupted just enough to screw the file but
not everything else

Compared to random "make install" invocations changing files in the
system and similar, that doesn't strike me as a big use case.

In any event, it's fairly easy to generate the checksum in the same
pass as generating the file, see the attached patch. (It's not as easy
to generalise to other hashes as the previous one, unfortunately)

If you're still worried, perhaps about having read() return bogus data
from the .deb that happens to still be valid when passed through
ungzip and untar and after you've already verified the entire file by
md5/sha1/sha256 when downloading, you're getting to the point of
trying to safely install on an actively malicious system, and
nothing's going to make that work.

Cheers,
aj

-- 
Anthony Towns 
diff -x configure -x '*.m4' -x Makefile.in -urbN dpkg-1.15.5.6/configure.ac dpkg-1.15.5.6-aj/configure.ac
--- dpkg-1.15.5.6/configure.ac	2010-01-08 18:00:34.0 +1000
+++ dpkg-1.15.5.6-aj/configure.ac	2010-03-07 04:38:32.547372468 +1000
@@ -51,6 +51,16 @@
 esac])
 AC_SUBST(admindir)
 
+# Allow alternative default hash function
+hashtype="md5"
+AC_ARG_WITH(hashtype,
+	AS_HELP_STRING([--with-hashtype=HASH],
+	   [hash function to use for .hashes files]),
+[case "$with_hashtype" in
+  "md5"|"none") hashtype="$with_hashtype" ;;
+  *) AC_MSG_ERROR([invalid hashtype specified]) ;;
+esac])
+AC_SUBST(hashtype)
 
 # Checks for programs.
 AC_PROG_CC
diff -x configure -x '*.m4' -x Makefile.in -urbN dpkg-1.15.5.6/debian/changelog dpkg-1.15.5.6-aj/debian/changelog
--- dpkg-1.15.5.6/debian/changelog	2010-01-09 04:02:03.0 +1000
+++ dpkg-1.15.5.6-aj/debian/changelog	2010-03-07 04:13:03.171356041 +1000
@@ -1,3 +1,11 @@
+dpkg (1.15.5.6+aj) unstable; urgency=low
+
+  * Non-maintainer upload.
+  * Automatically create /var/lib/dpkg/info/pkg.hashes containing MD5 hashes
+for unpacked files.
+
+ -- Anthony Towns   Sun, 07 Mar 2010 04:12:32 +1000
+
 dpkg (1.15.5.6) unstable; urgency=low
 
   * dpkg-source: with format "3.0 (quilt)" ensure quilt's .pc directory is
diff -x configure -x '*.m4' -x Makefile.in -urbN dpkg-1.15.5.6/lib/dpkg/buffer.c dpkg-1.15.5.6-aj/lib/dpkg/buffer.c
--- dpkg-1.15.5.6/lib/dpkg/buffer.c	2010-01-09 03:23:06.0 +1000
+++ dpkg-1.15.5.6-aj/lib/dpkg/buffer.c	2010-03-07 15:50:33.379710844 +1000
@@ -60,6 +60,13 @@
 	case BUFFER_WRITE_MD5:
 		buffer_md5_init(write_data);
 		break;
+	case BUFFER_WRITE_DUP:
+		{
+		  struct buffer_data *bddup = write_data->arg.ptr;
+		  buffer_init(NULL, &bddup[0]);
+		  buffer_init(NULL, &bddup[1]);
+		}
+		break;
 	}
 	return 0;
 }
@@ -90,6 +97,13 @@
 	case BUFFER_WRITE_MD5:
 		buffer_md5_done(write_data);
 		break;
+	case BUFFER_WRITE_DUP:
+		{
+		  struct buffer_data *bddup = write_data->arg.ptr;
+		  buffer_done(NULL, &bddup[0]);
+		  buffer_done(NULL, &bddup[1]);
+		}
+		break;
 	}
 	return 0;
 }
@@ -126,6 +140,14 @@
 	case BUFFER_WRITE_MD5:
 		MD5Update(&(((struct buffer_write_md5ctx *)data->arg.ptr)->ctx), buf, length);
 		break;
+	case BUFFER_WRITE_DUP:
+		{
+		  struct buffer_data *bddup = data->arg.ptr;
+  ret = buffer_write(&bddup[0], buf, length, desc);
+		  if (ret != length) return ret;
+  ret = buffer_write(&bddup[1], buf, length, desc);
+		}
+		break;
 	default:
 		internerr("unknown data type '%i' in buffer_write",
 		  data->type);
diff -x configure -x '*.m4' -x Makefile.in -urbN dpkg-1.15.5.6/lib/dpkg/buffer.h dpkg-1.15.5.6-aj/lib/dpkg/buffer.h
--- dpkg-1.15.5.6/lib/dpkg/buffer.h	2010-01-08 18:00:34.0 +1000
+++ dpkg-1.15.5.6-aj/lib/dpkg/buffer.h	2010-03-07 15:53:23.319707965 +1000
@@ -36,6 +36,7 @@
 #define BUFFER_WRITE_NULL		3
 #define BUFFER_WRITE_STREAM		4
 #define BUFFER_WRITE_MD5		5
+#define BUFFER_WRITE_DUP		6
 
 #define BUFFER_READ_FD			0
 #define BUFFER_READ_STREAM		1
@@ -52,6 +53,14 @@
 	buffer_hash(buf, hash, BUFFER_WRITE_MD5, limit)
 
 #if HAVE_C99
+# define fd_fdmd5(fd1, fd2, hash, limit, ...) \
+	do { \
+	struct buffer_data fdhash[2]; \
+fdhash[0].arg.i = fd2; fdhash[0].type = BUFFER_WRITE_FD; \
+fdhash[1].arg.ptr = hash; fdhash[1].type = BUFFER_WRITE_MD5; \
+buffer_copy_IntPtr(fd1, BUFFER_READ_FD, fdhash, BUFFER_WRITE_DUP, \
+	   limit, __VA_ARGS__); \
+} while(0)
 # define fd_md5(fd, hash, limit, ...) \
 	buffer_copy_IntPtr(fd, BUFFER_READ_

Team uploads.

[Charles Plessy]

Dear all,

It was proposed in 2009 to formalise "Team uploads" in analogy to the "QA
uploads", as a special case of NMU, where most conventions are relaxed.

http://lists.debian.org/e13a36b30904052052g73850787vcc8b2035640d7...@mail.gmail.com

While there was interest, the discussion eventually ended with no action.

In the context of the Debian Med packaging team, I have started to make “Team
uploads”, for packages that are maintained by my team but for which I do not
want to add myself in the Uploaders field.

For these NMUs, I apply the following exceptions:

 - Normal incementation of version number,
 - no delay,
 - no patch to the BTS, but a direct commit to our repository,
 - start the changelog by a “Team upload.” entry.

This triggers Lintian warnings, that I ignore. The attached untested patch
may solve the problem.

If others are interested to follow the same approach, I propose to use the 
following
wiki page to draft a common reference: http://wiki.debian.org/TeamUpload.

Have a nice sunday,

-- 
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
diff --git a/checks/nmu b/checks/nmu
index fd2febf..80a5eb5 100644
--- a/checks/nmu
+++ b/checks/nmu
@@ -42,6 +42,7 @@ my $info = shift;
 my $changelog_mentions_nmu = 0;
 my $changelog_mentions_local = 0;
 my $changelog_mentions_qa = 0;
+my $changelog_mentions_team_upload = 0;
 
 # This isn't really an NMU check, but right now no other check looks at
 # debian/changelog in source packages.  Catch a debian/changelog file that's a
@@ -60,7 +61,7 @@ my $changes = $entry->Changes;
 $changes =~ s/^(\s*\n)+//;
 my $firstline = (split('\n', $changes))[0];
 
-# Check the first line for QA and NMU mentions.
+# Check the first line for QA, NMU or team upload mentions.
 if ($firstline) {
 	local $_ = $firstline;
 	if (/\bnmu\b/i or /non-maintainer upload/i) {
@@ -70,6 +71,8 @@ if ($firstline) {
 	}
 	$changelog_mentions_local = 1 if /\blocal\s+package\b/i;
 	$changelog_mentions_qa = 1 if /orphan/i or /qa (?:group )?upload/i;
+	$changelog_mentions_team_upload = 1 if /* Team upload/i;
+
 }
 
 my $version = $info->field("version");
@@ -111,6 +114,9 @@ if ($maintainer =~ /packag...@qa.debian.org/) {
 		if $version_nmuness == 1;
 	tag "changelog-should-mention-qa", ""
 		if !$changelog_mentions_qa;
+} elsif ($changelog_mentions_team_upload) {
+	tag "team-upload-has-incorrect-version-number", "$version"
+		if $version_nmuness == 1;
 } else {
 	# Local packages may be either NMUs or not.
 	unless ($changelog_mentions_local || $version_local) {

Bug#572853: ITP: thunarx-python -- Python bindings for the Thunar file manager

[Jason Heeris]

Package: wnpp
Severity: wishlist
X-Debbugs-CC: debian-devel@lists.debian.org

   Package name: thunarx-python
Version: 0.2.0
Upstream Author: Adam Plumb 
URL: http://github.com/adamplumb/thunarx-python
License: GPL3+
Description:  Python bindings for the Thunar file manager, allowing
property page and menu extensions to be written in Python.



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4b9325b5.5070...@gmail.com

Re: including full package source code in the debian release

[Michael Gilbert]

On Sat, 06 Mar 2010 19:29:22 -0800 Jamie Morken wrote:
> so including compressed package source code would have a very minor impact on 
> the overall file size of the debian release.

you can achieve your goal by burning the isos and having them on hand.
or you can create less physical waste by loop mounting the isos.

mike


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20100306225243.5b36de6c.michael.s.gilb...@gmail.com

Re: including full package source code in the debian release

[Russ Allbery]

Jamie Morken  writes:

> Debian releases have 25,000 or so packages and don't include the source to
> them,

Huh?  We most certainly do.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87lje4n688@windlord.stanford.edu

including full package source code in the debian release

[Jamie Morken]

Hi,

Debian releases have 25,000 or so packages and don't include the source to 
them, so to recompile the package you use apt-get to connect to the internet 
and download the source to one package at a time if you want the source code.  
I did some calculations to see how much bulk adding the package source code to 
the debian release would add and using high compression it appears that it 
would increase the overall release size by less than 1%.  Most users won't care 
or benefit from having the source code to these 25,000 packages included in the 
distribution, but some will probably like the ability to be able to have this 
source code and be able to recompile packages without requiring internet access 
to download package source code.

Here are my calculations, they are rough estimates:

packages included in Debian 4.0 etch (283 million lines of code)
source: http://en.wikipedia.org/wiki/Debian

assuming 32KB per 1000 lines of code, this would be about 8.6GB for 283 million 
lines of code.

assuming a factor of 100 for the compression of this code, this would give 
approx 88MB of extra space required in the debian distribution for all of this 
source code.

the total size of the current full Debian distribution is over 18GB:
http://cdimage.debian.org/debian-cd/5.0.4/i386/iso-dvd/

so including compressed package source code would have a very minor impact on 
the overall file size of the debian release.

cheers,
Jamie

Re: md5sums files

[Peter Samuelson]

> Peter Samuelson  writes:
> > Be that as it may, I don't think the md5sums file was ever intended to
> > be an integrity check of the .deb itself.  Fortunately, the .deb also
> > includes checksums of control.tar.gz and data.tar.gz, thanks to use of
> > the gzip container format.

[Goswin von Brederlow]
> That is not about the integrity of the deb. It is about the integrity of
> the files on the system. And if you do have faulty memory (or any of the
> other problems) then calculating the checksum locally will have a high
> risk of calculating it from already corrupted data and miss the error.

How many times do I have to say "the .deb also includes checksums of
control.tar.gz and data.tar.gz, thanks to use of the gzip container
format" before you notice?
-- 
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100307032816.gm18...@p12n.org

Re: Bug#572733: support for mounting other kernel filesystems

[Marco d'Itri]

On Mar 06, Petter Reinholdtsen  wrote:

> > - cgroups (needed for accounting and management of system resources)
> I thought the libcgroup package handled this one, but might be wrong.
I understand that the cgconfig program from cgroup-bin does mount it
(in /mnt/cgroups/cpu, which I believe to be totally broken since /mnt
is not supposed to have subdirectories), but I do not think everybody
needs or wants to use cgconfig.

> In my opinion initscripts should limit itself to mount special file
> system that should be available on all or almost all installations and
> that do not introduce a security problem when mounted, and leave it to
> other packages to handle mounting of less common file system types.
Write access to cgroups and hugetlbfs is restricted to root by default.

> The request for mounting debugfs by default has so far been denied, as
> it have a rather special use case and can be seen as a security
> problem.
debugfs also already has a /sys/kernel/debug/ so it can easily be dealt
with in fstab. Since AFAIK it is not used by regular programs I see no
harm in having to configure it manually.

> For file system that should be mounted on Linux, depending on udev
> mith be ok, but for file systems that also should be mounted on
> kFreeBSD, I am told udev do not work and using udev is not an option.
> I suspect cgroups and hugetlbfs are linux specific, but wanted to
> mention the issue to be sure it is known.
Obviously these are Linux-specific filesystems. The porters to other
kernels will deal with their own system-specific filesystems...

> On Linux, using udev is supposed to be optional, but it is getter
> harder and harder to to avoid it, so I am not sure we want to spend
> extra effort to make it easier to drop udev from an installation.
udev is not intended to be optional, even if some systems currently work
well enough without udev nobody should rely on this to be true in the
future as well.

> I have no opinion on the mount point location, but would like all
> distributions to agree on the same location to make it easier for
> users and application writers to move applications from OS to OS.
Good luck. Obviously this will not happen in time for squeeze, so we
need a solution anyway.
The cgroups upstream maintainers stated that they are not interested in
mandating any specific location.

> > The upstream developers do not take a position either, but
> > /dev/cgroup/ and /dev/cgroups/ are popular choices.
> What are the other choices?  What do Ubuntu, SuSe, Mandriva, Gentoo
> and the other distributions do?  Which mount point is most commonly
> used?
I do not think other distributions have infrastructure to mount this
filesystem except for what is provided by cgconfig.
The upstream developers stated that they do not want to recommend any
specific directory, I think the only common alternative I have seen is
/cgroups/.

> Do not know.  For me it depend on how many installations should have
> these file systems mounted.  If it is all or almost all -> init.d
> scripts in the initscripts package and if it is some instalations ->
> sysadmin can add it himself or separate pacage.
The sysadmin currently CANNOT mount these filesystems by itself below
/dev because there is no mount point at boot time.
This may be the best solution, but it still needs some work in a
standard package.


On Mar 06, Mike Hommey  wrote:

> /dev/something just feels so wrong. /dev contains block and character
> devices, and almost nothing else (except some udev and initramfs files)
> Why should cgroups control files, which are hardly device files, be
> found under /dev ?
Why not? /dev/pts/ is a kernel filesystem mounted below /dev, and /dev
itself is a kernel filesystem (tmpfs or devtmpfs).
And /dev will still be much less controversial than mount points in / .

-- 
ciao,
Marco


signature.asc
Description: Digital signature

Re: md5sums files

[Goswin von Brederlow]

Anthony Towns  writes:

> (I'm not subscribed to this list, so go ahead and Cc me)
>
> On Thu, Mar 4, 2010 at 02:05, Peter Samuelson  wrote:
>> [Wouter Verhelst]
>> > I must say I was somewhat surprised by these numbers. Out of 2483
>> > packages installed on my laptop, 2340 install md5sums.
>> The surprising part, perhaps, is that dpkg itself didn't just generate
>> the other 143 md5sums files at installation time.
>
> The easy (and usually correct) reason for things like that is "dpkg's
> source is scary".
>
>> I suggested this a long time ago and of course was met with "so where's
>> your patch?"  Of course I was not willing to do the work.
>
> See? Anyway, my patch is attached. It makes dpkg create a "foo.hashes"
> when unpacking foo, whose contents looks like:
>
> MD5:32b5e22f8e336b2f34e0dd87652e6dfc  usr/share/doc/mawk/changelog.gz
> MD5:87a34f1f55ac3f7fec2c7fc82565e8eb  usr/share/doc/mawk/changelog.Debian.gz
> ...
>
> Verification is a matter of something like:
>
> $ cat /var/lib/dpkg/info/*.hashes | sed -n 's/^MD5://p' | (cd /;
> md5sum -c) | grep -v ': OK$'
>
> There's an option (--hash) that you can set to "none" to avoid
> spending time calculating md5s if you so choose. Adding support for
> sha1/sha256/whatever should be straightforward; afaik dpkg only has
> code for md5 already built in though (though just invoking
> /usr/bin/sha1sum etc would be an option of course).
>
> Of course another option is just to pull the md5sums directly from the deb:
>
> $ ar p /var/cache/apt/archives/ifupdown_0.6.9_i386.deb data.tar.gz |
> tar --to-command='printf "%s%s\n" "$(md5sum - | sed s/-$//)"
> "${TAR_FILENAME#./}"'  -xzf - |
> diff - /var/lib/dpkg/info/ifupdown.md5sums
> 1,3d0
> < 346208729633adf45e2fa3f2bd3b19c6  etc/init.d/ifupdown
> < c6fffaae03271f1641920105ce68796b  etc/init.d/ifupdown-clean
> < fab851ca87c5deb9d6f665e610184648  etc/default/ifupdown
> 4a2
>> a0f11cf1809a468c49b72e0aa0a8e26b  sbin/ifup
>
> (md5sums doesn't normally list conffiles, but does list hardlinks; the
> above command does the opposite)
>
>> But
>> fundamentally, shipping a md5sums file is really just a tradeoff in
>> download size vs. installation speed, not unlike gzip vs. bzip2.
>
> Advantages of doing in when unpacking:
>  - choice of checksum is the admin's decision
>  - we can quickly roll out support for sha1/sha256/crc/... checksums
> by just changing one package
>  - admin has hashes of exactly what was unpacked, no matter the source
>  - no concerns about bugs in dh_md5sums or similar resulting in bad checksums
>
> Advantages of doing it when uploading:
>  - provides some sort of double check of what's being uploaded
>  - saves CPU time on users' machines
   - avoids having bad checksums due to the user having bad hardware
 (which is one big use case of the files)

> For me, I'd rather have dpkg generate the hashes.
>
> Cheers,
> aj
>
> --
> Anthony Towns 

MfG
Goswin


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87r5nxou09@frosties.localdomain

Re: md5sums files

[Goswin von Brederlow]

Michael Banck  writes:

> On Fri, Mar 05, 2010 at 07:54:29PM +0100, Goswin von Brederlow wrote:
>> Bernd Zeimetz  writes:
>> > I think its about time to require to generate checksums for packages and 
>> > make
>> > all packages which do not do so RC buggy.
>> 
>> If a checksum file becomes required then it really is not the job of the
>> package to build it. Instead dpkg should generate one and include it
>> automatically. 
>
> Following that logic, dpkg should do everything anyway (like rpm), no
> need for debhelper, dh, or anything.

That does not follow. Nobody says "you MUST use debhelper" but people in
this thread want "you MUST have a checksum file".

> I don't think that just because something is required, it should be
> necessarily part of dpkg.  So far, we are talking about a policy of
> including md5sum in our .debs, *not* about changing the .deb format to
> require md5sums (at least, as far as I can tell).

Yes we do. If not having a md5sum (or rather sha256sum) file in the
package is a policy violation of a MUST directive then the .deb format
is effectively changed.

> Once the former works out fine for squeeze, we can still think about
> doing the latter by modifying dpkg-builddeb or something, but I think
> this is premature at this point (in the release cycle).

Doing the later already solves the former without any extra work for any
(other) maintainer. Every single deb being build from then on would
instantly and automatically have the new checksum field. No need for
every maintainer for add a dh_sha256sum to debian/rules. Having every
maintainer add/change the checksum generation just so they can remove it
again in the next phase would be truely pointless. No package would
become RC buggy due to lack of generating the checksum file.

MfG
Goswin


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87vdd9ou47@frosties.localdomain

Re: md5sums files

[Goswin von Brederlow]

Peter Samuelson  writes:

>> > On Wed, Mar  3, 2010 at 10:05:11 -0600, Peter Samuelson wrote:
>> >> fundamentally, shipping a md5sums file is really just a tradeoff in
>> >> download size vs. installation speed, not unlike gzip vs. bzip2.  One
>
>> Julien Cristau  writes:
>> > Only if you assume that disks never fail and thus files never get
>> > corrupted when the package gets unpacked.
>
> [Goswin von Brederlow]
>> Or the memory, the cpu, the pci bus, the ide bus, ... have a bit
>> toggler. There are many ways file can be corrupted between being
>> downloaded (where apt checks them) and them being unpacked and
>> checksumed locally.
>
> Be that as it may, I don't think the md5sums file was ever intended to
> be an integrity check of the .deb itself.  Fortunately, the .deb also
> includes checksums of control.tar.gz and data.tar.gz, thanks to use of
> the gzip container format.

That is not about the integrity of the deb. It is about the integrity of
the files on the system. And if you do have faulty memory (or any of the
other problems) then calculating the checksum locally will have a high
risk of calculating it from already corrupted data and miss the error.

MfG
Goswin


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87zl2louls@frosties.localdomain

Re: dpkg-deb temporary storage directory

[Roberto C . Sánchez]

On Sat, Mar 06, 2010 at 05:14:48PM -0500, Adam C Powell IV wrote:
> On Sat, 2010-03-06 at 18:15 +0100, Cyril Brulebois wrote:
> > Adam C Powell IV  (06/03/2010):
> > > How can I change the temporary directory where it builds the
> > > tarballs? I don't see anything in the manpage or dpkg-deb --help
> > > output.
> > 
> > (Untested)
> > 
> > It probably honours TMP/TMPDIR environment variables.
> 
> Ah yes, a search for "dpkg-deb TMPDIR" turns up bug 247086 where someone
> asked to include this in the documentation.  The reply: but it's so
> obvious, no need to include it.  Version 1.10.22 claimed to fix it, but
> only fixed one part of the bug, the TMPDIR issue remained.
> 
> I've been a DD for ten years and didn't think of it, so I can't agree
> about the obviousness...  I think I'll open a new bug.
> 
This makes me wonder if it would be a good idea to include something
in policy about applications respecting TMP/TMPDIR.  I have run across
several applications in Debian at various times that do not really
support TMP and TMPDIR to the degree that one would expect.

Regards,

-Roberto

-- 
Roberto C. Sánchez
http://people.connexer.com/~roberto
http://www.connexer.com


signature.asc
Description: Digital signature

Re: md5sums files

[Don Armstrong]

On Sat, 06 Mar 2010, Andreas Metzler wrote:
> Russ Allbery  wrote:
> > Figuring out a better solution for why the files in
> > /var/lib/ispell and /var/lib/aspell are excluded from the md5sums
> > generation because they change after installation is probably
> > needed
>
> I think these hash files are not arch independent. Which is why the
> packages only ship empty files in the deb and generate the correct
> binary data in postinst. This way the hash files are listed in dpkg
> -L, and dpkg takes care of removal instead of relying on
> maintainerscripts.

So there's a period on upgrade where the file has been overwritten
with an file before the new file has been generated?

That's just wrong.

This is why you don't ship files in the package that you're going to
regenerate later. It's not like they can't be removed upon purge or
removal fairly trivially.


Don Armstrong

-- 
I shall require that [a scientific system's] logical form shall be
such that it can be singled out, by means of emperical tests, in a
negative sense: it must be possible for an emperical scientific system
to be refuted by experience.
 -- Sir Karl Popper _Logic of Scientific Discovery_ §6

http://www.donarmstrong.com  http://rzlab.ucr.edu


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100306222825.gp28...@volo.donarmstrong.com

Re: dpkg-deb temporary storage directory

[Adam C Powell IV]

On Sat, 2010-03-06 at 18:15 +0100, Cyril Brulebois wrote:
> Adam C Powell IV  (06/03/2010):
> > How can I change the temporary directory where it builds the
> > tarballs? I don't see anything in the manpage or dpkg-deb --help
> > output.
> 
> (Untested)
> 
> It probably honours TMP/TMPDIR environment variables.

Ah yes, a search for "dpkg-deb TMPDIR" turns up bug 247086 where someone
asked to include this in the documentation.  The reply: but it's so
obvious, no need to include it.  Version 1.10.22 claimed to fix it, but
only fixed one part of the bug, the TMPDIR issue remained.

I've been a DD for ten years and didn't think of it, so I can't agree
about the obviousness...  I think I'll open a new bug.

> > [Please CC me in replies.]
> 
> [Done.]

Thanks!

-Adam
-- 
GPG fingerprint: D54D 1AEE B11C CE9B A02B  C5DD 526F 01E8 564E E4B6

Engineering consulting with open source tools
http://www.opennovation.com/


signature.asc
Description: This is a digitally signed message part

Re: md5sums files

[Anthony Towns]

(I'm not subscribed to this list, so go ahead and Cc me)

On Thu, Mar 4, 2010 at 02:05, Peter Samuelson  wrote:
> [Wouter Verhelst]
> > I must say I was somewhat surprised by these numbers. Out of 2483
> > packages installed on my laptop, 2340 install md5sums.
> The surprising part, perhaps, is that dpkg itself didn't just generate
> the other 143 md5sums files at installation time.

The easy (and usually correct) reason for things like that is "dpkg's
source is scary".

> I suggested this a long time ago and of course was met with "so where's
> your patch?"  Of course I was not willing to do the work.

See? Anyway, my patch is attached. It makes dpkg create a "foo.hashes"
when unpacking foo, whose contents looks like:

MD5:32b5e22f8e336b2f34e0dd87652e6dfc  usr/share/doc/mawk/changelog.gz
MD5:87a34f1f55ac3f7fec2c7fc82565e8eb  usr/share/doc/mawk/changelog.Debian.gz
...

Verification is a matter of something like:

$ cat /var/lib/dpkg/info/*.hashes | sed -n 's/^MD5://p' | (cd /;
md5sum -c) | grep -v ': OK$'

There's an option (--hash) that you can set to "none" to avoid
spending time calculating md5s if you so choose. Adding support for
sha1/sha256/whatever should be straightforward; afaik dpkg only has
code for md5 already built in though (though just invoking
/usr/bin/sha1sum etc would be an option of course).

Of course another option is just to pull the md5sums directly from the deb:

$ ar p /var/cache/apt/archives/ifupdown_0.6.9_i386.deb data.tar.gz |
tar --to-command='printf "%s%s\n" "$(md5sum - | sed s/-$//)"
"${TAR_FILENAME#./}"'  -xzf - |
diff - /var/lib/dpkg/info/ifupdown.md5sums
1,3d0
< 346208729633adf45e2fa3f2bd3b19c6  etc/init.d/ifupdown
< c6fffaae03271f1641920105ce68796b  etc/init.d/ifupdown-clean
< fab851ca87c5deb9d6f665e610184648  etc/default/ifupdown
4a2
> a0f11cf1809a468c49b72e0aa0a8e26b  sbin/ifup

(md5sums doesn't normally list conffiles, but does list hardlinks; the
above command does the opposite)

> But
> fundamentally, shipping a md5sums file is really just a tradeoff in
> download size vs. installation speed, not unlike gzip vs. bzip2.

Advantages of doing in when unpacking:
 - choice of checksum is the admin's decision
 - we can quickly roll out support for sha1/sha256/crc/... checksums
by just changing one package
 - admin has hashes of exactly what was unpacked, no matter the source
 - no concerns about bugs in dh_md5sums or similar resulting in bad checksums

Advantages of doing it when uploading:
 - provides some sort of double check of what's being uploaded
 - saves CPU time on users' machines

For me, I'd rather have dpkg generate the hashes.

Cheers,
aj

--
Anthony Towns 
diff -urb dpkg-1.15.5.6/debian/changelog dpkg-1.15.5.6-aj/debian/changelog
--- dpkg-1.15.5.6/debian/changelog	2010-01-09 04:02:03.0 +1000
+++ dpkg-1.15.5.6-aj/debian/changelog	2010-03-07 04:13:03.171356041 +1000
@@ -1,3 +1,11 @@
+dpkg (1.15.5.6+aj) unstable; urgency=low
+
+  * Non-maintainer upload.
+  * Automatically create /var/lib/dpkg/info/pkg.hashes containing MD5 hashes
+for unpacked files.
+
+ -- Anthony Towns   Sun, 07 Mar 2010 04:12:32 +1000
+
 dpkg (1.15.5.6) unstable; urgency=low
 
   * dpkg-source: with format "3.0 (quilt)" ensure quilt's .pc directory is
diff -urb dpkg-1.15.5.6/configure.ac dpkg-1.15.5.6-aj/configure.ac
--- dpkg-1.15.5.6/configure.ac	2010-01-08 18:00:34.0 +1000
+++ dpkg-1.15.5.6-aj/configure.ac	2010-03-07 04:38:32.547372468 +1000
@@ -51,6 +51,16 @@
 esac])
 AC_SUBST(admindir)
 
+# Allow alternative default hash function
+hashtype="md5"
+AC_ARG_WITH(hashtype,
+	AS_HELP_STRING([--with-hashtype=HASH],
+	   [hash function to use for .hashes files]),
+[case "$with_hashtype" in
+  "md5"|"none") hashtype="$with_hashtype" ;;
+  *) AC_MSG_ERROR([invalid hashtype specified]) ;;
+esac])
+AC_SUBST(hashtype)
 
 # Checks for programs.
 AC_PROG_CC
diff -urb dpkg-1.15.5.6/src/Makefile.am dpkg-1.15.5.6-aj/src/Makefile.am
--- dpkg-1.15.5.6/src/Makefile.am	2010-01-09 03:23:06.0 +1000
+++ dpkg-1.15.5.6-aj/src/Makefile.am	2010-03-07 04:28:18.771356095 +1000
@@ -6,6 +6,7 @@
 AM_CPPFLAGS = \
 	-DLOCALEDIR=\"$(localedir)\" \
 	-DADMINDIR=\"$(admindir)\" \
+	-DHASHTYPE=\"$(hashtype)\" \
 	-idirafter $(top_srcdir)/lib/compat \
 	-I$(top_builddir) \
 	-I$(top_srcdir)/lib
diff -urb dpkg-1.15.5.6/src/main.c dpkg-1.15.5.6-aj/src/main.c
--- dpkg-1.15.5.6/src/main.c	2010-01-09 03:23:06.0 +1000
+++ dpkg-1.15.5.6-aj/src/main.c	2010-03-07 04:29:59.271360858 +1000
@@ -187,6 +187,7 @@
 const char *admindir= ADMINDIR;
 const char *instdir= "";
 struct pkg_list *ignoredependss = NULL;
+const char *hashtype= HASHTYPE;
 
 static const struct forceinfo {
   const char *name;
@@ -516,6 +517,7 @@
   { "admindir",  0,   1, NULL,  &admindir, NULL,  0 },
   { "instdir",   0,   1, NULL,  &instdir,  NULL,  0 },
   { "ignore-depends",0,   1, NULL,  NULL,  ignoredepends, 0 },
+  { "hash",  0

Ban muon 1 shop online?

[vipharcovn]

 1267903060456-1267903060456-1267903060456


Ban muon rao vat? Muon mua ban tren mang? Ban muon co 1 khong gian rieng 
online? 1 cua hang online? 1 blog rieng? 1 web site rieng?Hay vao 
http://IMARKET.VN


 1267903060456-1267903060456-1267903060456


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/2.4.1267903060...@localhost

Re: linux-image-*-dbg for squeeze?

[Lucas Nussbaum]

[ Adding ftpmasters to To: ]

On 05/03/10 at 19:34 +0100, Bastian Blank wrote:
> On Fri, Mar 05, 2010 at 09:07:50AM +0100, Lucas Nussbaum wrote:
> > I wonder what we (as Debian) could do about it. Would it make sense to
> > sponsor a very fast machine that the kernel team could use to build the
> > kernels and upload from, replacing kernel-archive.buildserver.net ?
> 
> The easiest fix is the official blessing by the ftp team to upload only
> the architecture independant packages and build all architectures on the
> buildds.

Ftpmasters, would it be acceptable for the kernel team to upload only
the architecture independant packages ?
It would allow to provide debug packages on fast architectures without
forcing the developers to upload them using their slow internet
connections, solving #365349.
-- 
| Lucas Nussbaum
| lu...@lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F |


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100306180621.ga21...@xanadu.blop.info

Re: dpkg-deb temporary storage directory

[Cyril Brulebois]

Adam C Powell IV  (06/03/2010):
> How can I change the temporary directory where it builds the
> tarballs? I don't see anything in the manpage or dpkg-deb --help
> output.

(Untested)

It probably honours TMP/TMPDIR environment variables.

> [Please CC me in replies.]

[Done.]

Mraw,
KiBi.


signature.asc
Description: Digital signature

dpkg-deb temporary storage directory

[Adam C Powell IV]

Greetings,

I'm working on a package for Salomé, and some of its binaries are really
huge -- way too big, I'll split them up at some point.

But in the meantime, I'm getting the following error and want to avoid
it:

dpkg-deb: building package `salome-doc' in `../salome-doc_5.1.3-5opvk1_all.deb'.
dpkg-deb (subprocess): data: internal gzip error: read(4096) != write(0): No 
space left on device
dpkg-deb: subprocess  from tar -cf returned error exit status 2
dh_builddeb: dpkg-deb returned exit code 2

The partition where I'm building this has 6 gigs of free space, which is
more than enough room, and building within a chroot on that partition
works fine.  So I guess it's trying to build a tarball in a temporary
storage directory, and / has "only" 380 megs free.

How can I change the temporary directory where it builds the tarballs?
I don't see anything in the manpage or dpkg-deb --help output.

[Please CC me in replies.]

-Adam
-- 
GPG fingerprint: D54D 1AEE B11C CE9B A02B  C5DD 526F 01E8 564E E4B6

Engineering consulting with open source tools
http://www.opennovation.com/


signature.asc
Description: This is a digitally signed message part

RE : Re: Cmap file are now free: List of package to move to main

[Bastien ROUCARIES]

the bug report are already made.

Le 6 mars 2010 12:06, "Yves-Alexis Perez"  a écrit :

On lun., 2010-03-01 at 10:19 +0100, Bastien ROUCARIES wrote:
>
> I have send this mail two months a...
Maybe you just need to report bugs against concerned packages so the
maintainer changes the section.

Cheers,
--
Yves-Alexis

Re: md5sums files

[Michael Banck]

On Fri, Mar 05, 2010 at 07:54:29PM +0100, Goswin von Brederlow wrote:
> Bernd Zeimetz  writes:
> > I think its about time to require to generate checksums for packages and 
> > make
> > all packages which do not do so RC buggy.
> 
> If a checksum file becomes required then it really is not the job of the
> package to build it. Instead dpkg should generate one and include it
> automatically. 

Following that logic, dpkg should do everything anyway (like rpm), no
need for debhelper, dh, or anything.

I don't think that just because something is required, it should be
necessarily part of dpkg.  So far, we are talking about a policy of
including md5sum in our .debs, *not* about changing the .deb format to
require md5sums (at least, as far as I can tell).

Once the former works out fine for squeeze, we can still think about
doing the latter by modifying dpkg-builddeb or something, but I think
this is premature at this point (in the release cycle).


Michael


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20100306114901.gb13...@nighthawk.chemicalconnection.dyndns.org

Re: Cmap file are now free: List of package to move to main

[Yves-Alexis Perez]

On lun., 2010-03-01 at 10:19 +0100, Bastien ROUCARIES wrote:
> 
> I have send this mail two months ago and no answer, therefore I
> resend. Please add cmap to main. 

Maybe you just need to report bugs against concerned packages so the
maintainer changes the section.

Cheers,
-- 
Yves-Alexis


signature.asc
Description: This is a digitally signed message part

Re: md5sums files

[Jean-Christophe Dubacq]

On 06/03/2010 09:27, Andreas Metzler wrote:
> Russ Allbery  wrote:
>> Don Armstrong  writes:
> [...]
>>> Is there any reason why we can't just modify dpkg-deb to create
>>> DEBIAN/md5sums and DEBIAN/sha512sums and get archive coverage relatively
>>> quickly, automatically, as things get rebuilt?
> 
>> Figuring out a better solution for why the files in /var/lib/ispell and
>> /var/lib/aspell are excluded from the md5sums generation because they
>> change after installation is probably needed
> [...]
> 
> Hello,
> I think these hash files are not arch independent. Which is why the
> packages only ship empty files in the deb and generate the correct
> binary data in postinst. This way the hash files are listed in dpkg
> -L, and dpkg takes care of removal instead of relying on
> maintainerscripts.

Anyway, there are many more cases where dpkg -L does not list all files
pertaining to a single package. Configuration files managed by
postinsts, for example, instances data, etc.

It's a shame there is no standard way to declare the (package-)ownership
of a file (the cruft(1) format is good enough, but is under-used, to say
the least).

I have recently taken an interest in obsolete conffiles and it's a real
mess to check whether a conffile is really obsolete (can be removed) or
is now managed by postinst (or ucf). Some conffiles left behind can have
side-effects (usually not important ones).
-- 
Jean-Christophe Dubacq


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4b9223ef.6040...@free.fr

Re: md5sums files

[Andreas Metzler]

Russ Allbery  wrote:
> Don Armstrong  writes:
[...]
>> Is there any reason why we can't just modify dpkg-deb to create
>> DEBIAN/md5sums and DEBIAN/sha512sums and get archive coverage relatively
>> quickly, automatically, as things get rebuilt?

> Figuring out a better solution for why the files in /var/lib/ispell and
> /var/lib/aspell are excluded from the md5sums generation because they
> change after installation is probably needed
[...]

Hello,
I think these hash files are not arch independent. Which is why the
packages only ship empty files in the deb and generate the correct
binary data in postinst. This way the hash files are listed in dpkg
-L, and dpkg takes care of removal instead of relying on
maintainerscripts.

cu andreas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/d9g967-3i3@argenau.downhill.at.eu.org

Bug#572737: RFP: maven-scala-plugin -- A maven plugin for compiling, testing and running scala code in maven

[Ramakrishnan Muthukrishnan]

Package: wnpp
Severity: wishlist

* Package name: maven-scala-plugin
  Version : 2.12
  Upstream Author : David Bernard
* URL or Web page : 

* License : Apache License 2.0
  Description : A maven plugin for compiling, tetsing and running scala 
code in maven






-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1nnorc-0001mc...@rkrishnant61