from:"Steven Chamberlain"

Bastian Blank wrote:
> On Tue, Feb 28, 2017 at 10:00:01PM +0000, Steven Chamberlain wrote:
> > That differs from the latest version of my patch, and from what I sent
> > earlier today to the release team when asking about a potential unblock:
> > https://lists.debian.org/debian-release/2017/02/msg01033.html
> 
> This happens if you send incomplete patches and do uncoordinated unblock
> requests.

Maybe you just volunteered to do that, then.

You even said before you "don't have time" to write the cdebootstrap
patch, so I offered one, and the anna patch, the libdebian-installer
patch, all this after the initial discovery, triage and write-up.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Bastian Blank wrote:
> Adopted and commited to
> https://anonscm.debian.org/git/d-i/libdebian-installer.git, branch
> sha256

That differs from the latest version of my patch, and from what I sent
earlier today to the release team when asking about a potential unblock:
https://lists.debian.org/debian-release/2017/02/msg01033.html

I think we should wait for them to answer before doing anything else.

Based on KiBi's feedback I thought it better to swap sum[0] and sum[1],
and remove the SHA1 parsing also.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#856405: unblock: libdebian-installer/0.109 and others

Package: release.debian.org
User: release.debian@packages.debian.org
Usertags: unblock
X-Debbugs-Cc: debian-boot@lists.debian.org

Hi!

Attached are proposed debdiffs for anna, cdebootstrap and their
dependency libdebian-installer (Bug #856210).

Would the release team be willing to grant unblocks for these?
(It would also require an ACK from the d-i release manager).

In the installer, net-retriever verifies the Release file with SHA256,
but anna only validates the .udeb files with MD5, which was surprising.
The .udeb files are extracted and then their contents may be executed
with full privileges during the install (Bug #856211).

netboot images typically fetch .udeb files over unsecured HTTP.  Other
install media bundles those so they need not be downloaded, but it could
still happen if networking is configured during the install and a
network mirror has newer versions of any required .udeb files.  (Some
.udeb files are retrieved later, after installing the base system).

If not already considered a grave security flaw, it might be during the
lifetime of stretch (-2022?).  Even if fixed in a point release, any
install media created before then would remain vulnerable.

cdebootstrap - not used by the installer, but available in stretch - 
only verifies Release files using MD5 (Bug #856215), which is an
unintended regression since jessie.

The changes to libdebian-installer are ABI-compatible, such that only
reverse-dependencies that use the md5sum field should be affected
(thought to be just anna and cdebootstrap).  They would FTBFS until
patched, and already-built binaries would report a "md5sum mismatch" if
they used this new version of the library at run-time, since the new
SHA256 hashes would not match the MD5 hashes they expect.

unblock libdebian-installer/0.109
unblock anna/1.58
unblock cdebootstrap/0.7.7

Thanks!

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 10.1-0-amd64
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
diff -Nru libdebian-installer-0.108/debian/changelog 
libdebian-installer-0.109/debian/changelog
--- libdebian-installer-0.108/debian/changelog  2016-08-30 05:45:17.0 
+0100
+++ libdebian-installer-0.109/debian/changelog  2017-02-28 16:36:45.0 
+
@@ -1,3 +1,15 @@
+libdebian-installer (0.109) unstable; urgency=high
+
+  * In structs di_release and di_package, replace md5sum member with a
+sha256 member.  This is ABI-compatible, but reverse-dependencies
+will fail if they still try to verify with MD5 (Closes: #856212).
+  * Parse SHA256 fields instead of MD5Sum fields in Packages files.
+  * Parse SHA256 fields instead of MD5Sum fields in Release files.
+  * No longer try to parse SHA1 field, which is no longer present in
+Release files as of stretch.
+
+ -- Steven Chamberlain <stev...@debian.org>  Tue, 28 Feb 2017 16:36:45 +
+
 libdebian-installer (0.108) unstable; urgency=medium
 
   [ Helmut Grohne ]
diff -Nru libdebian-installer-0.108/include/debian-installer/package.h 
libdebian-installer-0.109/include/debian-installer/package.h
--- libdebian-installer-0.108/include/debian-installer/package.h
2011-03-03 02:00:33.0 +
+++ libdebian-installer-0.109/include/debian-installer/package.h
2017-02-28 16:33:59.0 +
@@ -112,7 +112,7 @@
   di_slist depends; /**< Any different 
dependency types */
   char *filename;   /**< Filename field */
   size_t size;  /**< Size field */
-  char *md5sum; /**< MD5Sum field */
+  char *sha256; /**< SHA256 field */
   char *short_description;  /**< Description 
field, first part*/
   char *description;/**< Description 
field, second part */
   unsigned int resolver;/**< @internal */
diff -Nru libdebian-installer-0.108/include/debian-installer/package_internal.h 
libdebian-installer-0.109/include/debian-installer/package_internal.h
--- libdebian-installer-0.108/include/debian-installer/package_internal.h   
2011-03-03 02:00:33.0 +
+++ libdebian-installer-0.109/include/debian-installer/package_internal.h   
2017-02-28 16:33:59.0 +
@@ -52,7 +52,7 @@
   internal_di_package_parser_field_enhances,
   internal_di_package_parser_field_filename,
   internal_di_package_parser_field_size,
-  internal_di_package_parser_field_md5sum,
+  internal_di_package_parser_field_sha256,
   internal_di_package_parser_field_description;
 
 /**
diff -Nru libdebian-installer-0.108/include/debian-installer/release.h 
libdebian-installer-

Bug#856211: anna: please implement SHA256 verification of .udeb files

Updated patch, which assumes the libdebian-installer4-dev package will
not be renamed.  Build-Depend on a recent enough version that provides
sha256 fields.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/debian/changelog b/debian/changelog
index d6682ca..20e33a0 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+anna (1.58) UNRELEASED; urgency=medium
+
+  * Team upload.
+  * Replace md5sum verification with sha256sum (Closes: #856211).
+- (Build-)Depend on libdebian-installer4-dev >= 0.109 which provides
+  those sha256 fields.
+
+ -- Steven Chamberlain <stev...@debian.org>  Mon, 27 Feb 2017 15:13:37 +
+
 anna (1.57) unstable; urgency=medium
 
   [ Updated translations ]
diff --git a/debian/control b/debian/control
index def2af9..4998489 100644
--- a/debian/control
+++ b/debian/control
@@ -3,7 +3,7 @@ Section: debian-installer
 Priority: standard
 Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
 Uploaders: Bastian Blank <wa...@debian.org>, Christian Perrier <bubu...@debian.org>
-Build-Depends: debhelper (>= 9), dpkg-dev (>= 1.15.7), libdebconfclient0-dev (>= 0.46), libdebian-installer4-dev (>= 0.41)
+Build-Depends: debhelper (>= 9), dpkg-dev (>= 1.15.7), libdebconfclient0-dev (>= 0.46), libdebian-installer4-dev (>= 0.109)
 Vcs-Browser: https://anonscm.debian.org/cgit/d-i/anna.git
 Vcs-Git: https://anonscm.debian.org/git/d-i/anna.git
 
diff --git a/anna.c b/anna.c
index 4b68816..e03d34a 100644
--- a/anna.c
+++ b/anna.c
@@ -318,8 +318,8 @@ install_modules(di_packages *status, di_packages *packages) {
 	}
 }
 
-if (! md5sum(package->md5sum, dest_file)) {
-	di_log(DI_LOG_LEVEL_WARNING, "bad md5sum");
+if (! sha256sum(package->sha256, dest_file)) {
+	di_log(DI_LOG_LEVEL_WARNING, "bad sha256sum");
 	if (!quiet)
 		/* error handling may use a progress bar, so stop the current one */
 		debconf_progress_stop(debconf);
diff --git a/util.c b/util.c
index 39af3db..7d09cf8 100644
--- a/util.c
+++ b/util.c
@@ -224,23 +224,26 @@ int load_templates (di_packages *packages) {
 }
 #endif /* LOADTEMPLATES */
 
-/* Check whether the md5sum of file matches sum. If not, return 0. */
-int md5sum(const char *sum, const char *file) {
+/* Length of a SHA256 hash in hex representation */
+#define SHA256_HEX_LENGTH 64
+
+/* Check whether the sha256sum of file matches sum. If not, return 0. */
+int sha256sum(const char *sum, const char *file) {
 	FILE *fp;
 	char line[1024];
 
-	/* Trivially true if the Packages file doesn't have md5sum lines */
+	/* Trivially true if the Packages file doesn't have sha256sum lines */
 	if (sum == NULL)
 		return 1;
-	snprintf(line, sizeof(line), "/usr/bin/md5sum %s", file);
+	snprintf(line, sizeof(line), "/usr/bin/sha256sum %s", file);
 	fp = popen(line, "r");
 	if (fp == NULL)
 		return 0;
 	if (fgets(line, sizeof(line), fp) != NULL) {
 		pclose(fp);
-		if (strlen(line) < 32)
+		if (strlen(line) < SHA256_HEX_LENGTH)
 			return 0;
-		line[32] = '\0';
+		line[SHA256_HEX_LENGTH] = '\0';
 		return !strcmp(line, sum);
 	}
 	pclose(fp);
diff --git a/util.h b/util.h
index 71135e0..0c9e0a2 100644
--- a/util.h
+++ b/util.h
@@ -10,7 +10,7 @@ bool is_installed(di_package *p, di_packages *status);
 size_t package_to_choice(di_package *package, char *buf, size_t size);
 char *list_to_choices(di_package **packages, int c_values);
 int get_package (di_package *package, char *dest);
-int md5sum(const char* sum, const char *file);
+int sha256sum(const char* sum, const char *file);
 int skip_package(di_package *p);
 int package_name_compare(const void *v1, const void *v2);
 void take_includes(di_packages *packages);


signature.asc
Description: Digital signature

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Steven Chamberlain wrote:
> replace sum[0] with sha256 and leave sum[1] empty; 
> [...] (we would drop the MD5- and SHA1-parsing code
> and make absolutely sure nobody is still using those).

The new patch attached would do that, and it remains otherwise
ABI-compatible.

It aims to be the most minimal diff, so it does not extend the testsuite
for example, which still passes even though the Packages file testcase
has no SHA256 fields.

In src/release.c:  file->sum[1] is initialised to NULL by a calloc().
In the future, someone might want to put SHA512 hashes there.  It does
not hurt to keep the existing di_free(file->sum[1]) in place.

Within the installer, this should only break anna, until the patch from
#856211 is applied.

Outside of the installer, cdebootstrap would break, until #856212 is
patched.

If we missed any other reverse-depends, they should FTBFS if they
dereference the md5sum fields.  Already-built binaries should report a
"md5sum mismatch", if they use the patched libdebian-installer at
run-time and still try to do verification with MD5.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/debian/changelog b/debian/changelog
index 3dd29e1..748a78b 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,7 +1,17 @@
 libdebian-installer (0.109) UNRELEASED; urgency=medium
 
+  [ Samuel Thibault ]
   * Fix build with gcc-7. Closes: #853489
 
+  [ Steven Chamberlain ]
+  * In structs di_release and di_package, replace md5sum member with a
+sha256 member.  This is ABI-compatible, but reverse-dependencies
+will fail if they still try to verify with MD5 (Closes: #856212).
+  * Parse SHA256 fields instead of MD5Sum fields in Packages files.
+  * Parse SHA256 fields instead of MD5Sum fields in Release files.
+  * No longer try to parse SHA1 field, which is no longer present in
+Release files as of stretch.
+
  -- Samuel Thibault <sthiba...@debian.org>  Tue, 31 Jan 2017 11:09:16 +0100
 
 libdebian-installer (0.108) unstable; urgency=medium
diff --git a/include/debian-installer/package.h b/include/debian-installer/package.h
index 72d7444..e1f699d 100644
--- a/include/debian-installer/package.h
+++ b/include/debian-installer/package.h
@@ -112,7 +112,7 @@ struct di_package
   di_slist depends; /**< Any different dependency types */
   char *filename;   /**< Filename field */
   size_t size;  /**< Size field */
-  char *md5sum; /**< MD5Sum field */
+  char *sha256; /**< SHA256 field */
   char *short_description;  /**< Description field, first part*/
   char *description;/**< Description field, second part */
   unsigned int resolver;/**< @internal */
diff --git a/include/debian-installer/package_internal.h b/include/debian-installer/package_internal.h
index f6357d1..d410ce2 100644
--- a/include/debian-installer/package_internal.h
+++ b/include/debian-installer/package_internal.h
@@ -52,7 +52,7 @@ const di_parser_fieldinfo
   internal_di_package_parser_field_enhances,
   internal_di_package_parser_field_filename,
   internal_di_package_parser_field_size,
-  internal_di_package_parser_field_md5sum,
+  internal_di_package_parser_field_sha256,
   internal_di_package_parser_field_description;
 
 /**
diff --git a/include/debian-installer/release.h b/include/debian-installer/release.h
index 223a4f8..29b0cfb 100644
--- a/include/debian-installer/release.h
+++ b/include/debian-installer/release.h
@@ -40,7 +40,7 @@ struct di_release
   char *origin; /**< Origin field */
   char *suite;  /**< Suite field */
   char *codename;   /**< Codename field */
-  di_hash_table *md5sum;/**< checksum fields, includes di_release_file */
+  di_hash_table *sha256;/**< checksum fields, includes di_release_file */
   di_mem_chunk *release_file_mem_chunk; /**< @internal */
 };
 
@@ -55,7 +55,7 @@ struct di_release_file
 di_rstring key; /**< @internal */
   };
   unsigned int size;/**< size */
-  char *sum[2]; /**< checksums, currently md5 and sha1 */
+  char *sum[2]; /**< sum[0] is sha256, sum[1] is empty */
 };
 
 di_release *di_release_alloc (void);
diff --git a/src/package.c b/src/package.c
index 653b5dd..82c7653 100644
--- a/src/package.c
+++ b/src/package.c
@@ -38,7 +38,7 @@ void di_package_destroy (di_package *package)
   di_free (package->architecture);
   di_free (package->version);
   di_free (package->filename);
-  di_free (package->md5sum);
+  di_

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Thanks for your comments!

Cyril Brulebois wrote:
> Steven Chamberlain <ste...@pyro.eu.org> (2017-02-27):
> > (If we really wanted, we could maybe avoid the ABI bump: [...]
> 
> Given the number of reverse dependencies, I doubt this is worth abusing
> md5 storage for sha256 things.

Maybe I should clarify that;  the current libdebian-installer/0.108 has:

di_package struct {
...
char *md5sum; // -> dynamically allocated md5sum field
...
}

and we'd be changing it to:

di_package struct {
...
char *sha256; // -> dynamically allocated sha256 field
...
}

("Sum" was dropped from name of that field in the Release file, so I do
the same here)

Changing the name, causes reverse-deps using that field to FTBFS.  I
think that is just anna and cdebootstrap, which we'd patch anyway.

The md5sum/sha256 field is a pointer to a dynamically-allocated field.
The struct size, and the offset of other members does not change, so
nothing else should need rebuilding with the newer package.h

"If" somehow, we missed something, which tries to dereference
package->md5sum at run-time with a new version of libdebian-installer,
it would find a sha256 hash there instead of md5.  That should fail
"safely" by complaining of a md5sum mismatch (even if it only compares
the first 32 bytes, as cdebootstrap does currently).

That's why I think an ABI bump could be safely avoided.  (And I think
Bastian agrees now?)

> Bumping the ABI seems reasonable to me,
> even if that's effectively starting a mini-transition from a release
> point of view.

[...]

> > -Package: libdebian-installer4-dev
> > +Package: libdebian-installer5-dev
> 
> Please don't!

You suggest to "bump the ABI" but not rename the packages?  or...?
Maybe the argument above is convincing enough to just not bump the ABI?

> > --- a/include/debian-installer/release.h
> > +++ b/include/debian-installer/release.h
> > @@ -40,7 +40,7 @@ struct di_release
> >char *origin; /**< Origin field */
> >char *suite;  /**< Suite field */
> >char *codename;   /**< Codename field */
> > -  di_hash_table *md5sum;/**< checksum fields, 
> > includes di_release_file */
> > +  di_hash_table *sha256;/**< checksum fields, 
> > includes di_release_file */
> >di_mem_chunk *release_file_mem_chunk; /**< @internal */
> >  };
> 
> So md5sum goes away from the di_release struct…

Yes, the same as with di_package;  that preserves ABI compatibility,
and getting rid of md5sum is also our intent.

> 
> >  
> > @@ -55,7 +55,7 @@ struct di_release_file
> >  di_rstring key; /**< @internal */
> >};
> >unsigned int size;/**< size */
> > -  char *sum[2]; /**< checksums, currently 
> > md5 and sha1 */
> > +  char *sum[2]; /**< checksums, currently 
> > md5 and sha256 */
> 
> … but is kept in the di_release_file one?

Right, this struct currently contains:

char *sum[0] -> dynamically allocated md5sum field
char *sum[1] -> dynamically allocated sha1 field

so that is what reverse-depends expect to be in those fields,
currently.  To keep ABI comptibility, I should keep two items there.

The sha1 field is always empty, since that was removed from the Release
file.  We could either:

  1. replace sum[0] with sha256 and leave sum[1] empty;  or
  2. leave sum[0] containing md5 but replace sum[1] with sha256

My patch did 2. because it results in a smaller diff.  But I like the
idea of doing 1. instead (we would drop the MD5- and SHA1-parsing code
and make absolutely sure nobody is still using those).

If I did 1. and we didn't bump the ABI, it should be easy to test:
  * we'd patch+update only libdebian-installer, then test:
anna should abort the install, due to mismatching md5sums;
  * then we'd patch anna, and it should all work again;  one could
also delete the /usr/bin/md5sum symlink while testing.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Bastian Blank wrote:
> This change breaks the existing ABI and therefor needs an ABI bump, but
> it is missing from the patch.

The attached patch tries to bump the soname to 5.  This makes the diff
much larger, but the code changes are the same.

I think libdebian-installer-extra nowadays gets a soname bump at the
same time as libdebian-installer (whereas in the past it was possible to
set a different soname for each).

(If we really wanted, we could maybe avoid the ABI bump:  no library
functions are being added/removed, only the name and meaning of a struct
member (a pointer, which remains the same length).  The
dynamically-sized buffer it points to, would change from storing an MD5
to a SHA256 hash, and would only cause a regression where something is
still trying to validate MD5).

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/configure.ac b/configure.ac
index d559d64..2f276b8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -16,9 +16,9 @@ AS_IF([test "x$enable_check" != xno],[
 ])
 AM_CONDITIONAL([ENABLE_CHECK],[test "x$enable_check" != xno])
 
-LIBRARY_VERSION_MAJOR=4
+LIBRARY_VERSION_MAJOR=5
 LIBRARY_VERSION_MINOR=0
-LIBRARY_VERSION_REVISION=7
+LIBRARY_VERSION_REVISION=0
 LIBRARY_VERSION="$LIBRARY_VERSION_MAJOR.$LIBRARY_VERSION_MINOR.$LIBRARY_VERSION_REVISION"
 LIBRARY_VERSION_LIBTOOL="$LIBRARY_VERSION_MAJOR:$LIBRARY_VERSION_MINOR:$LIBRARY_VERSION_REVISION"
 
diff --git a/debian/changelog b/debian/changelog
index 3dd29e1..09c194e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,7 +1,18 @@
 libdebian-installer (0.109) UNRELEASED; urgency=medium
 
+  [ Samuel Thibault ]
   * Fix build with gcc-7. Closes: #853489
 
+  [ Steven Chamberlain ]
+  * Parse SHA256 fields instead of MD5Sum fields in Packages files.
+  * Parse SHA256 fields instead of (no longer existing) SHA1 fields in
+Release files.
+  * In structs di_release and di_package, add new sha256 member and
+remove the md5sum member (a backward-incompatible change, this will
+force reverse-dependencies to stop using MD5 for verification)
+(Closes: #856212).
+  * Bump soname as advised by Bastian Blank.
+
  -- Samuel Thibault <sthiba...@debian.org>  Tue, 31 Jan 2017 11:09:16 +0100
 
 libdebian-installer (0.108) unstable; urgency=medium
diff --git a/debian/control b/debian/control
index 0949fd9..f53f55c 100644
--- a/debian/control
+++ b/debian/control
@@ -8,7 +8,7 @@ Standards-Version: 3.9.6
 Vcs-Browser: https://anonscm.debian.org/cgit/d-i/libdebian-installer.git
 Vcs-Git: https://anonscm.debian.org/git/d-i/libdebian-installer.git
 
-Package: libdebian-installer4
+Package: libdebian-installer5
 Architecture: any
 Multi-Arch: same
 Pre-Depends: ${misc:Pre-Depends}
@@ -19,10 +19,10 @@ Description: Library of common debian-installer functions
  working on debian-installer or building your own install system based
  on debian-installer, then you probably don't need this library.
 
-Package: libdebian-installer4-dev
+Package: libdebian-installer5-dev
 Section: libdevel
 Architecture: any
-Depends: ${misc:Depends}, libdebian-installer4 (= ${binary:Version}), libdebian-installer-extra4 (= ${binary:Version})
+Depends: ${misc:Depends}, libdebian-installer5 (= ${binary:Version}), libdebian-installer-extra5 (= ${binary:Version})
 Conflicts: libdebian-installer-dev
 Provides: libdebian-installer-dev
 Description: Library of common debian-installer functions
@@ -33,7 +33,7 @@ Description: Library of common debian-installer functions
  .
  This package contains files needed to do libdebian-installer development.
 
-Package: libdebian-installer4-udeb
+Package: libdebian-installer5-udeb
 Package-Type: udeb
 Section: debian-installer
 Architecture: any
@@ -44,22 +44,22 @@ Description: Library of common debian-installer functions
  working on debian-installer or building your own install system based
  on debian-installer, then you probably don't need this library.
 
-Package: libdebian-installer-extra4
+Package: libdebian-installer-extra5
 Architecture: any
 Multi-Arch: same
-Depends: ${shlibs:Depends}, ${misc:Depends}, libdebian-installer4 (= ${binary:Version})
+Depends: ${shlibs:Depends}, ${misc:Depends}, libdebian-installer5 (= ${binary:Version})
 Description: Library of some extra debian-installer functions
  This library is used by debian-installer to perform common functions
  such as logging messages and executing commands. If you aren't
  working on debian-installer or building your own install system based
  on debian-installer, then you probably don't need this library.
 
-Package: libdebian-installer-extra4-udeb
+Package: libdebian-installer-extra5-udeb
 Package-Type: udeb
 Section: debian-installer
 Architecture: any
-Depends: ${shlibs:Depends}, ${misc:Depends}, libdebian-installer4-udeb (= ${binary:Version})
-Provides: libdebian-installer-extra4
+Depends: ${shlibs:Depends}, ${misc:Depends}, libdebian-installer5-udeb (= ${binary:Version})
+Provides: libd

Bug#856211: anna: please implement SHA256 verification of .udeb files

Control: tags -1 + patch

Hi,

Attached is a minimal patch intended to implement SHA256 verification.
It would depend on libdebian-installer being patched first (#856210) and
bumping the soname to 5.

"#define SHA256_HEX_LENGTH 64" is made explicit as possible so that one
remembers to increase it if changing SHA256 to SHA512 in the future.  A
more thorough rework of this code might store the hash type (as an enum)
and length, in the di_package struct instead.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#856211: anna: please implement SHA256 verification of .udeb files

Steven Chamberlain wrote:
> Attached is [...]

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/anna.c b/anna.c
index 4b68816..e03d34a 100644
--- a/anna.c
+++ b/anna.c
@@ -318,8 +318,8 @@ install_modules(di_packages *status, di_packages *packages) {
 	}
 }
 
-if (! md5sum(package->md5sum, dest_file)) {
-	di_log(DI_LOG_LEVEL_WARNING, "bad md5sum");
+if (! sha256sum(package->sha256, dest_file)) {
+	di_log(DI_LOG_LEVEL_WARNING, "bad sha256sum");
 	if (!quiet)
 		/* error handling may use a progress bar, so stop the current one */
 		debconf_progress_stop(debconf);
diff --git a/debian/changelog b/debian/changelog
index d6682ca..c885457 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,11 @@
+anna (1.58) UNRELEASED; urgency=medium
+
+  * Team upload.
+  * Replace md5sum verification with sha256sum (Closes: #856211).
+- (Build-)Depend on libdebian-installer soname version 5.
+
+ -- Steven Chamberlain <stev...@debian.org>  Mon, 27 Feb 2017 15:13:37 +
+
 anna (1.57) unstable; urgency=medium
 
   [ Updated translations ]
diff --git a/debian/control b/debian/control
index def2af9..20ff1c3 100644
--- a/debian/control
+++ b/debian/control
@@ -3,7 +3,7 @@ Section: debian-installer
 Priority: standard
 Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
 Uploaders: Bastian Blank <wa...@debian.org>, Christian Perrier <bubu...@debian.org>
-Build-Depends: debhelper (>= 9), dpkg-dev (>= 1.15.7), libdebconfclient0-dev (>= 0.46), libdebian-installer4-dev (>= 0.41)
+Build-Depends: debhelper (>= 9), dpkg-dev (>= 1.15.7), libdebconfclient0-dev (>= 0.46), libdebian-installer5-dev
 Vcs-Browser: https://anonscm.debian.org/cgit/d-i/anna.git
 Vcs-Git: https://anonscm.debian.org/git/d-i/anna.git
 
diff --git a/util.c b/util.c
index 39af3db..7d09cf8 100644
--- a/util.c
+++ b/util.c
@@ -224,23 +224,26 @@ int load_templates (di_packages *packages) {
 }
 #endif /* LOADTEMPLATES */
 
-/* Check whether the md5sum of file matches sum. If not, return 0. */
-int md5sum(const char *sum, const char *file) {
+/* Length of a SHA256 hash in hex representation */
+#define SHA256_HEX_LENGTH 64
+
+/* Check whether the sha256sum of file matches sum. If not, return 0. */
+int sha256sum(const char *sum, const char *file) {
 	FILE *fp;
 	char line[1024];
 
-	/* Trivially true if the Packages file doesn't have md5sum lines */
+	/* Trivially true if the Packages file doesn't have sha256sum lines */
 	if (sum == NULL)
 		return 1;
-	snprintf(line, sizeof(line), "/usr/bin/md5sum %s", file);
+	snprintf(line, sizeof(line), "/usr/bin/sha256sum %s", file);
 	fp = popen(line, "r");
 	if (fp == NULL)
 		return 0;
 	if (fgets(line, sizeof(line), fp) != NULL) {
 		pclose(fp);
-		if (strlen(line) < 32)
+		if (strlen(line) < SHA256_HEX_LENGTH)
 			return 0;
-		line[32] = '\0';
+		line[SHA256_HEX_LENGTH] = '\0';
 		return !strcmp(line, sum);
 	}
 	pclose(fp);
diff --git a/util.h b/util.h
index 71135e0..0c9e0a2 100644
--- a/util.h
+++ b/util.h
@@ -10,7 +10,7 @@ bool is_installed(di_package *p, di_packages *status);
 size_t package_to_choice(di_package *package, char *buf, size_t size);
 char *list_to_choices(di_package **packages, int c_values);
 int get_package (di_package *package, char *dest);
-int md5sum(const char* sum, const char *file);
+int sha256sum(const char* sum, const char *file);
 int skip_package(di_package *p);
 int package_name_compare(const void *v1, const void *v2);
 void take_includes(di_packages *packages);


signature.asc
Description: Digital signature

Bug#856211: anna: please implement SHA256 verification of .udeb files

Cyril Brulebois wrote:
> IIRC MD5sum field was kept (as in: added
> back) because debian-cd needs it at the moment, which partly explains why this
> wasn't fixed earlier.

I think backward-compatibility would have been okay as long as *either*:

  * the archive published Release files with old+new hash algorithms; or
  * the utilities consuming it, supported the old/new hash algorithms;

but here we had done both of those things, which allowed for a downgrade
to go unnoticed.

I think right now it is easier to fix anna+cdebootstrap than debian-cd?

> but referencing places where stuff like parsing happens
> (Release, Packages, etc.), and where checkums are used,

Yesss, but only if someone updated that documentation with what the code
is doing.  Removal of SHA1 in Relases had an action-at-a-distance effect
on cdebootstrap, so it wouldn't be clear that the documentation needed
to change then.

In the ideal world, the code itself would be the clear, authoritative
reference of what it is doing.  I wish that we can remove all references
to md5 and sha1 there.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#856211: anna: please implement SHA256 verification of .udeb files

Hello!

Cyril Brulebois wrote:
> AFAICT net-retriever does the fetching and checking work?

Mayyybe...

Although with 
http://ftp.de.debian.org/debian/dists/testing/main/installer-i386/20170127/images/netboot/mini.iso
I observed md5sum and sha256sum only being executed as indicated in the
attached log.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
/usr/bin/sha256sum /tmp/net-retriever-1817-Packages
/usr/bin/sha256sum /tmp/net-retriever-1872-Packages
/usr/bin/sha256sum /tmp/net-retriever-1872-Packages
/usr/bin/sha256sum /tmp/net-retriever-1872-Packages
/usr/bin/md5sum /var/cache/anna/apt-mirror-setup_0.123_all.udeb
/usr/bin/md5sum /var/cache/anna/apt-setup-udeb_0.123_i386.udeb
/usr/bin/md5sum /var/cache/anna/base-installer_1.167_all.udeb
/usr/bin/md5sum /var/cache/anna/bootstrap-base_1.167_i386.udeb
/usr/bin/md5sum /var/cache/anna/btrfs-progs-udeb_4.7.3-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/clock-setup_0.132_i386.udeb
/usr/bin/md5sum /var/cache/anna/di-utils-mapdevfs_1.117_i386.udeb
/usr/bin/md5sum /var/cache/anna/debootstrap-udeb_1.0.88_all.udeb
/usr/bin/md5sum /var/cache/anna/dosfstools-udeb_4.1-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/e2fsprogs-udeb_1.43.4-2_i386.udeb
/usr/bin/md5sum /var/cache/anna/elilo-installer_1.31_i386.udeb
/usr/bin/md5sum /var/cache/anna/finish-install_2.75_all.udeb
/usr/bin/md5sum /var/cache/anna/fuse-udeb_2.9.7-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/libfuse2-udeb_2.9.7-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/grub-installer_1.137_i386.udeb
/usr/bin/md5sum /var/cache/anna/grub-mount-udeb_2.02~beta3-5_i386.udeb
/usr/bin/md5sum /var/cache/anna/disk-detect_1.121_i386.udeb
/usr/bin/md5sum /var/cache/anna/installation-locale_1.7_i386.udeb
/usr/bin/md5sum /var/cache/anna/jfsutils-udeb_1.1.15-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/kickseed-common_0.62_all.udeb
/usr/bin/md5sum /var/cache/anna/libbsd0-udeb_0.8.3-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/lilo-installer_1.52_i386.udeb
/usr/bin/md5sum /var/cache/anna/ata-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/btrfs-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/cdrom-core-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/efi-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/ext4-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/fat-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/firewire-core-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/isofs-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/jfs-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/kernel-image-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/loop-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/md-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/nic-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/nic-pcmcia-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/nic-shared-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/nic-usb-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/nic-wireless-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/pata-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/pcmcia-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/pcmcia-storage-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/sata-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/scsi-core-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/scsi-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum 
/var/cache/anna/usb-storage-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/xfs-modules-4.9.0-1-686-di_4.9.6-3_i386.udeb
/usr/bin/md5sum /var/cache/anna/dmsetup-udeb_1.02.137-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/libdevmapper1.02.1-udeb_1.02.137-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/liblzo2-2-udeb_2.08-1.2_i386.udeb
/usr/bin/md5sum /var/cache/anna/mdadm-udeb_3.4-4_i386.udeb
/usr/bin/md5sum /var/cache/anna/network-console_1.62_i386.udeb
/usr/bin/md5sum /var/cache/anna/nobootloader_1.47_all.udeb
/usr/bin/md5sum /var/cache/anna/ntfs-3g-udeb_2016.2.22AR.1-4_i386.udeb
/usr/bin/md5sum /var/cache/anna/open-iscsi-udeb_2.0.874-2_i386.udeb
/usr/bin/md5sum /var/cache/anna/libisns-nocrypto0-udeb_0.97-1_i386.udeb
/usr/bin/md5sum /var/cache/anna/openssh-server-udeb_7.4p1-6_i386.udeb
/usr/bin/md5sum /var/cache/anna/os-prober-udeb_1.74_i386.udeb
/usr/bin/md5sum /var/cache/anna/partconf-find-partitions_1.50_i386.udeb
/usr/bin/md5sum /var/cache/anna/libparted-fs-resize0-udeb_3.2-17_i386.udeb
/usr/bin/md5sum /var/cache/anna/libparted2-udeb_3.2-17_i386.udeb
/usr/bin/md5sum /var/cache/anna/partman-auto_137_i386.udeb
/usr/bin/md5

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Hi,

Bastian Blank wrote:
> This change breaks the existing ABI and therefor needs an ABI bump, but
> it is missing from the patch.

I agree, that should be done.  Thanks.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Hi,

Colin Watson wrote:
> Just FYI, since it's not clear from
> https://wiki.debian.org/InstallerDebacle that you know this, the
> installer in fact uses debootstrap rather than cdebootstrap to install
> the base system.

I didn't realise that, thanks.  There was still a cdebootstrap-udeb in
wheezy, so that installer is affected?  But not releases since.

base-installer seems it would (still now) use it in preference to
regular debootstrap, *if* it was available in the installer:
http://sources.debian.net/src/base-installer/1.168/debian/bootstrap-base.postinst/?hl=145#L145

Do you know any places where cdebootstrap is still used?  (It is still
having new features added in the past months, so it may not be an option
to simply remove it from the stable release).

I found a random example in "gitlab-ci-multi-runner"
http://sources.debian.net/src/gitlab-ci-multi-runner/1.10.3%2Bdfsg-1/debian/mk-prebuilt-images.sh.in/?hl=62#L62

> AFAIK debootstrap has supported SHA256 since version
> 1.0.28 in 2011.

I looked at debootstrap in sid and it seems unaffected by these issues,
yes.

> >   + allow verifiers to check both MD5 *and* SHA256, for even stronger
> > authentication in case one or both algorithms are broken
> 
> Checking both adds only negligible security (look up "multicollisions")
> and is a waste of time.

I wouldn't dismiss it for that reason, but I think it adds such
complexity that we would likely make some more serious error, if we
tried.

> The usual reason to keep support for older hash algorithms is just to
> make transitions to newer ones less painful.

Maybe it makes sense to do that on the archive side (add new hash
algorithms before removing old ones);  but to do that here, in the
consuming utility, has turned out quite harmful, in retrospect.

From this, I would conclude that cdebootstrap should have dropped
all support for MD5 when SHA1 support was added, i.e. require a SHA1
field, and fail loudly if it's not there;  and prune out all of the MD5
code (which might have avoided #856213).

I think archive utils have had plenty of time (10 years!) to add SHA256
fields, so it is reasonable now to require a SHA256 field be present,
and validate only that?

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

With that patch, reverse-deps anna and cdebootstrap shall FTBFS with:

| gcc -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -g -O2 
-fdebug-prefix-map=/home/steven/git/anna=. 
-specs=/usr/share/dpkg/pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wall -W -ggdb -Wdate-time -D_FORTIFY_SOURCE=2 
-D_GNU_SOURCE  -c -o anna.o anna.c
| anna.c: In function ‘install_modules’:
| anna.c:321:25: error: ‘di_package {aka struct di_package}’ has no member 
named ‘md5sum’
|  if (! md5sum(package->md5sum, dest_file)) {
|  ^~

| gcc -DHAVE_CONFIG_H -I. -I../../src -I..  -I../../include -Wdate-time 
-D_FORTIFY_SOURCE=2  -g -O2 
-fdebug-prefix-map=/home/steven/git/cdebootstrap-0.7.6=. 
-specs=/usr/share/dpkg/pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -std=gnu99 -c -o gpg.o ../../src/gpg.c
| ../../src/check.c: In function ‘check_deb’:
| ../../src/check.c:61:40: error: ‘di_package {aka struct di_package}’ has no 
member named ‘md5sum’
|return check_sum (target, "md5sum", p->md5sum, message);
| ^~
| ../../src/check.c: In function ‘check_packages’:
| ../../src/check.c:75:35: error: ‘di_release {aka struct di_release}’ has no 
member named ‘md5sum’
|item = di_hash_table_lookup (rel->md5sum, );
|^~

so it should be quite clear that they must implement a new hashing
algorithm;  and this makes absolutely sure they are not still using MD5
unintentionally (which was the case in #856215).

If my libdebian-installer patch is okay, I will submit the patches for
anna and cdebootstrap (bugs are already filed against them).  Hopefully
no other reverse-dependencies would be affected (because they do not use
the md5sums field, and the struct size is not changing);  though if they
do use, I'd prefer they FTBFS so that we find out.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Control: tags -1 + patch

Hi,

The regression in Bug#856215 in cdebootstrap:
"since SHA1 removal from Release file, only MD5sums are used"
could only be fixed by adding support for the SHA256 fields.

An open question is whether to preserve any support for MD5.
Keeping it would:

  + reduce potential for breakage (in case MD5 is "good enough" for some
use-case or SHA256 is still impractical)
  + allow verifiers to check both MD5 *and* SHA256, for even stronger
authentication in case one or both algorithms are broken
  - add complexity

Otherwise, dropping MD5 entirely would:

  * break reverse-dependencies (hopefully just anna, cdebootstrap) thus
*forcing* us to stop using MD5 there, and implement SHA256

I've attached only the most minimal patch to allow reverse-depends do
implement SHA256.  They must adapt to the new names of struct members
*and* remember that the hash length is now different.  (The hash data is
stored in variable-length fields but the length is not recorded in the
structs, and the has is denoted by a magic number not an enum;  that
could be made better, but requiring a much larger diff).

A follow-up commit should extend the testsuite to check parsing of the
SHA256 fields;  that also would result in a larger diff however.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/debian/changelog b/debian/changelog
index 3dd29e1..1b7fcd8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,7 +1,16 @@
 libdebian-installer (0.109) UNRELEASED; urgency=medium
 
+  [ Samuel Thibault ]
   * Fix build with gcc-7. Closes: #853489
 
+  [ Steven Chamberlain ]
+  * Parse SHA256 fields instead of MD5Sum fields in Packages files.
+  * Parse SHA256 fields instead of (no longer existing) SHA1 fields in
+Release files.
+  * In structs di_release and di_package, add new sha256 member and
+remove the md5sum member (a backward-incompatible change, this will
+force reverse-dependencies to stop using MD5 for verification).
+
  -- Samuel Thibault <sthiba...@debian.org>  Tue, 31 Jan 2017 11:09:16 +0100
 
 libdebian-installer (0.108) unstable; urgency=medium
diff --git a/include/debian-installer/package.h b/include/debian-installer/package.h
index 72d7444..e1f699d 100644
--- a/include/debian-installer/package.h
+++ b/include/debian-installer/package.h
@@ -112,7 +112,7 @@ struct di_package
   di_slist depends; /**< Any different dependency types */
   char *filename;   /**< Filename field */
   size_t size;  /**< Size field */
-  char *md5sum; /**< MD5Sum field */
+  char *sha256; /**< SHA256 field */
   char *short_description;  /**< Description field, first part*/
   char *description;/**< Description field, second part */
   unsigned int resolver;/**< @internal */
diff --git a/include/debian-installer/package_internal.h b/include/debian-installer/package_internal.h
index f6357d1..d410ce2 100644
--- a/include/debian-installer/package_internal.h
+++ b/include/debian-installer/package_internal.h
@@ -52,7 +52,7 @@ const di_parser_fieldinfo
   internal_di_package_parser_field_enhances,
   internal_di_package_parser_field_filename,
   internal_di_package_parser_field_size,
-  internal_di_package_parser_field_md5sum,
+  internal_di_package_parser_field_sha256,
   internal_di_package_parser_field_description;
 
 /**
diff --git a/include/debian-installer/release.h b/include/debian-installer/release.h
index 223a4f8..8e3c572 100644
--- a/include/debian-installer/release.h
+++ b/include/debian-installer/release.h
@@ -40,7 +40,7 @@ struct di_release
   char *origin; /**< Origin field */
   char *suite;  /**< Suite field */
   char *codename;   /**< Codename field */
-  di_hash_table *md5sum;/**< checksum fields, includes di_release_file */
+  di_hash_table *sha256;/**< checksum fields, includes di_release_file */
   di_mem_chunk *release_file_mem_chunk; /**< @internal */
 };
 
@@ -55,7 +55,7 @@ struct di_release_file
 di_rstring key; /**< @internal */
   };
   unsigned int size;/**< size */
-  char *sum[2]; /**< checksums, currently md5 and sha1 */
+  char *sum[2]; /**< checksums, currently md5 and sha256 */
 };
 
 di_release *di_release_alloc (void);
diff --git a/src/package.c b/src/package.c
index 653b5dd..82c7653 100644
--- a/src/package.c
+++ b/src/package.c
@@ -38,7 +38,7 @@ void di_package_destroy (di_package *package)
   di_free (package->archi

Bug#856211: anna: please implement SHA256 verification of .udeb files

Source: anna
Version: 1.57
Severity: grave
Tags: security
X-Debbugs-Cc: secur...@debian.org
User: debian-rele...@lists.debian.org
Usertags: bsp-2017-02-de-Berlin
Control: block -1 by 856210

Hi,

To date, anna still only implements MD5 verification of .udeb files,
despite its formal deprecation as a digital signature algorithm by
RFC6151 (2011) and recommendations of academic literature years prior.

The files are typically downloaded via insecure HTTP transport, so the
checksum verification is critical for the security of the installed
system.  stretch is expected to be a supported release until 2022.  So
I'm tentatively filing this bug as RC-severity.

Further context and an overview of related bugs will be published at:
https://wiki.debian.org/InstallerDebacle

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#856210: libdebian-installer: please parse SHA256 field and add it to di_* structs

Source: libdebian-installer
Version: 0.108
Severity: serious
Tags: security
X-Debbugs-Cc: secur...@debian.org
User: debian-rele...@lists.debian.org
Usertags: bsp-2017-02-de-Berlin

Hi,

The 'etch' release (2007) added to the Release file, a field for SHA256
sums to authenticate Packages files.  But to date, libdebian-installer
does not parse it, so anna (which fetches .udeb installer component) and
cdebootstrap (which fetches .deb base system packages) can not yet
verify the SHA256 sums.

http://sources.debian.net/src/libdebian-installer/0.108/include/debian-installer/release.h/#L43
http://sources.debian.net/src/libdebian-installer/0.108/include/debian-installer/release.h/#L58
http://sources.debian.net/src/libdebian-installer/0.108/include/debian-installer/package.h/#L115

Further context and an overview of related bugs will be published at:
https://wiki.debian.org/InstallerDebacle

This bug is not itself RC, but it will be a blocking issue for RC bugs
I'm about to file.

I intend to submit a patch for this shortly.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: Bug#852215: FTBFS on non-release architectures

2017-02-08 Thread Steven Chamberlain

Cyril Brulebois wrote:
> [...] kfreebsd-* (which currently FTBFS anyway…).

Regarding that, is it okay I commit to sid this specific change for
kfreebsd-amd64:

--- a/build/Makefile
+++ b/build/Makefile
@@ -149,7 +149,7 @@ ifeq ($(DEB_HOST_ARCH),kfreebsd-i386)
 MFSROOT_LIMIT := 42m
 else ifeq ($(DEB_HOST_ARCH),kfreebsd-amd64)
 # See Bug#783773 for derivation.
-MFSROOT_LIMIT := 74m
+MFSROOT_LIMIT := 78m
 endif
 
 define mkfs.ufs1

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: Please dak copy-installer for jessie-kfreebsd

2016-10-21 Thread Steven Chamberlain

Hi,

Steven Chamberlain wrote:
> I have uploaded debian-installer/20150422+kbsd8u2+deb8u4 [...]

That has been superseded already by 20150422+kbsd8u3, which seems to
have already been processed through BYHAND, perhaps automatically.

You may wish to remove 20150422+kbsd8u2+deb8u4 from the BYHAND queue
unless that is automatically decrufted sometime.

> Please could you 'dak copy-installer' as appropriate?

Also that was not the right thing to ask.  It merely needed unpacking;
dak's scripts/debian/byhand-di does this.  AFAICT the procedure is (in
case this ever needs to be done by hand in future) :

# cd /srv/ftp-master.debian.org/queue/byhand
# /srv/ftp-master.debian.org/dak/scripts/debian/byhand-di 
debian-installer-images_20150422+kbsd8u3_kfreebsd-amd64.tar.gz 20150422+kbsd8u3 
kfreebsd-amd64 debian-installer_20150422+kbsd8u3_kfreebsd-amd64.changes 
jessie-kfreebsd-proposed-updates
# /srv/ftp-master.debian.org/dak/scripts/debian/byhand-di 
debian-installer-images_20150422+kbsd8u3_kfreebsd-amd64.tar.gz 20150422+kbsd8u3 
kfreebsd-i386 debian-installer_20150422+kbsd8u3_kfreebsd-i386.changes 
jessie-kfreebsd-proposed-updates

Thanks,
Regards,
-- 
Steven Chamberlain
stev...@debian.org

signature.asc
Description: Digital signature

Bug#840523: debian-installer: undefined source format; .git/ may end up in source tarball

2016-10-21 Thread Steven Chamberlain

Control: severity -1 wishlist

Cyril Brulebois wrote:
> debcheckout's output lets you know you're getting a git repository. I'm
> not sure running debuild -S without -i/-I makes this an important bug in
> debian-installer…

Ok, I see.  This is perhaps only a desirable feature, then, for anyone
who likes to work that way with Debian packages.

> I don't like the idea of unneeded boilerplate in source packages just
> because dpkg developers want to force v3 onto people so badly.

I see.

> In case someone wants to merge this, I would like to see this very
> carefully reviewed for possible side effects. Getting an error during
> the next release, or worse a regression only seen afterwards, would
> really be annoying.

Since I needed to make a new jessie-kfreebsd upload of debian-installer,
stripping the .git/ directory out of the source tarball, I can try out
this patch (3.0 native and .tar.xz) with that to begin with.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#840523: debian-installer: undefined source format; .git/ may end up in source tarball

2016-10-12 Thread Steven Chamberlain

Package: src:debian-installer
Version: 20160704
Severity: important
Tags: patch

Hi!

Suppose a debian-installer source package is produced this way:

$ debcheckout debian-installer
$ cd debian-installer
$ debuild -S
[...]
dpkg-source: warning: no source format specified in debian/source/format, see 
dpkg-source(1)
dpkg-source: warning: source directory 'debian-installer' is not 
- 'debian-installer-20160704'
dpkg-source: info: using source format '1.0'
dpkg-source: info: building debian-installer in debian-installer_20160704.tar.gz
dpkg-source: info: building debian-installer in debian-installer_20160704.dsc

In source format 1.0, dpkg-source does *not* exclude the .git/ directory
from the generated tarball!  So if you built the source package that way,
your Git working tree's refs and config are all included in the upload,
which happened with my uploads here:
http://ftp.debian.org/debian/pool/main/d/debian-installer/debian-installer_20150422+kbsd8u2.tar.gz
http://ftp.debian.org/debian/pool/main/d/debian-installer/debian-installer_20150422+kbsd8u2+deb8u4.tar.gz

Note that source format 1.0 is only used because one is not specified in
debian/source/format.  dpkg-source(1) recommends to specify a version and
the lack of a debian/source/format file may be considered an error in
future.

dpkg-source(1) also recommends choosing a newer format.  3.0 (native)
by default already excludes VCS directories such as .git/ from the
generated tarball, already fixing the issue above.

3.0 (native) does however default to .tar.xz compression, rather than
.tar.gz as used at the moment.  I'm not sure if that may be an issue for
other tools.  Maybe they should be fixed in that case.  Or _if_ it's
preferred to still use .tar.gz, that could be specified in
debian/source/options:
compression = "gzip"

Attached is my proposed patch, for consideration.

Thanks!

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 10.1-0-amd64
Locale: LANG=en_GB.UTF-8, LC_CTYPE= (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
>From d74e62562b5814a496ff485c4ba4f6bbfa542a50 Mon Sep 17 00:00:00 2001
From: Steven Chamberlain <ste...@pyro.eu.org>
Date: Wed, 12 Oct 2016 12:47:02 +
Subject: [PATCH] Switch to dpkg-source format 3.0 (native)

By default this will also exclude VCS directories like .git/ from the
generated source tarball.

This changes the default compression from .tar.gz to .tar.xz also.
---
 debian/source/format | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 debian/source/format

diff --git a/debian/source/format b/debian/source/format
new file mode 100644
index 000..89ae9db
--- /dev/null
+++ b/debian/source/format
@@ -0,0 +1 @@
+3.0 (native)
-- 
2.1.4

Please dak copy-installer for jessie-kfreebsd

2016-10-12 Thread Steven Chamberlain

Dear FTP masters,

I have uploaded debian-installer/20150422+kbsd8u2+deb8u4 to
stable-kfreebsd-proposed-updates and the binaries are built+uploaded on
both kfreebsd-* arches.  These are now waiting in BYHAND and I think
just need to be installed now.

Please could you 'dak copy-installer' as appropriate?

I think the destination suite is stable-kfreebsd-proposed-updates:
http://ftp.debian.org/debian/dists/stable-kfreebsd-proposed-updates/main/installer-kfreebsd-amd64/
Possibly the source suite is also stable-kfreebsd-proposed-updates?

Many thanks,
Regards,
-- 
Steven Chamberlain
stev...@debian.org


signature.asc
Description: Digital signature

Re: Bug#830894: override: initscripts:admin/optional

2016-07-27 Thread Steven Chamberlain

Hi,

Michael Biebl wrote:
> Afaics, there weren't any concerns raised by our -hurd@ and -bsd@
> maintainers so far. If you need more time to evaluate the change, please
> speak up now. Otherwise I'd ask the ftp-masters to proceed with
> implementing the change.

I don't foresee any problems.  I presume this doesn't affect the
installer;  when we start to build live images we'll ensure the
necessary init system packages get installed;  if any packages need to
be manually specified now when debootstrapping a BSD jail, I will update
the Wiki instructions (we don't actually need /sbin/init for those, so
this change is probably an improvement)..

Thanks for heads-up.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Re: [codeh...@debian.org: Bug#828854: RM: debian-installer [kfreebsd-amd64 kfreebsd-i386] -- RoQA; Fails to build on kfreebsd]

2016-06-29 Thread Steven Chamberlain

Hello,

Steve McIntyre wrote:
> [...] we're proposing to remove
> them. We're *fairly* sure this shouldn't cause problems for you as
> these binaries are ancient - please confirm!

I'm quite sure this is okay.  The binaries we care most about are in the
separate jessie-kfreebsd-proposed-updates suite.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

[SUSPECTED SPAM]

2016-06-26 Thread Steven Chamberlain

Bug#821249: debian-installer: may ignore mkinitramfs failure

2016-04-16 Thread Steven Chamberlain

Package: src:debian-installer
Version: 20160112
Severity: important
Tags: patch pending
User: debian-...@lists.debian.org
Usertags: kfreebsd

Hi,

d-i daily builds of kfreebsd-amd64 started to have problems
beginning with:
https://d-i.debian.org/daily-images/kfreebsd-amd64/20160406-00:06/build_netboot-gtk-10.log

When makefs created the initramfs, it had an error, but left a
broken/truncated output file.  Within the case statement, an error there
is ignored and gzip will try to compress the output file.  If that
succeeds, it returns a success status to make, and the build
continues...

This could affect linux too.  A patch is attached;  I'll commit it as
soon as the BTS gives me a bug number.  Thanks!

| case ufs1 in \
| ext2) \
| genext2fs -d ./tmp/netboot-gtk-10/tree -b `expr $(du -s 
./tmp/netboot-gtk-10/tree | cut -f 1) + $(expr $(find ./tmp/netboot-gtk-10/tree 
| wc -l) \* 2)` -m 0 ./tmp/netboot-gtk-10/initrd; \
| e2fsck -fy ./tmp/netboot-gtk-10/initrd || true; \
|   gzip -n -v9f ./tmp/netboot-gtk-10/initrd; \
| ;; \
| initramfs) \
| (cd ./tmp/netboot-gtk-10/tree && find . | sort | cpio --quiet -o -H 
newc) >  ./tmp/netboot-gtk-10/initrd; \
|   gzip -n -v9f ./tmp/netboot-gtk-10/initrd; \
| ;; \
| jffs2) \
|  -d ./tmp/netboot-gtk-10/tree -o ./tmp/netboot-gtk-10/initrd.gz; \
| ;; \
| ufs1) \
| sh -c 'makefs -t ffs -T 1459901061 -s 68m -f 3000 -o 
minfree=0,version=1 $0 ./tmp/netboot-gtk-10/tree' ./tmp/netboot-gtk-10/initrd; \
|   gzip -n -v9f ./tmp/netboot-gtk-10/initrd; \
| ;; \
| *) \
|   echo "Unsupported filesystem type"; \
|   exit 1 ;; \
| esac
| Calculated size of `./tmp/netboot-gtk-10/initrd': 71303168 bytes, 4595 inodes
| Extent size set to 8192
| ./tmp/netboot-gtk-10/initrd: 68.0MB (139264 sectors) block size 8192, 
fragment size 1024
|   using 2 cylinder groups of 52.26MB, 6689 blks, 3584 inodes.
| super-block backups (for fsck -b #) at:
|  32, 107056,makefs: Writing inode 1578 
(./tmp/netboot-gtk-10/tree/sbin/devd), bytes 188416 + 8192: No space left on 
device
| 
| Populating `./tmp/netboot-gtk-10/initrd'
| ./tmp/netboot-gtk-10/initrd:   61.6% -- replaced with 
./tmp/netboot-gtk-10/initrd.gz

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 10.1-0-amd64
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
>From 93af24652f09144720d730a8de7f0e5e574c9947 Mon Sep 17 00:00:00 2001
From: Steven Chamberlain <ste...@pyro.eu.org>
Date: Sun, 17 Apr 2016 00:09:33 +0100
Subject: [PATCH] Handle error creating initramfs (Closes: #xx)

The case statement returns only the last-executed command's status,
potentially ignoring errors creating the initramfs.  Use `set -e` so
that any error is returned immediately to make.
---
 build/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/build/Makefile b/build/Makefile
index ec1953b..cff44e0 100644
--- a/build/Makefile
+++ b/build/Makefile
@@ -707,6 +707,7 @@ endif
 
 	install -d $(TEMP)
 
+	set -e; \
 	case $(INITRD_FS) in \
 	ext2) \
 		$(genext2fs) $(TEMP)/initrd; \
-- 
1.8.4.rc3

Re: Missing d-i daily builds on kfreebsd-*

2016-03-09 Thread Steven Chamberlain

Steven Chamberlain wrote:
> Cyril Brulebois wrote:
> > I've noticed kfreebsd-* are lacking d-i daily builds. Looking on the
> > porterboxes for both archs, the issue is the following:

> There were many things broken by [new glibc] in sid, kfreebsd-i386
> especially.

kfreebsd-amd64 was fixed yesterday by a p11-kit binNMU (and the nightly
d-i build succeeded).

kfreebsd-i386 was fixed today by Aurelien's new glibc upload and the
p11-kit binNMU.  I don't think that'll be dinstalled in time for
tonight's d-i daily build, but it should be okay tomorrow night.

Thanks for mentioning it.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Re: Missing d-i daily builds on kfreebsd-*

2016-03-08 Thread Steven Chamberlain

Hi KiBi,

Cyril Brulebois wrote:
> I've noticed kfreebsd-* are lacking d-i daily builds. Looking on the
> porterboxes for both archs, the issue is the following:
> | The following packages have unmet dependencies:
> |  librsvg2-bin : Depends: libgtk-3-0 (>= 3.2.0) but it is not going to be 
> installed
> |  wget : Depends: libgnutls30 (>= 3.4.2) but it is not going to be installed
> | E: Unable to correct problems, you have held broken packages.

The timestamps at
https://d-i.debian.org/daily-images/daily-build-overview.html#kfreebsd-amd64
suggest these broke after the glibc/2.22-1 upload.

There were many things broken by this in sid, kfreebsd-i386 especially.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: Bug#805321: debian-installer: builds unreproducible netboot images

2016-01-10 Thread Steven Chamberlain

Hi KiBi,

Please could you or I:
git cherry-pick ff722581b71778707551433e0b302e4c3a341619
into master, as-is?

As described by:
https://anonscm.debian.org/cgit/d-i/debian-installer.git/commit/?id=ff722581b71778707551433e0b302e4c3a341619
this is needed for kfreebsd netboot images to be reproducible.  The
required functionality is provided by makefs 20100306-6, now in
sid+stretch.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#571136: Fwd: Re: Bug#571136: please remove useless devices from devices.tar.gz

2016-01-10 Thread Steven Chamberlain

Hi,

Marco d'Itri wrote:
> On Jan 10, Cyril Brulebois <k...@debian.org> wrote:
> > We have a bug report with a patch by Marco against debootstrap (see
> > attachment), which changes how devices are generated; I can't really
> > tell how much this might affect all of you (especially with debootstrap
> It is not supposed to, since both hurd and kfreebsd already used 
> a different method to manage /dev.

Yes, the code Marco changed cannot be reached when HOST_OS is kfreebsd*,
freebsd* or hurd*.  kfreebsd devfs provides all the device nodes without
manually creating any or using devices.tar.gz;  even for more exotic use
cases like BSD jails.  hurd appears to have something equivalent.

Thanks for letting us know.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#805321: [Reproducible-builds] Bug#805321: debian-installer: builds unreproducible netboot images

2015-11-27 Thread Steven Chamberlain

Hi,

Cyril Brulebois wrote:
> I've cherry-picked 3 patches from there onto master locally and I'm
> currently running diffoscope to see how that goes (and it's taking
> ages…):

I'm guessing the initrd would differ if the Linux tool to generate it
stores timestamps.  If its compressed size varies much due to those
differences, the .iso block numbers may vary as a result.

> FWIW, I'm not exactly entirely convinced by the exporting of the
> SOURCE_DATE_EPOCH variable from debian/rules; all other variables have
> been passed without exporting so I'm wondering if we shouldn't adapt
> this to behave like other variables, reducing possible surprise for
> users.

Just to explain that -- if it's defined in the environment, it requires
no special handling and doesn't need to be (re-)exported.  I think this
is maybe the case now for dpkg-buildpackage in sid?

If the dpkg-buildpackage environment doesn't have SOURCE_DATE_EPOCH
(e.g. jessie), debian/rules sets it to the correct value, and so must
export that.

Or (for jessie or sid), in case build/Makefile is used directly, outside
of a package build, we set SOURCE_DATE_EPOCH to a dummy value ("now") if
undefined (since ../debian/changelog may not exist), which we need
when calling makefs from within that Makefile.  We export it for use by
gen-tarball to avoid duplication there.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#805321: [Reproducible-builds] Bug#805321: debian-installer: builds unreproducible netboot images

2015-11-22 Thread Steven Chamberlain

Hi,

I rewrote the patches according to KiBi's feedback and they are
now uploaded to our jessie-kfreebsd suite, and this Git branch:
https://anonscm.debian.org/cgit/d-i/debian-installer.git/log/?h=jessie-kfreebsd

In my own testing on ZFS, file ordering was still an issue for the
makefs tool that builds the initrd.  But if I were to try again
on UFS, I hope to be able to reproduce the entire
netboot-installer-images tarball as built by the buildds.

This tarball includes bits that are bundled onto the official release
images by debian-cd tools.  Making this reproducible is a prerequisite
for someday having reproducibly-built official release images.

I could merge these patches into sid if they seem okay?  The only
commit that should not be merged is this one, which is specific to
jessie-kfreebsd and must be slightly changed for sid:
kfreebsd: use makefs -T to clamp timestamps

I expect that Linux d-i builds will have some reproducibility issues
in whatever generates the initrd or ISOs, but I may look into that after
the jessie-kfreebsd release is done.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#805321: debian-installer: builds unreproducible netboot images

2015-11-16 Thread Steven Chamberlain

Package: debian-installer
Version: 20150422
Severity: wishlist
Tags: patch
User: reproducible-bui...@lists.alioth.debian.org
Usertag: timestamps fileordering infrastructure
X-Debbugs-Cc: reproducible-bui...@lists.alioth.debian.org
X-Debbugs-Cc: debian-...@lists.debian.org

Hi!

The debian-installer package build produces netboot.tar.gz and
the mini.iso netboot install media.  It doesn't do this in an easily
reproducible way:

  * the d-i initrd/mfsroot is a filesystem image, having variable
mtime/ctime/atime timestamps from package build time;
  * likewise in the generated mini.iso;
  * netboot.tar.gz also has varying timestamps;  the order of files
may also vary depending on the filesystem;
  * likewise in the cd info tarball;
  * likewise in the debian-installer-images tarball;
  * all gzipped outfile files have a timestamp in the header.

I have a patch aimed at jessie-kfreebsd that should fix all of the
above.  It should be possible to do the same in sid with much less
code, due to new GNU tar features and other reproducible builds work.

I've 'clamped' timestamps to be no later than the most recent
debian/changelog entry date.  That way, the non-useful timestamps
from during the build are adjusted to a constant value.  Older
timestamps, actually indicating how old a file is, are untouched.
The BUILD_DATE, actually the package version number, is unchanged.

Specifically on kfreebsd, the generated mfsroot is a ffs filesystem
having file atimes, and another timestamp in the filesystem superblock.
I intend to patch makefs so that it can clamp timestamps to a given
SOURCE_DATE_EPOCH.

Besides a file ordering issue in makefs, all output files including
netboot.tar.gz and mini.iso then seem to be reproducible for
jessie-kfreebsd, at least.  :)

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#805321: debian-installer: builds unreproducible netboot images

2015-11-16 Thread Steven Chamberlain

Attached is my jessie-kfreebsd implementation.  As I said, it should be
much cleaner to implement this in sid with newer GNU tar.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/build/Makefile b/build/Makefile
index ec5a084..6261a4d 100644
--- a/build/Makefile
+++ b/build/Makefile
@@ -56,7 +56,7 @@
 # Add to PATH so dpkg will always work, and so local programs will be found.
 PATH := util:$(PATH):/usr/sbin:/sbin
 EATMYDATA = $(shell which eatmydata 2>/dev/null)
-GZIP = $(shell which pigz gzip | head -1)
+GZIP = $(shell which pigz gzip | head -1) -n
 
 # We don't want this to be run each time we re-enter.
 ifndef DEB_HOST_ARCH
@@ -149,7 +149,7 @@ MFSROOT_LIMIT := 68m
 endif
 
 define mkfs.ufs1
-  sh -c 'makefs -t ffs -s $(MFSROOT_LIMIT) -f 3000 -o minfree=0,version=1 $$0 ${TREE}'
+  sh -c 'makefs -t ffs -T $(SOURCE_DATE_EPOCH) -s $(MFSROOT_LIMIT) -f 3000 -o minfree=0,version=1 $$0 ${TREE}'
 endef
 
 define e2fsck
@@ -803,7 +803,14 @@ $(TEMP_MINIISO): $(TEMP_BOOT_SCREENS) arch_miniiso
 
 # various kinds of information, for use on debian-cd isos
 $(DEBIAN_CD_INFO): $(TEMP_BOOT_SCREENS) $(TEMP_CD_INFO_DIR)
-	(cd $(TEMP_CD_INFO_DIR); tar czf - .) > $@
+	# Clamp timestamps to be no newer than last changelog entry, see
+	# https://wiki.debian.org/ReproducibleBuilds/TimestampsInTarball
+	find $(TEMP_CD_INFO_DIR) -newermt "@$(SOURCE_DATE_EPOCH)" -print0 | xargs -0r touch --no-dereference --date="@$(SOURCE_DATE_EPOCH)"
+	# Create tarball with files sorted in a stable order, see
+	# https://wiki.debian.org/ReproducibleBuilds/FileOrderInTarballs
+	# and without timestamp in the gzip header, see
+	# https://wiki.debian.org/ReproducibleBuilds/TimestampsInGzipHeaders
+	( cd $(TEMP_CD_INFO_DIR) && find . -print0 | LC_ALL=C sort -z | GZIP=-n tar --no-recursion --null -T - -czf -) > $@
 	update-manifest $@ $(MANIFEST-DEBIAN_CD_INFO)
 
 # a directory full of files for netbooting
@@ -822,7 +829,14 @@ $(NETBOOT_TAR): $(TEMP_NETBOOT_DIR)
 	# Create an version info file.
 	echo 'Debian version:  $(DEBIAN_VERSION)' > $(TEMP_NETBOOT_DIR)/version.info
 	echo 'Installer build: $(BUILD_DATE)' >> $(TEMP_NETBOOT_DIR)/version.info
-	(cd $(TEMP_NETBOOT_DIR); tar czf - .) > $@
+	# Clamp timestamps to be no newer than last changelog entry, see
+	# https://wiki.debian.org/ReproducibleBuilds/TimestampsInTarball
+	find $(TEMP_NETBOOT_DIR) -newermt "@$(SOURCE_DATE_EPOCH)" -print0 | xargs -0r touch --no-dereference --date="@$(SOURCE_DATE_EPOCH)"
+	# Create tarball with files sorted in a stable order, see
+	# https://wiki.debian.org/ReproducibleBuilds/FileOrderInTarballs
+	# and without timestamp in the gzip header, see
+	# https://wiki.debian.org/ReproducibleBuilds/TimestampsInGzipHeaders
+	( cd $(TEMP_NETBOOT_DIR) && find . -print0 | LC_ALL=C sort -z | GZIP=-n tar --no-recursion --null -T - -czf -) > $@
 	update-manifest $@ $(MANIFEST-NETBOOT_TAR) $(UDEB_LISTS)
 
 $(TEMP_BOOT_SCREENS): arch_boot_screens
diff --git a/build/config/x86.cfg b/build/config/x86.cfg
index 3caadd2..b0fc9a2 100644
--- a/build/config/x86.cfg
+++ b/build/config/x86.cfg
@@ -332,6 +332,11 @@ arch_miniiso: x86_syslinux x86_grub_efi
 			| todos > $(TEMP_CD_TREE)/win32-loader.ini; \
 	fi
 
+	# Clamp timestamps to be no newer than last changelog entry, see
+	# https://wiki.debian.org/ReproducibleBuilds/TimestampsInTarball
+	find $(TEMP_CD_TREE) -newermt "$(SOURCE_DATE)" -print0 \
+	 | xargs -0r touch --no-dereference --date="$(SOURCE_DATE)"
+
 	if [ "$(GRUB_EFI)" = y ]; then \
 		xorriso -as mkisofs -r -J -b isolinux.bin -c boot.cat \
 			-no-emul-boot -boot-load-size 4 -boot-info-table \
diff --git a/debian/changelog b/debian/changelog
index 42aed37..09c8a02 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,21 @@
+debian-installer (20150422+kbsd8u2) jessie-kfreebsd; urgency=medium
+
+  * Improve reproducibility of debian-installer netboot images:
+(Closes: #805321)
+- clamp timestamps in the d-i ramdisk to be no later than
+  the most recent debian/changelog entry of this package
+  - raise makefs dependency on >= 20100306-5+kbsd8u1
+- clamp timestamps in the mini.iso similarly
+- clamp timestamps in the netboot tarball;  store files in a
+  stable order
+- clamp timestamps in the cd info tarball;  store files in a
+  stable order
+- clamp timestamps in the output debian-installer-images tarball;
+  store files in a stable order
+- disable timestamps in gzip output (e.g. initrd.gz and tarballs)
+
+ -- Steven Chamberlain <ste...@pyro.eu.org>  Tue, 10 Nov 2015 21:38:46 +
+
 debian-installer (20150422+kbsd8u1) jessie-kfreebsd; urgency=medium
 
   * Rebuild using udebs from the jessie-kfreebsd suite, also using
diff --git a/debian/control b/debian/control
index 100ca5a..6f4df5b 100644
--- a/debian/control
+++ b/debian/control
@@ -162,7 +162,7 @@ Build-Depends:
 #		arc

Bug#805321: debian-installer: builds unreproducible netboot images

2015-11-16 Thread Steven Chamberlain

Hi KiBi,

Many thanks for reviewing this.

Cyril Brulebois wrote:
> Please make sure not to depend on features which are not found in stable
> (I'm not entirely sure about oldstable at this point), which might hinder
> our ability to cherry-pick bits and pieces from master to jessie.

I think I could make that easier by splitting into smaller commits:

  * parts only needed for kfreebsd
  * parts more appropriate for stable, oldstable, etc.
  * parts only appropriate for sid (or just not use new GNU tar
features yet)

> > --- a/build/Makefile
> > +++ b/build/Makefile
> > @@ -56,7 +56,7 @@
> >  # Add to PATH so dpkg will always work, and so local programs will be 
> > found.
> >  PATH := util:$(PATH):/usr/sbin:/sbin
> >  EATMYDATA = $(shell which eatmydata 2>/dev/null)
> > -GZIP = $(shell which pigz gzip | head -1)
> > +GZIP = $(shell which pigz gzip | head -1) -n
> 
> I think I already added -n to a bunch of calls. Not sure whether adding
> it here once and for all would be better than adding it where it's
> missing though. Anyway, not my biggest question/comment/concern here.

Seems reasonable to factor out and put it here.  If we don't, someone
may add a new $GZIP call later, forget -n and make it unreproducible
again.

Although it is a macro here, GZIP is also the name of an environment
variable used by gzip (but not pigz?), which is likely confusing.  And
the later tar invocations call gzip (not pigz) in quite a few places
regardless of the GZIP macro contents;  those do look at the GZIP
environment variable though.

> I think those 3(.5) occurrences really should be factorized, especially
> given the logic is the same: replacing "cd foo && tar bar ." with more
> code. Somewhere under build/util would probably be suitable.

I agree.  It is a very common pattern in other Debian packages too, and
it often needs patching for reproducibility.

> > --- a/build/config/x86.cfg
> > +++ b/build/config/x86.cfg
> > @@ -332,6 +332,11 @@ arch_miniiso: x86_syslinux x86_grub_efi
> > | todos > $(TEMP_CD_TREE)/win32-loader.ini; \
> > fi
> >  
> > +   # Clamp timestamps to be no newer than last changelog entry, see
> > +   # https://wiki.debian.org/ReproducibleBuilds/TimestampsInTarball
> > +   find $(TEMP_CD_TREE) -newermt "$(SOURCE_DATE)" -print0 \
> > +| xargs -0r touch --no-dereference --date="$(SOURCE_DATE)"
> > +

> [...] above is using SOURCE_DATE, which is undefined as
> far as I can tell since SOURCE_DATE_EPOCH is what's getting defined.
> Maybe it should call the same new util (with different parameters since
> we only need touch here, and no tar call)?

That's a typo, and was buggy - it would touch all timestamps, not just
the ones necessary.  I may drop this part as I don't think it's needed
any more with the newer makefs, which will clamp the timestamps itself.

> Finally, not everything is built using debian/rules targets (with or
> without dpkg-buildpackage). One should still be able to just run e.g.:
> “make -C build build_netboot-gtk USE_UDEBS_FROM=sid”
> 
> See BUILD_DATE handling, for example. We end up with a default setting
> through:
> |build/config/common:BUILD_DATE ?= $(shell date -u '+%Y%m%d-%H:%M')

That would not be reproducible, then!  (it is embedded in the tarballs)

> [ Please note that calling $(shell dpkg-parsechangelog -SDate) to set
> SOURCE_DATE_EPOCH there would only work when building from the toplevel
> directory, and not from the build/ subdirectory for example. ]

If it's anyway not going to be reproducible, we could similarly fall
back to a SOURCE_DATE_EPOCH ?= now;  or the caller could choose to
provide them.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#801161: anna: handle multiple udeb versions in Packages file

2015-10-06 Thread Steven Chamberlain

Package: anna
Version: 1.52
Severity: wishlist
Tags: patch
Control: block 345408 by -1

Hi,

If we had multiple udeb sources (Bug#345419), it could mean that
anna reads a Packages file having multiple versions for some udebs.
It's very easy to make net-retriever generate such a file, combining
multiple suites, e.g. stable + stable-proposed-updates;  or maybe
stable + stable-backports.

Currently anna can't handle this - it tries to download and install
those udebs multiple times.  My patch allows anna to disregard
duplicates and keep only the last-listed entry for each package.  It
seems to work great.  In the usual situation (only one udeb source) it
should make no modification.

An earlier version of this patch did try to compare versions and
pick the latest one, but libdebian-installer's parsing seems broken
so didn't work.  (I think it parses hello/1 and hello/2 but returns
a linked list having hello/2 and hello/2.  So it is necessary anyway
for the Packages file to be ordered with preferred versions last, for
now).

Also not working yet, freeing memory for the orphaned package and node
data structures.  I think my code was right, but freeing either one
triggers an immediate segfault.  I went to some lengths to get core
dumps and debug symbols, but couldn't get a readable backtrace.  I'm
just nulling the pointers for now.

Would this be suitable for sid as-is or is further work needed?
Thanks!

-- System Information:
Debian Release: 7.8
  APT prefers stable-kfreebsd
  APT policy: (500, 'stable-kfreebsd'), (500, 'oldstable-proposed-updates'), 
(500, 'oldstable')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
diff --git a/anna.c b/anna.c
index 4b68816..1dd4461 100644
--- a/anna.c
+++ b/anna.c
@@ -8,6 +8,49 @@ static char *running_kernel = NULL;
 static const char *subarchitecture;
 static int quiet = 0, verbose = 0;
 
+/* If any package name appears more than once in the Packages file,
+ * keep only the last one. */
+void remove_duplicates (di_packages *packages) {
+	di_slist_node *n1, *n2, *n2_prev;
+	di_package *p1, *p2;
+
+	for (n1 = packages->list.head; n1; n1 = n1->next) {
+		n2_prev = n1;
+		/* compare n1 to each node after it, as n2 */
+		for (n2 = n1->next; n2; n2 = n2->next) {
+			/* compare package name */
+			p1 = n1->data;
+			p2 = n2->data;
+			if (strcmp(p1->package, p2->package) != 0) {
+/* keep this node;  remember the previous one */
+n2_prev = n2;
+continue;
+			}
+
+			di_log (DI_LOG_LEVEL_DEBUG,
+			"ignoring duplicate Packages entry for %s",
+			p1->package);
+
+			/* delete p1, move p2 into n1 */
+/* XXX:			di_package_destroy(p1); */
+			n1->data = p1 = p2;
+			n2->data = p2 = NULL;
+
+			/* update list bottom, if n2 was the last entry */
+			if (packages->list.bottom == n2) {
+packages->list.bottom = n2_prev;
+			}
+
+			/* unlink n2 from the list and delete it */
+			n2_prev->next = n2->next;
+/* XXX:			di_free(n2); */
+
+			/* rewind to the previous node */
+			n2 = n2_prev;
+		}
+	}
+}
+
 di_packages *get_packages (void) {
 	di_packages_allocator *packages_allocator = di_system_packages_allocator_alloc();
 	di_packages *packages = retriever_packages(packages_allocator);
@@ -26,6 +69,7 @@ di_packages *get_packages (void) {
 		}
 	}
 
+	remove_duplicates(packages);
 	return packages;
 }

Bug#801161: anna: handle multiple udeb versions in Packages file

2015-10-06 Thread Steven Chamberlain

Hi,

Cyril Brulebois wrote:
> TBH, I'm not sure whether this should be dealt with in anna or in
> net-retriever. [...]
> 
> ISTR some shell dance in net-retriever from my old patchset; having some
> C code in anna instead doesn't seem crazier.

Managing groups of RFC822 fields in shell sounded a bit of a nightmare.
We have the C functions in libdebian-installer (some used by anna) to be
able to do this properly, and even the proper version compare algorithm.
(Though I'd still say, it's odd d-i is not built from higher-level OO
scripting languages yet).

This might be better handled by libdebian-installer when parsing the
Packages file - it already uses hash tables internally - so was
surprised that it didn't already return just one record per unique
package name.  As I mentioned that's slightly buggy and maybe ought
to be fixed anyway.

Although if anna has this, handling multiple versions of udebs in
Packages could be seen as a feature.  Some other tool could run after
net-retriever to append extra things to it.  I think it's closer to
how APT works.  And it feels like the exact logic of choosing which
version to install, belongs in anna anyway, and you'd need to know all
the available udeb versions at that point.  It can also be verbose
about how it makes the choice.

Thanks!
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Re: Testing d-i for stable-p-u

2015-10-05 Thread Steven Chamberlain

Steven Chamberlain wrote:
> Next, it would be nice to test that the base system in stable-p-u is
> still installable.  d-i's debootstrap can't seem to do this as it
> doesn't recognise "-proposed-updates" suites, so would need a patch to
> add those.  I tried that but ran into some other issue next.

Oops, of course it doesn't make sense to do that.  stable-p-u contains
the updated packages and not the whole archive.

I can't think of an obvious way to make anna (udeb fetcher) and
d-i debootstrap use stable-p-u in addition to stable.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Testing d-i for stable-p-u

2015-10-05 Thread Steven Chamberlain

Hi!

Does anyone test d-i for stable-proposed-updates?  If so, how?
It seems rather difficult currently.

You could build the d-i jessie branch from Git, or get release-candidate
stable-p-u images from
http://ftp.debian.org/debian/dists/jessie-proposed-updates/main/installer-amd64/current/images/

but choose-mirror will default to installing jessie.  To override it,
you may need to use preseed/early_command or execute in a shell:

sed -i 's/$/-proposed-updates/' /etc/default-release

If you don't do that, the installer may fetch some of its udebs from
stable, rather than newer ones in stable-p-u.  (Or otherwise you could
build a custom d-i image having all the latest udebs from stable-p-u,
but I've become tired of that).

Next, it would be nice to test that the base system in stable-p-u is
still installable.  d-i's debootstrap can't seem to do this as it
doesn't recognise "-proposed-updates" suites, so would need a patch to
add those.  I tried that but ran into some other issue next.

Where I'm going with this is, that jenkins.debian.net does great QA of
the installer and base system from stable.  But doing regular c-i tests
of stable isn't as interesting, as being able to find bugs *before* they
are introduced in a point release.

Thanks!
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: Testing d-i for stable-p-u

2015-10-05 Thread Steven Chamberlain

Hi,

Cyril Brulebois wrote:
> Depending on particular changes I'd like to check (e.g. grub-installer),
> I indeed end up building an image with debian-cd.

I think this is what I must do.  It just feels a bit excessive for
recreational testing, or even within a Jenkins job.

Jenkins also tests PXE netboot installs, and I see no easy way to
test udebs from stable-p-u in that context.

> You're missing net-retriever in this picture. You might find the
> following somewhat helpful:
>   https://lists.debian.org/debian-cd/2014/01/msg00026.html

Yes, there are some similarities with that use-case.

I already tried changing /etc/udebs-source, but that either gives me
stable's udebs, or an incomplete set of udebs from stable-p-u, but not
both.  I could maybe try to extend net-retriever.

debootstrap can just install stable, and dist-upgrade to stable-p-u
later, and that's good enough to do some basic testing.

Thanks a lot!
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org

signature.asc
Description: Digital signature

Bug#795360: zfsutils: mountpoints not set except for root

2015-08-13 Thread Steven Chamberlain

Package: partman-zfs
Version: 45
Severity: important
Tags: pending

Hi,

partman-zfs recently stopped setting a mountpoint, for filesystems
other than the root.  They are mounted to /target, but not mounted
at first boot of the installed system.

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Bug#769616: tasksel: fails to preseed desktop on kfreebsd, hurd

2015-05-10 Thread Steven Chamberlain

Samuel Thibault wrote:
 I believe it's completely fixed by now: using
 
 tasksel tasksel/first multiselect standard, xfce-desktop
 
 works to preseed the desktop on !linux, and the tasksel/desktop
 preseeding works as expected.

The same is also true on linux jessie and sid.  This preseeds to
install XFCE:

tasksel tasksel/first multiselect standard, xfce-desktop

whereas this (traditional way) no longer does anything, and you would
still get GNOME or whatever is default:

tasksel tasksel/desktop multiselect xfce

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150510214156.gc49...@pyro.eu.org

Bug#784927: debootstrap: doesn't know jessie-kfreebsd suite

2015-05-10 Thread Steven Chamberlain

Package: debootstrap
Version: 1.0.67
Severity: important
Tags: patch pending
User: debian-...@lists.debian.org
Usertags: kfreebsd

Hi,

debootstrap in wheezy and sid doesn't know the new jessie-kfreebsd
suite, which prevents the installer or standalone debootstraps from
installing it.

The jessie-kfreebsd script can be the same as jessie which is
a symlink to said.  Patched in the jessie-kfreebsd Git branch:

https://anonscm.debian.org/cgit/d-i/debootstrap.git/commit/?h=jessie-kfreebsdid=045a2e63427caf5ad5f75525ff804bb7eabdc2d8

Thanks.

-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages debootstrap depends on:
ii  wget  1.14-4

Versions of packages debootstrap recommends:
ii  debian-archive-keyring  2014.3
ii  gnupg   1.4.18-6

debootstrap suggests no packages.

-- no debconf information


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20150510192129.94185.7015.report...@sid.kfreebsd-amd64.pyro.eu.org

Re: Any specific reason for grub-installer specific version numbering?

2015-05-01 Thread Steven Chamberlain

Cyril Brulebois wrote:
 Christian PERRIER bubu...@debian.org (2015-05-01):
  I was about to build+upload when I noticed this specifi version and
  wondered whether there is indeed a specific plan that requires not
  using 1.119 as version number.
 
 I guess the kfreebsd branch got pushed to master w/o updating the
 (specific) version number. Feel free to adjust it before uploading.

Yes, sorry;  that's exactly what I did.  Thanks Christian.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#783775: rootskel: supporting low-memory kfreebsd installs

2015-04-30 Thread Steven Chamberlain

tags 783773 + pending
tags 783775 + pending
thanks

Christian PERRIER wrote:
 May I suggest youjust apply these?
 
 It's earyl development time and therefore time to avoid letting
 patches and improvement rot in the BTS...

Okay, let's do that.  Have merged these into Git master.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150430115243.gf57...@pyro.eu.org

Bug#783773: debian-installer: supporting low-memory kfreebsd installs

Package: debian-installer
Version: 20150422
Severity: wishlist
Tags: patch

To follow shortly are some patches in the jessie-kfreebsd branch that
should be applicable to sid also.

Reducing MFSROOT size very slightly allows the kfreebsd-amd64 installer
to work again on systems with 144 MiB total memory. Whether using the
GTK or non-GTK image, it will run in low-memory text mode.

There seems to be enough free memory to run in GTK mode even on 256 MiB
systems, so I've enabled that. This is highly desirable, since GTK mode
enables many more locales than are available in text mode.

256 MiB is probably the smallest amount of memory that will be found in
a real amd64 machine. Virtual machines will typically have much more
than this, but a notable exception is qemu-system-x84_64 in wheezy,
which defaults to 128 MiB; there is no way we can support that in
jessie-kfreebsd.

The most memory-hungry thing is ZFS. Giving only 128 MiB swap, a GTK
install can be completed to ZFS with only 256 MiB total memory. partman
does already prompt users to create swap *and* recommend 512 MiB memory
to use ZFS. (Without any swap, install to ZFS will fail even in text
mode and with 512 MiB memory, so using GTK mode has no detriment on
ZFS installs).

-- System Information:
Debian Release: jessie/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'oldstable')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

--
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
https://lists.debian.org/20150429230911.45776.60302.report...@sid.kfreebsd-amd64.pyro.eu.org

Bug#783775: rootskel: supporting low-memory kfreebsd installs

Patch can be found at
https://anonscm.debian.org/cgit/d-i/rootskel.git/commit/?h=jessie-kfreebsd

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150429233450.gh44...@pyro.eu.org

Bug#783773: debian-installer: supporting low-memory kfreebsd installs

Steven Chamberlain wrote:
 Reducing MFSROOT size very slightly allows the kfreebsd-amd64 installer
 to work again on systems with 144 MiB total memory.

... should have read 160 MiB total memory.

Patch can be found at
https://anonscm.debian.org/cgit/d-i/debian-installer.git/commit/?h=jessie-kfreebsd

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150429233323.gg44...@pyro.eu.org

Bug#783773: debian-installer: supporting low-memory kfreebsd installs

Steven Chamberlain wrote:
 Reducing MFSROOT size very slightly allows the kfreebsd-amd64 installer
 to work again on systems with 144 MiB total memory.

... should have read 160 MiB total memory.

Patch can be found at
https://anonscm.debian.org/cgit/d-i/debian-installer.git/commit/?h=jessie-kfreebsd

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150429233242.gf44...@pyro.eu.org

Bug#783548: choose-mirror: lists releases that don't include this arch

2015-04-27 Thread Steven Chamberlain

Package: choose-mirror
Version: 2.62
Severity: wishlist
Tags: patch
User: debian-...@lists.debian.org
Usertags: jessie kfreebsd

Hi,

When installing kfreebsd or hurd in Expert mode, choose-mirror offers to
install jessie - stable and stretch - testing even though the
architecture being installed isn't part of those releases.

I've attached a patch that adds support for the Architectures: header,
to not mention a release if it doesn't include the current arch.

Thanks.

-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'oldstable')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
diff --git a/choose-mirror.c b/choose-mirror.c
index 65885f6..b7d4019 100644
--- a/choose-mirror.c
+++ b/choose-mirror.c
@@ -300,9 +300,9 @@ static int get_release(struct release_t *release, const char *name) {
 	char *command;
 	FILE *f = NULL;
 	char *wget_options, *hostname, *directory;
-	char line[80];
+	char line[BUFFER_LENGTH];
 	char *p;
-	char buf[SUITE_LENGTH];
+	char buf[BUFFER_LENGTH];
 
 	hostname = add_protocol(hostname);
 	debconf_get(debconf, hostname);
@@ -321,7 +321,7 @@ static int get_release(struct release_t *release, const char *name) {
 	}
 
 	wget_options = get_wget_options();
-	command = xasprintf(wget %s %s://%s%s/dists/%s/Release -O - | grep -E '^(Suite|Codename):',
+	command = xasprintf(wget %s %s://%s%s/dists/%s/Release -O - | grep -E '^(Suite|Codename|Architectures):',
 			wget_options, protocol, hostname, directory, name);
 	di_log(DI_LOG_LEVEL_DEBUG, command: %s, command);
 	f = popen(command, r);
@@ -337,12 +337,14 @@ static int get_release(struct release_t *release, const char *name) {
 			if (line[strlen(line) - 1] == '\n')
 line[strlen(line) - 1] = '\0';
 			if ((value = strstr(line, : )) != NULL) {
-strncpy(buf, value + 2, SUITE_LENGTH - 1);
-buf[SUITE_LENGTH - 1] = '\0';
+strncpy(buf, value + 2, BUFFER_LENGTH - 1);
+buf[BUFFER_LENGTH - 1] = '\0';
 if (strncmp(line, Codename:, 9) == 0)
 	release-name = strdup(buf);
 if (strncmp(line, Suite:, 6) == 0)
 	release-suite = strdup(buf);
+if (strncmp(line, Architectures:, 14) == 0)
+	release-archs = strdup(buf);
 			}
 		}
 		if (release-name != NULL  strcmp(release-name, name) == 0)
@@ -354,6 +356,14 @@ static int get_release(struct release_t *release, const char *name) {
 		!(release-status  IS_VALID))
 			log_invalid_release(name, Suite or Codename);
 
+		/* Does the release include this arch? */
+		if (release-archs != NULL  strstr(release-archs, ARCH_TEXT) == NULL) {
+			/* No:  disregard this release */
+			log_invalid_release(name, Architectures);
+			release-status = ~IS_VALID;
+			release-name = NULL;
+		}
+
 		/* Cross-validate the Release file */
 		if (release-status  IS_VALID)
 			if (! cross_validate_release(release))
diff --git a/mirrors.h b/mirrors.h
index e592b7a..f73aefb 100644
--- a/mirrors.h
+++ b/mirrors.h
@@ -17,6 +17,12 @@ struct mirror_t {
  */
 #define MANUAL_ENTRY manual
 
+/*
+ * Allow to read the full Architectures: line from a Release file,
+ * which is up to 123 bytes long at time of writing.
+ */
+#define BUFFER_LENGTH 256
+
 #define SUITE_LENGTH 32
 
 /* Stack of suites */
@@ -43,6 +49,7 @@ static const char suites[][SUITE_LENGTH] = {
 struct release_t {
 	char *name;
 	char *suite;
+	char *archs;
 	int status;
 };

d-i for jessie-kfreebsd

2015-04-27 Thread Steven Chamberlain

Hi KiBi,

debian-installer needs a small change to build and work with the new
jessie-kfreebsd suite.  debootstrap needs a change also.  choose-mirror
and some other d-i packages could be further refined before we decide
to release.  (I have it working now).

Would it be okay for me to create a jessie-kfreebsd branch in each
of their Git repos and commit my patches there for now?

I think these should be in a VCS if we come to upload modified packages
to jessie-kfreebsd.  It would help with review also.  Some changes may
be suitable for inclusion in sid and this would make those easy to
merge.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Bug#783548: choose-mirror: lists releases that don't include this arch

2015-04-27 Thread Steven Chamberlain

Hi,

Cyril Brulebois wrote:
 Steven Chamberlain ste...@pyro.eu.org (2015-04-27):
  +   if (strncmp(line, Architectures:, 14) == 0)
  +   release-archs = strdup(buf);
 
 I think you're missing some free() matching this strdup().

Thanks, yes.  Actually several, fixed now in jessie-kfreebsd branch.

I've confirmed in Valgrind many more leaks, and at least one error
path that will still leak a whole t_release item.  I think I can say
it's no worse now than when I touched it, and I've learned a few things
in the process.

I should point out, the strstr() match doesn't consider word boundaries,
so mips will match on mipsel for example, which could someday result
in false negatives (and thus display a suite by accident) but not false
positives.  I don't feel like re-implementing regex matching inside this
C function at this time.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150428000855.gb79...@pyro.eu.org

Bug#782976: debian-installer-netboot-images packages kfreebsd images but kfreebsd is not in jessie.

2015-04-24 Thread Steven Chamberlain

Hi peter,

I think it's been removed today already:
https://packages.qa.debian.org/d/debian-installer-netboot-images/news/20150423T215008Z.html

Do you agree?  And this bug can be closed.

 It could presumablly be
 modified to look in an unofficial location

Yeah, I guess we'll have a jessie-kfreebsd suite, where we patch this
package to do exactly that.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: kFreeBSD CD/DVD builds for Jessie

2015-04-22 Thread Steven Chamberlain

Hi Steve,

Steve McIntyre wrote:
 I know that the kFreeBSD ports are not being officially released
 [1]. What plans (if any) have been made for any *un*official releases?
 I've just removed the BSD builds from the debian-cd architecture lists
 in preparation for next weekend, but I've not heard anything more...

kfreebsd as a whole, I feel has been in a releaseable state for some
months (massive improvement over wheezy, and no bugs that are absolutely
RC).  I'd still like to make a stable release out of it.

But since we need to move it to a jessie-kfreebsd suite first, I think
we are behind schedule with that.  I'm not familiar in that area, but
Christoph was speaking with ftpmaster about it.

I expect it will require changes in debian-installer (for new repository
URLs), and there are other d-i issues I'd quite like to patch if
possible, and lots of testing I wanted to do (incl. of CD images).
Given how busy I am at the  moment, I couldn't do that in time for the
official release.

Would it be reasonable and practical - even preferable - to postpone
release of kfreebsd until at least 1-2 weeks after the official release?

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


signature.asc
Description: Digital signature

Re: Hints for d-i jessie RC3, part 2

2015-04-12 Thread Steven Chamberlain

Ivo De Decker wrote:
 On Thu, Apr 09, 2015 at 11:09:13PM +0200, Cyril Brulebois wrote:
  # no objections for d-i (kfreebsd-specific), but please check with BSD
  # guys (cc'd) before the matching unblock:
  unblock-udeb kfreebsd-10/10.1~svn274115-4
 
 Added all of these. For kfreebsd, the unblock is still needed.

Please feel free to unblock, it has security fixes only.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150412133415.ga92...@pyro.eu.org

Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror

2015-03-09 Thread Steven Chamberlain

Hi Mike,

Michael Milligan wrote:
 Thanks for fixing!  I will grab a daily image with partman 43 in it and
 give it a go.

Be careful that you might hit http://bugs.debian.org/780115 next.

If you create a pool on only one disk at first, you can easily attach
another later as a mirror, after installing.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150309133403.gb21...@squeeze.pyro.eu.org

Bug#780115: grub-installer: fails on multi-device btrfs, zfs

2015-03-09 Thread Steven Chamberlain

Package: grub-installer
Version: 1.112
Severity: important
User: debian-...@lists.debian.org
Usertags: kfreebsd
X-Debbugs-Cc: debian-...@lists.debian.org

Hi,

A section of code in grub-installer (dating back to lilo-installer)
assumes that `grub-probe -t device` or 'findfs' returns only one device.
Otherwise it fails with:
Wrong number of args: mapdevfs path

That's coming from here:
http://anonscm.debian.org/cgit/d-i/grub-installer.git/tree/grub-installer?id=84d9a8936986f388feac192b785cc7ba98f9ce1d#n244
after rootfs, bootfs, bootfstype and rootfstype were all
properly set.

For a multi-disk mdraid or LVM, only one device path is returned, e.g.
/dev/md0, /dev/mapper/foo-bar

When modern filesystems btrfs and zfs are striped or mirrored across
multiple devices, then findfs returns a list of devices, one per line.
I'm not familiar if partman-btrfs allows to create multi-device sets.
The kfreebsd installer allows to create such a configuration. This can
happen using unofficial ZFS-on-Linux installer images too.

That's the bug this patch refers to:
https://lists.debian.org/debian-boot/2014/11/msg00181.html
and fixed differently with this earlier proposal:
http://anonscm.debian.org/cgit/d-i/grub-installer.git/commit/?h=zolid=c96fd6c89445758fc6b5964d47c9d811bcb44238

It appears to have been reported and fixed in Ubuntu, differently again:
https://bugs.launchpad.net/ubuntu/+source/grub-installer/+bug/1354730
https://code.launchpad.net/~tj/ubuntu/trusty/grub-installer/lp1354730/+merge/230222

All three fixes look a bit of a hack to me, but picking any one device
works as a placeholder until this code reaches a conclusion that is
inevitable.

The purpose of this code is to find indications it should fall back to
grub-legacy. In the case of btrfs, zfs, and some other situations it is
mandatory to use grub2 anyway:
http://anonscm.debian.org/cgit/d-i/grub-installer.git/tree/grub-installer?id=84d9a8936986f388feac192b785cc7ba98f9ce1d#n374

I think if bootfstype was tested earlier, a lot of this legacy code
could be skipped over or 'optimised out'. Awkwardly though, variables
stay in scope until the end of this long script, and it's hard to see if
something gets (re-)used later. findfs gets used in some more places
and I'm not sure yet if it's satisfactory in *all* places to to consider
only the first one.

Regards,
--
Steven Chamberlain
ste...@pyro.eu.org

--
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150309125830.ga21...@squeeze.pyro.eu.org

Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror

2015-03-08 Thread Steven Chamberlain

tags 775395 + patch pending
thanks

Hi KiBi,

Could we have the attached fix for partman-zfs in sid please?
This would be an RC bug, except that it is kfreebsd-any.

Furthermore, could I step in as maintainer (add myself to Uploaders) for
this and just upload it myself?  I believe I'd need a dak dm
authorisation for that.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
From bd30e842861d39b66936716e84dd34c799a69b10 Mon Sep 17 00:00:00 2001
From: Steven Chamberlain ste...@pyro.eu.org
Date: Sun, 8 Mar 2015 13:11:58 +
Subject: [PATCH] Only enable the zpool features GRUB 2.02 supports (Closes:
 #775395)

Do not enable zpool features by default, except those that are
read-only compatible, and lz4_compress which is supported by
grub-pc  2.02~ (Closes: #775395)
---
 debian/changelog |  9 +
 lib/zfs-base.sh  | 12 +++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/debian/changelog b/debian/changelog
index 2ddf4f1..a975046 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+partman-zfs (43) unstable; urgency=medium
+
+  * Team upload.
+  * Do not enable zpool features by default, except those that are
+read-only compatible, and lz4_compress which is supported by
+grub-pc  2.02~ (Closes: #775395)
+
+ -- Steven Chamberlain ste...@pyro.eu.org  Fri, 27 Feb 2015 23:50:29 +
+
 partman-zfs (42) unstable; urgency=low
 
   [ Updated translations ]
diff --git a/lib/zfs-base.sh b/lib/zfs-base.sh
index 636902e..3b7b78a 100644
--- a/lib/zfs-base.sh
+++ b/lib/zfs-base.sh
@@ -386,7 +386,17 @@ vg_create() {
 	vg=$1
 	shift
 
-	log-output -t partman-zfs zpool create -f -m none -o altroot=/target $vg $* || return 1
+	# Feature explicitly supported by grub-pc  2.02~, see
+	# spa_feature_names[] in grub-core/fs/zfs/zfs.c
+	features=-o feature@lz4_compress=enabled
+
+	# Read-only compatible features, according to zpool-features(7)
+	for feature in async_destroy empty_bpobj spacemap_histogram
+	do
+		features=$features -o feature@${feature}=enabled
+	done
+
+	log-output -t partman-zfs zpool create -f -m none -d $features -o altroot=/target $vg $* || return 1
 
 	# Some ZFS versions don't create cachefile when -o altroot is used.
 	# Request it explicitly.
-- 
1.8.4.rc3



signature.asc
Description: Digital signature

Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror

2015-02-27 Thread Steven Chamberlain

tags 775395 + confirmed
user 775395 debian-...@lists.debian.org
usertags 775395 + kfreebsd
thanks

Hi Mike,

On 09/02/15 03:02, Michael Milligan wrote:
 This may help...  10.1 added a bunch of feature flags that grub may need
 to now support in the ZFS code and are probably tripping this all up.

You're absolutely right, some of the new feature flags are breaking
GRUB;  I just confirmed it in a fresh test-install with ZFS mounted on /boot

This is my fault;  I was so sure that GRUB in jessie supported these,
that I requested that `zpool create` no longer use -d:
http://anonscm.debian.org/cgit/d-i/partman-zfs.git/commit/?id=46b42715da4ee17c95762129600958ff75f6d02d

GRUB *does* support feature@lz4_compress, but otherwise, only features
that are 'read-only compatible' according to zpool-features(7):
http://man.freebsd.org/zpool-features

Here is the list of not read-only compatible' features that are
supported by GRUB - currently only lz4_compress:
http://anonscm.debian.org/cgit/pkg-grub/grub.git/tree/grub-core/fs/zfs/zfs.c#n277

The most problematic ones are hole_birth and embedded_data, because they
go active as soon as enabled.  Some of the others wouldn't initially
be a problem, until that functionality is used, but it's not safe to
enable those by default.

I think most of the new features are valuable and we should keep them if
possible, most certainly lz4_compress (which can be used to compress
metadata, even if you don't set compression=on for a dataset).  So I
suggest we use -d but still enable lz4 and all read-only compatible
features.  (`zpool create` without -d would enable *all* features by
default).

Thanks for reporting this!

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/54f0ff23.5090...@pyro.eu.org

Re: Bug#778734: unblock: bind9/9.9.5.dfsg-9

2015-02-26 Thread Steven Chamberlain

Cyril Brulebois wrote:
 No objection on my side, but let's give BSD folks a heads-up since
 isc-dhcp-client-udeb depends on bind9's udebs.

Thanks, the changes seem to only relate to authoritative DNS zones
though.  No foreseeable impact on ISC DHCP or d-i.

Regards
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150226235625.ga5...@squeeze.pyro.eu.org

Re: Bug#778734: unblock: bind9/9.9.5.dfsg-9

2015-02-26 Thread Steven Chamberlain

Steven Chamberlain wrote:
 Cyril Brulebois wrote:
  No objection on my side, but let's give BSD folks a heads-up since
  isc-dhcp-client-udeb depends on bind9's udebs.
 
 Thanks, the changes seem to only relate to authoritative DNS zones
 though.  No foreseeable impact on ISC DHCP or d-i.

Actually, the libs in the udebs have identical object code to before,
only their BuildID differs.

(I think this kind of analysis could be quite useful in the future, and
is becoming easier thanks to the ongoing work on reproducible builds.)

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150227001150.gb5...@squeeze.pyro.eu.org

Bug#768188: Jessie Installer hangs after processing DHCPv6 stateful addressing

2015-02-19 Thread Steven Chamberlain

Philipp Kern wrote:
 one-shot mode (-1) and will exit after it acquired a lease successfully.

dhclient isn't doing that, at least on kfreebsd.  I'm not sure that's
what -1 means.  It will try only once to get a lease, initially.  If
successful it stays running - I assumed it continues to refresh the
lease - and starting in the jessie version, will also give up the lease
on SIGINT (that was #757711).

I think reverting to what we had before reintroduces bugs, and would
break downstream Ubuntu.  I think a workaround should be more
targetted at udhcpc/dhcp6c.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150219112053.ga25...@squeeze.pyro.eu.org

Bug#768188: Jessie Installer hangs after processing DHCPv6 stateful addressing

2015-02-18 Thread Steven Chamberlain

Cyril Brulebois wrote:
 Philipp Kern pk...@debian.org (2015-02-18):
  So now I guess the question is if we revert the change that broke it:
  
Don't kill_dhcp_client without reason (Closes: #757711, #757988)
 
Do not kill_dhcp_client after setting the hostname and
domain, otherwise Linux udhcpc will stop renewing its lease, and
on other platforms dhclient will de-configure the network interface
(Closes: #757711, #757988)
  
  At this point kFreeBSD is no longer a release architecture and the other
  platform using dhclient is Ubuntu.
 
 GNU/kFreeBSD people are (AFAICT) going to try and get an unofficial
 release out, so pushing a regression in their way doesn't look too good
 to me. Maybe using an #ifdef here to avoid killing the DHCP client on
 kfreebsd, and reinstating the previous codepath on linux would be an
 acceptable compromise until some evolved signal/process handling pops
 up (during the stretch release cycle)?

Firstly, thanks for the heads-up.

We did expect that during freeze, some regressions may be introduced
that affect only GNU/kFreeBSD, and we'd have to fix things up in our
unofficial release, perhaps rolling packages back to an older version,
or uploading a patched version with +kfreebsd suffix.  So, I'm happy if
you decide to revert this.

At first glance, it reads like a limitation of udhcpc/dhcp6c only?
Killing it sounds like a workaround (which perhaps creates other
issues), and an ifdef linux also seems wrong in this context (and for
Ubuntu).

kill-all-dhcp could be told never to kill ISC dhclient, but that too is
wrong, as this is also used to implement the 'Cancel' button in the
netcfg dialogs.

Maybe there is still a better solution?

Or perhaps we could add something that kills *only* udhcpc/dhcp6c, could
clearly annotate it as this is a workaround for bug #768188.  Then it
shouldn't affect Ubuntu, or derivatives/ports using ISC DHCP at all.
And if many years pass before someone comes back to look at this, they
should understand why it's there.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150218220527.gc22...@squeeze.pyro.eu.org

Re: Bug#778366: unblock: kfreebsd-10/10.1~svn274115-2

2015-02-14 Thread Steven Chamberlain

Hi,

Michael Gilbert wrote:
 Please consider unblocking kfreebsd-10.  It fixes 2 security issues:
 https://security-tracker.debian.org/kfreebsd-10

A debdiff is attached.

The other change is to limit the arch-dep packages to kfreebsd-any
(which was forgotten in the previous upload).

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff -Nru kfreebsd-10-10.1~svn274115/debian/changelog 
kfreebsd-10-10.1~svn274115/debian/changelog
--- kfreebsd-10-10.1~svn274115/debian/changelog 2014-12-28 11:41:23.0 
+
+++ kfreebsd-10-10.1~svn274115/debian/changelog 2015-01-28 01:18:06.0 
+
@@ -1,3 +1,16 @@
+kfreebsd-10 (10.1~svn274115-2) unstable; urgency=high
+
+  * Pick SVN r277808 from FreeBSD 10.1-RELEASE to fix:
+- SA-15:02: SCTP SCTP_SS_VALUE kernel memory corruption and
+  disclosure vulnerability (CVE-2014-8612) (Closes: #776415)
+- SA-15:03: SCTP stream reset vulnerability (CVE-2014-8613)
+  (Closes: #776416)
+  * Build kernel images only on kfreebsd-any arches, so that any
+security or other RC-severity kernel bugs will not affect the
+official jessie release
+
+ -- Steven Chamberlain ste...@pyro.eu.org  Tue, 27 Jan 2015 20:02:52 +
+
 kfreebsd-10 (10.1~svn274115-1) unstable; urgency=medium
 
   [ Steven Chamberlain ]
@@ -6,9 +19,6 @@
   (CVE-2014-8476) (Closes: #768108)
   * Replace non-DFSG-free ar9300_devid.h with a 3-clause BSD substitute
 derived from Linux ath9k driver (Closes: #767583)
-  * Build kernel images only on kfreebsd-any arches, so that any
-security or other RC-severity kernel bugs will not affect the
-official jessie release
 
   [ Christoph Egger ]
   * Upload to unstable
diff -Nru kfreebsd-10-10.1~svn274115/debian/control 
kfreebsd-10-10.1~svn274115/debian/control
--- kfreebsd-10-10.1~svn274115/debian/control   2014-10-20 22:19:28.0 
+0100
+++ kfreebsd-10-10.1~svn274115/debian/control   2015-01-27 20:40:49.0 
+
@@ -51,7 +51,7 @@
 
 
 Package: kfreebsd-image-10.1-0-amd64
-Architecture: any-amd64
+Architecture: kfreebsd-amd64
 Depends: ${misc:Depends},
  freebsd-utils (= 8.1-5) [kfreebsd-any], kldutils (= 7.1) [kfreebsd-any],
  devd [kfreebsd-any] | freebsd-utils ( 8.2+ds2-9) [kfreebsd-any],
@@ -79,7 +79,7 @@
  This package is compiled for a amd64-class machine.
 
 Package: kfreebsd-image-10-amd64
-Architecture: any-amd64
+Architecture: kfreebsd-amd64
 Depends: kfreebsd-image-10.1-0-amd64, ${misc:Depends}
 Description: kernel of FreeBSD 10 image (meta-package)
  This package depends on the latest binary image for kernel of FreeBSD 10 on
@@ -496,7 +496,7 @@
  This package contains zlib modules.
 
 Package: kfreebsd-image-10.1-0-486
-Architecture: any-i386
+Architecture: kfreebsd-i386
 Depends: ${misc:Depends},
  freebsd-utils (= 8.1-5) [kfreebsd-any], kldutils (= 7.1) [kfreebsd-any],
  devd [kfreebsd-any] | freebsd-utils ( 8.2+ds2-9) [kfreebsd-any],
@@ -524,7 +524,7 @@
  This package is compiled for a 486-class machine.
 
 Package: kfreebsd-image-10-486
-Architecture: any-i386
+Architecture: kfreebsd-i386
 Depends: kfreebsd-image-10.1-0-486, ${misc:Depends}
 Description: kernel of FreeBSD 10 image (meta-package)
  This package depends on the latest binary image for kernel of FreeBSD 10 on
@@ -549,7 +549,7 @@
  486-class machines.
 
 Package: kfreebsd-image-10.1-0-686
-Architecture: any-i386
+Architecture: kfreebsd-i386
 Depends: ${misc:Depends},
  freebsd-utils (= 8.1-5) [kfreebsd-any], kldutils (= 7.1) [kfreebsd-any],
  devd [kfreebsd-any] | freebsd-utils ( 8.2+ds2-9) [kfreebsd-any],
@@ -577,7 +577,7 @@
  This package is compiled for a 686-class machine.
 
 Package: kfreebsd-image-10-686
-Architecture: any-i386
+Architecture: kfreebsd-i386
 Depends: kfreebsd-image-10.1-0-686, ${misc:Depends}
 Description: kernel of FreeBSD 10 image (meta-package)
  This package depends on the latest binary image for kernel of FreeBSD 10 on
@@ -602,7 +602,7 @@
  686-class machines.
 
 Package: kfreebsd-image-10.1-0-xen
-Architecture: any-i386
+Architecture: kfreebsd-i386
 Depends: ${misc:Depends},
  freebsd-utils (= 8.1-5) [kfreebsd-any], kldutils (= 7.1) [kfreebsd-any],
  devd [kfreebsd-any] | freebsd-utils ( 8.2+ds2-9) [kfreebsd-any],
@@ -630,7 +630,7 @@
  This package is compiled for a xen-class machine.
 
 Package: kfreebsd-image-10-xen
-Architecture: any-i386
+Architecture: kfreebsd-i386
 Depends: kfreebsd-image-10.1-0-xen, ${misc:Depends}
 Description: kernel of FreeBSD 10 image (meta-package)
  This package depends on the latest binary image for kernel of FreeBSD 10 on
diff -Nru kfreebsd-10-10.1~svn274115/debian/patches/SA-15_02.kmem.patch 
kfreebsd-10-10.1~svn274115/debian/patches/SA-15_02.kmem.patch
--- kfreebsd-10-10.1~svn274115/debian/patches/SA-15_02.kmem.patch   
1970-01-01 01:00:00.0 +0100
+++ kfreebsd-10-10.1~svn274115/debian/patches/SA-15_02.kmem.patch   
2015-01-27 20:37:34.0 +
@@ -0,0 +1,51 @@
+Description:
+ Fix SCTP SCTP_SS_VALUE kernel memory corruption

Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror

2015-01-27 Thread Steven Chamberlain

Control: severity -1 important

Michael Milligan wrote:
 Package: partman-zfs
 Version: 42
 Severity: grave
 Tags: d-i
 Justification: renders package unusable for ZFS-based install
 
 (which is probably the reason someone is trying Debian/kFreeBSD .. to
 use ZFS)

Since this is a kfreebsd-any package, and we're not part of the
official stable release, I have to lower this to non-RC severity.

We should still fix it for the GNU/kFreeBSD release though.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150127211921.gg3...@squeeze.pyro.eu.org

Bug#775395: partman-zfs in d-i jessie image does not create grub-compatible /boot ZFS mirror

2015-01-27 Thread Steven Chamberlain

Hi,

Michael Milligan wrote:
 But installing Grub failed with unknown filesystem.  In
 troubleshooting the issue, it seems grub2 (version currently is
 2.02~beta2-19) does not recognize the feature@lz4_compress option (and
 had been previously reported) of the ZFS pools that kFreeBSD kernel 10.1
 creates.

Thanks for the report.

It is actually expected that grub2 since 2.02 should understand these
feature flags, including LZ4 compression.  I thought it was working for
me when I last tested.

Hopefully this can be figured out without having to default to using
-d (disabling new features including LZ4 metadata compression).

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150127210217.ge3...@squeeze.pyro.eu.org

Re: Last hints for d-i, upload tomorrow

2015-01-05 Thread Steven Chamberlain

Hi!

On 06/01/15 00:04, Cyril Brulebois wrote:
 Ivo De Decker iv...@debian.org (2015-01-06):
 First of all: would you be ok with an unblock-udeb for kfreebsd-10?
 
 Provided Steven/BSD people are fine with it (possibly with urgenting),
 I'm very OK with having it in testing before d-i gets uploaded. I almost
 asked but decided to try and skip some more back and forth. Let's do that
 now anyway, then. :)

Yes I'm fine with this, thanks.

(There's a reason I didn't ask for unblocks sooner but that's a long
story and unrelated to d-i).

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Re: Hints for d-i

2014-12-08 Thread Steven Chamberlain

Hi KiBi,

Cyril Brulebois wrote:
 # TODO: Check with debian-bsd@
 unblock kfreebsd-10/10.1~svn273874-1
 unblock-udeb kfreebsd-10/10.1~svn273874-1

The above is fine by me, although we did plan to make one more
upload first (and then would have requested an unblock).

If that next upload doesn't materialise for whatever reason,
please feel free to go with the above.  Thanks.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141208205518.gb24...@squeeze.pyro.eu.org

tasksel desktop preseed issues (was: Re: Failure: g-i-installation_debian_sid_daily_hurd_lxde/71)

2014-11-14 Thread Steven Chamberlain

Hi,

There are a few tasksel-related issues revealed by Jenkins testing of
sid daily d-i builds, which are now active for hurd and kfreebsd as
well.  We also capture d-i syslog on those arches.

Firstly could someone please tell me what display manager this is?
https://jenkins.debian.net/view/g-i-installation/job/g-i-installation_debian_sid_daily_hurd_lxde/70/artifact/results//snapshot_008244.png

hurd-i386:
https://jenkins.debian.net/view/g-i-installation/job/g-i-installation_debian_sid_daily_hurd_lxde/70/artifact/results/serial.log

* AFAICT no desktop was installed, despite preseed file asking for lxde:
 tasksel tasksel/first multiselect standard, desktop
 tasksel tasksel/desktop multiselect lxde

* standard task still seems to bring in a huge amount of dependencies
via gnupg2  gnupg-agent  pinentry-gtk2  xserver-*

kfreebsd-amd64:
https://jenkins.debian.net/view/g-i-installation/job/g-i-installation_debian_sid_daily_kfreebsd/440/artifact/results/serial.log

* has the same issues as above, except xfce was requested:
 tasksel tasksel/first multiselect standard, desktop
 tasksel tasksel/desktop multiselect xfce

linux-amd64:
https://jenkins.debian.net/view/g-i-installation/job/g-i-installation_debian_jessie_xfce/65/artifact/results//snapshot_003706.png

* we seemed to get gnome3, despite preseed file asking for xfce:
 tasksel tasksel/first multiselect standard, desktop
 tasksel tasksel/desktop multiselect xfce

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#769616: tasksel: fails to preseed desktop on kfreebsd, hurd

2014-11-14 Thread Steven Chamberlain

Package: src:tasksel
Version: 3.29
Severity: important

Hi,

(this bug may also affect linux release architectures other than
i386|amd64|powerpc*, and their CD media, so severity may be higher)

It was seen on kfreebsd and hurd that preseeding with:
  tasksel tasksel/first multiselect standard, desktop
  tasksel tasksel/desktop multiselect xfce
no longer works, a regressions since wheezy;  no desktop gets installed:
https://lists.debian.org/debian-hurd/2014/11/msg00074.html

The logic gets ridiculously complex, but I gather that:

  * tasksel/first item desktop, used to be the only thing listed;
selecting or preseeding it would previously install task-desktop
as well as task-xfce-desktop (whatever was default, or preseeded as
tasksel/desktop)

  * now, preseeding tasksel/desktop seems to work *only* if
/usr/lib/tasksel/tests/desktop decides it should install a desktop;
irrespective of whether tasksel/first includes desktop

  * as such, on kfreebsd, hurd and some other arches, selecting or
preseeding tasksel/first with desktop only leads to task-desktop
being installed (the parent item), but not the individual task for
desktop, despite setting tasksel/desktop

So I think possible ways to fix this are:

 1. simply adding kfreebsd-* and hurd-* to the list of common
desktop architectures may inadvertently fix this - that determines
whether tasksel preselects the 'desktop' option by default;
currently the list has only (linux-) i386|amd64|powerpc* and may
explain why the bug wasn't noticed there

 2. make tests/default-desktop not conditional on check_desktop_wanted,
if tasksel/first has desktop, or tasksel/desktop is preseeded

 3. make tests/desktop (whether to install a desktop) answer yes
(exit 2) if tasksel/first has desktop, or tasksel/desktop is
preseeded, overriding any guess it would make based on architecture,
diskspace, RAM etc. that it does currently

I'll send a patch for consideration that probably does all three in
some way.

-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'stable')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20141115022053.63520.58836.report...@sid.kfreebsd-amd64.pyro.eu.org

Bug#769190: Bug#757941: Bug#769190: busybox-static: DNS resolver is broken again with the last upload

2014-11-12 Thread Steven Chamberlain

Hi,

Michael Tokarev wrote:
 Since this is all alternatives, is it really necessary to list the [arch]
 names?  I mean, just list of pkgs with versions should be enough I think,
 each arch will pick the right name, no?

I could be wrong, but I think within sbuild only the first of the
alternatives is considered.  I'm not sure whether it skips over
the missing ones on that arch, or would fail with BD-Uninstallable.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141112211037.gd19...@squeeze.pyro.eu.org

Bug#767474: Installation kFreeBSD-amd64, Kernel 10.1: OK, but still quirks

2014-10-31 Thread Steven Chamberlain

Hi,

Many thanks for testing on real hardware.

Adding my comments below;  I'll further investigate some of the issues
myself, but anyone else is welcome to look into these or the other
points in the original mail:

On 31/10/14 10:41, Herbert Kaminski wrote:
  - grub-install was not able to install grub if the file system root was
on a logical partition.

I once thought that was a limitation of what the FreeBSD kernel can use
as a root device.  In that case it should be more clearly documented.
Or if not, it may turn out to be a bug in grub-probe.

 Putting /boot on a primary partition did not help.

This needs to be investigated using grub-install and grub-probe by hand.
 It may also be worth trying to move /lib/modules into /boot/ then
creating a symlink /lib/modules - /boot/modules.

  - a regression: now xfburn finds the CDR drive only when started by root.
/dev/cd0 is owned by root:cdrom, and the standard user is member of the
cdrom group.

Is /dev/cd0 readable and/or writable by the group?  Does changing that
fix it?  This will be determined by devfs rules in /etc.

  - in XFCE, all menu entries or buttons to shutdown, reboot etc are greyed,

FWIW I don't see this bug on my own kfreebsd-amd64 sid/jessie install.

and the notification area in the panel is only a vertcal bar.

Obvious question, but were any applications running that would actually
display something there?  If I install+run something like psi-plus, then
it correctly displays there.

  - in the installer, DHCP did not work.

Fixed in sid AIUI: http://lists.debian.org/20141031114225.2a0918b1@toshi

 So, kFreeBSD on the desktop is usable after some manual intervention, if you 
 do not use a NVidia graphics card.

The nv driver should support at least *some* nVidia cards, perhaps older
and not integrated chipsets.  I've started a Wiki page where we can
start to collect data on this:
https://wiki.debian.org/Debian_GNU/kFreeBSD_Hardware#Graphics_cards

The vesa fallback should be good enough for at least some uses.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/5453838a.1030...@pyro.eu.org

Re: Bug#762762: Updating isc-dhcp udeb to dynamically link bind

2014-10-31 Thread Steven Chamberlain

On 31/10/14 10:44, Michael Tokarev wrote:
 Is it on kfreebsd, or on linux kernel too?  I wonder maybe we should
 switch to isc-dhcp on all variants/arches, and ditch udhcpc...

Linux d-i only uses udhcpc at the moment.  (Ubuntu uses isc-dhcp though
IIRC).  We did discuss converging on a single DHCP client across all
Debian architectures, but:

  * it's way too late to do this for jessie, I doubt KiBi would even
hear of it!  d-i can be very sensitive to changes and this might break
in non-obvious use cases that don't get tested much

  * for jessie+1, I think it's still early to decide this yet.  I know
that systemd will have a DHCP client - d-i doesn't currently use
systemd, but who knows what could happen in 2 years;  we also don't know
yet if there'll be a kfreebsd in jessie+1 officially

So I think we might come back to this a while after jessie is released.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/54539536.9030...@pyro.eu.org

d-i bugs and freeze

2014-10-31 Thread Steven Chamberlain

Hi KiBi,

I'd like to fix d-i bugs, but I wonder how the impending freeze
affects this:
  * may I still commit a non-RC bug fix to master
  * is it likely to be uploaded to sid
  * is it likely to get an unblock

As a real example, right now I have a fix for an issue mentioned in
two install reports, which I'd consider severity 'important'.

https://lists.debian.org/debian-bsd/2014/09/msg00125.html
| grub-mkdevicemap (in /target chroot) did not detect any (virtio)
| hard disks.  I had to Enter device manually in the dialog and
| specify /dev/vtbd0, which worked.

https://lists.debian.org/debian-bsd/2014/10/msg00445.html
| Apart from that, the grub install step was not able to find the
| disk/partition by default, so I had to specify it in the prompt as
| '/dev/xbd0'.

And the rather simple change is attached.  Not sure what to do.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org
diff --git a/debian/changelog b/debian/changelog
index 463efd1..dcff27e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+grub-installer (1.101) UNRELEASED; urgency=medium
+
+  * In device_to_disk, recognise vtbd (KVM virtio) and xbd (Xen)
+disk devices on kfreebsd
+
+ -- Steven Chamberlain ste...@pyro.eu.org  Fri, 31 Oct 2014 23:28:01 +
+
 grub-installer (1.100) unstable; urgency=medium
 
   [ Colin Watson ]
diff --git a/grub-installer b/grub-installer
index 2b61fb6..c29f2f1 100755
--- a/grub-installer
+++ b/grub-installer
@@ -134,7 +134,7 @@ hurd_convert () {
 # This should probably be rewritten using udevadm or similar.
 device_to_disk () {
 	echo $1 | \
-		sed 's:\(/dev/\(cciss\|ida\|rs\)/c[0-9]d[0-9][0-9]*\|/dev/mmcblk[0-9]\|/dev/\(ad\|ada\|da\)[0-9]\+\|/dev/[hs]d[0-9]\+\|/dev/[a-z]\+\).*:\1:'
+		sed 's:\(/dev/\(cciss\|ida\|rs\)/c[0-9]d[0-9][0-9]*\|/dev/mmcblk[0-9]\|/dev/\(ad\|ada\|da|vtbd|xbd\)[0-9]\+\|/dev/[hs]d[0-9]\+\|/dev/[a-z]\+\).*:\1:'
 }
 
 # Run update-grub in $ROOT

Re: Bug#762762: Updating isc-dhcp udeb to dynamically link bind (was: Bug#762762: nmu fixing bind issues)

2014-10-30 Thread Steven Chamberlain

On 20/10/14 01:09, Michael Gilbert wrote:
 The new isc-dhcp is now uploaded.  Please let me know how your testing goes.

After the upload of bind9/1:9.9.5.dfsg-5, this does seem to be working
well now in sid d-i.  Thanks.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#766960: debian-installer-netboot-images: Please provide packages for Debian 8

2014-10-28 Thread Steven Chamberlain

On 28/10/14 08:07, Didier 'OdyX' Raboud wrote:
 Le lundi, 27 octobre 2014, 11.57:33 Cyril Brulebois a écrit :
 I understand the reasoning but I am not keen on moving files around at
 this very late stage; if adding a symlink works, this would probably
 be better.

Symlink sounds good to me;  perhaps there are other tools out there
assuming the versioned or unversioned directory name.

 I've given more thought to this, and I think we should get d-i-n-i 
 through NEW rather sooner than later (all binary package names get 
 changed). Also, this change is jessie-specific (as d-i-n-i downloads the 
 netboot images from testing), so I've gone ahead and implemented a 
 workaround in d-i-n-i directly.
 
 Unless there are objections, I will upload this later today towards NEW.

No problem;  there's no reason we can't do both?

 Otherwise, maybe move back files under an unversioned directory and
 keep a symlink from the versioned directory. Not sure how
 {well,badly} tftp servers deal with symlinks anyway…

I don't expect any issues there.  The top-level grub2pxe file is a
symlink anyway and I've used that with various TFTP servers in the past.

 Afaik, the tftp servers are not involved here, we're talking about the 
 layout on mirrors.

Because netboot.tar.gz is also what people deploy on TFTP servers for
PXE booting.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#766960: debian-installer-netboot-images: Please provide packages for Debian 8

2014-10-28 Thread Steven Chamberlain

On 28/10/14 12:56, Cyril Brulebois wrote:
 There was no hurry AFAICT.
[...]
 Since there were no 8.x uploads at all during the release cycle, [...]

Selfishly I'm keen for the existing debian-installer-netboot-images
packages to go away as soon as possible, as they're still Built-Using:
kfreebsd-8 and kfreebsd-9, so UDD lists their bugs in the RC bug views
although they're not really issues for jessie.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Re: Bug#762401: Installation kFreeBSD-amd64: OK with quirks

2014-10-26 Thread Steven Chamberlain

Herbert Kaminski wrote:
   - unable to mount ext2/3 file systems: error message /dev/ada0s7: no such 
 device, but e.g. dd can read from /dev/ada0s7 
 This again was a different bug: As the mount man page does not provide a 
 list
 of supported filesystem names, I just tried -t ext2 and got the misleading
 error message above.

Ahhh I see.  I wonder does it work with no -t argument specified at all?

   - unable to mount NTFS filesystem (after installing ntfs-3g): unable to 
 read a NTFS bitmap. But I can mount the VFAT filesystem of an USB stick
 Still no clue why ntfs-3g has problems reading another NTFS bitmap again.

NTFS is working for me with the latest fuse4bsd package in sid;  you
may have to `kldload fuse` if it is not loaded already.  (Perhaps
another issue for documentation?)

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141026150519.gb20...@squeeze.pyro.eu.org

Bug#766430: isc-dhcp-client-udeb: dependency on missing libnsl.so.1

2014-10-23 Thread Steven Chamberlain

Package: isc-dhcp-client-udeb
Version: 4.3.1-5
Severity: grave
Tags: d-i
User: debian-...@lists.debian.org
Usertags: kfreebsd
X-Debbugs-Cc: debian-...@lists.debian.org, herbert.kamin...@gmx.de

On 01:41, Steven Chamberlain wrote:
 Herbert Kaminski wrote:
  Image: 
  http://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/kfreebsd-amd64/iso-cd/debian-testing-kfreebsd-amd64-netinst.iso
 [...]
  This time, DHCP (IPv4) failed during installation, I had to configure the 
  net manually. After reboot, I could switch back to DHCP.
 Please could you try again, only as far as the Configure network step,
 using the same install media;  just to see if the DHCP issue happens
 always or only sometimes?

Never mind, it's reproducible for me every time.
Thank you for noticing this.

It is working fine in mini.iso netboot images, and on an installed
system, but in this netinst .iso it is broken somehow:

from cdrom:
|Oct 23 01:12:19 anna[403]: DEBUG: retrieving libc0.1-udeb 2.19-11
|Oct 23 01:12:40 anna[403]: DEBUG: retrieving isc-dhcp-client-udeb 4.3.1-5
...
| dhclient: error while loading shared libraries: libnsl.so.1: cannot open 
shared object file: No such file or directory

libnsl.so.1 is provided by libc0.1 but not libc0.1-udeb.

This actually may need to be fixed in src:bind9, but it's not safe to
let isc-dhcp-udeb migrate until then.

Curiously:

https://buildd.debian.org/status/fetch.php?pkg=bind9arch=kfreebsd-amd64ver=1%3A9.9.5.dfsg-4.3stamp=1413318487
| dpkg-shlibdeps: warning: package could avoid a useless dependency if 
debian/libdns-export100-udeb/usr/lib/libdns-export.so.100.2.2 was not linked 
against libnsl.so.1 (it uses none of the library's symbols)
| dpkg-shlibdeps: warning: package could avoid a useless dependency if 
debian/libisc-export95-udeb/usr/lib/libisc-export.so.95.5.0 was not linked 
against libnsl.so.1 (it uses none of the library's symbols)
| dpkg-shlibdeps: warning: package could avoid a useless dependency if 
debian/libisccfg-export90-udeb/usr/lib/libisccfg-export.so.90.1.0 was not 
linked against libnsl.so.1 (it uses none of the library's symbols)
| dpkg-shlibdeps: warning: package could avoid a useless dependency if 
debian/libirs-export91-udeb/usr/lib/libirs-export.so.91.0.0 was not linked 
against libnsl.so.1 (it uses none of the library's symbols)

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141023014133.gd20...@squeeze.pyro.eu.org



-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141023014133.gd20...@squeeze.pyro.eu.org

Bug#766423: Installation-Report kFreeBSD daily build: no progress

2014-10-22 Thread Steven Chamberlain

Herbert Kaminski wrote:
 This time, DHCP (IPv4) failed during installation, I had to configure the 
 net manually. After reboot, I could switch back to DHCP.

OK thanks.

Please could you try again, only as far as the Configure network step,
using the same install media;  just to see if the DHCP issue happens
always or only sometimes?

 From the XFCE desktop, there is no way to shut down the system, all those
 options are greyed.

Strange, I have those options (in XFCE on kfreebsd-amd64 jessie).

 Had to switch to VT1 and log in as root, su in the
 XTerm was unsuccessful.

In what way;  what exactly does it do?  What about sudo instead?

 All the other problems from the reports #762401

I'll follow up on those to that bug log instead.

 #762590 are still there, 

That's a linux-amd64 issue though I think.

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141023004130.gc20...@squeeze.pyro.eu.org

Re: Need known bug list for Debian Linux

2014-10-21 Thread Steven Chamberlain

Hi Chethana,

On 20/10/14 12:29, Priyadarshini, Chethana (GE Healthcare) wrote:
 We are using the software product ‘Debian Linux’ version 5.0 on one of
 our embedded medical device. [...]

 We are performing a SOUP(Software of Unknown Pedigree) anomaly for the
 purpose of audit. With regard to this we need to check if the 3^rd party
 software which we are using in our product has any known defects and its
 impact on our product.

With respect to software security, you could perhaps refer to this list
of known, unfixed security issues in Debian 5.0 (oldstable);  then for
the individual packages you use, determine potential impact/severity in
your situation (if any) :
https://security-tracker.debian.org/tracker/status/release/oldstable

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Re: [Pkg-gnupg-maint] Bug#764292: gnupg2: brings too many dependencies into standard task

2014-10-20 Thread Steven Chamberlain

On 20/10/14 01:41, Eric Dorland wrote:
 task-gnome-desktop, task-xfce-desktop, task-lxde-desktop: recommend
 pinentry-gtk2

There are also tasks for the new MATE and Cinnamon desktops (both
GTK-based).  I wonder if there's a way to refactor this...

 task-kde-desktop: recommend pinentry-qt4

I wonder what would take precedence if task-desktop (common to all the
desktop tasks) recommended pinentry-gtk2, and task-kde-desktop
recommended pinentry-qt4?  Or would that not work at all?

 But other than bringing in too many packages, does it break anything
 else?

I suppose not, unless you're limited on diskspace / bandwidth /
performance;  that is where you'd most likely do a no-desktop install.
And perhaps some would not like to run extraneous daemons such as udev
(which was being brought in only due to pinentry-gtk2 dependencies).

So I think it's a rather important issue for tasksel 'standard' task,
and can be satisfactorily addressed there, if gnupg2 adjusts its
dependencies also.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#764277: Graphical DE task don't install any DE

2014-10-19 Thread Steven Chamberlain

tags 764277 + gift
user debian-...@lists.debian.org
usertags 764277 + kfreebsd
thanks

Hi,

Thanks for the bug report and your work on this so far.

On 19/10/14 18:06, Baptiste Jammet wrote:
 Dixit Joey Hess, le 08/10/2014 :
 These package which were not installed, are mostly direct Recommends
 of the task-desktop package. So it seems that Recommends were either
 not being installed, despite tasksel running apt-get -o
 APT::Install-Recommends=true or some dependency issue prevented
 installing those Recommends.

 Also, tasksel should default to selecting the xfce task on kfreebsd.
 This can be tested in an already installed system by running:
 sudo tasksel -t --new-install
 
 I don't know if someone is waiting for me to do something...
 So the above command shows
 apt-get -o APT::Install-Recommends=true
 task-desktop task-ssh-server task-french-desktop task-french
 if I only select Graphical DE, and
 apt-get -o APT::Install-Recommends=true
 task-french task-xfce-desktop task-ssh-server
 if I only select Xfce.

I'm leaving this issue for someone else to try first to see what might
be going wrong.  Perhaps try:

# apt-get -o APT::Install-Recommends=true task-desktop
# apt-get -o APT::Install-Recommends=true task-xfce-desktop
in sid on linux vs. kfreebsd, to see which packages it would try to install.

Try some of the APT debug flags:
-o Debug::pkgDepCache::AutoInstall=yes
-o Debug::pkgProblemResolver=yes

Perhaps look up those tasks on http://packages.debian.org and check
installability of task-xfce-desktop in particular, maybe using the
edos-distcheck tool or reviewing https://qa.debian.org/dose/

I'll probably check on this bug before d-i Beta 3 in case it hasn't been
figured out by then.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Re: partman-zfs upload?

2014-10-15 Thread Steven Chamberlain

On 09/10/14 11:46, Steven Chamberlain wrote:
 On 09/10/14 07:44, Christian PERRIER wrote:
   * When creating new zpools, disable optional features including
 lz4_compress, because that will require grub-pc  2.02~, which is
 in sid but not jessie yet. (Closes: #764287)
 Please revert this after grub-pc  2.02~ migrates.
 
 Yes, please upload that for me;  thank you!

Thanks Christian for uploading that;  although newer GRUB was hinted
into testing today, so I'd like to revert that change already:

http://anonscm.debian.org/cgit/d-i/partman-zfs.git/commit/?id=46b42715da4ee17c95762129600958ff75f6d02d

Please could you upload partman-zfs/40 sometime before freeze?

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Re: [Pkg-gnupg-maint] Bug#764292: gnupg2: brings too many dependencies into standard task

2014-10-14 Thread Steven Chamberlain

On 14/10/14 20:18, Eric Dorland wrote:
 Thanks for the report. We're working on this,

Okay thanks, although I notice you filed a bug only against gnome:

  * Why only gnome?  If it is important to have a gtk2 dialog then
shouldn't other desktops have it too?

  * Since this bug affects tasksel, why not also address it in tasksel,
such as in task-desktop?

  * Why only severity 'wishlist' if it is blocking a bug of IMHO higher
severity?

 but I'm not sure I
 understand the justification to make this a serious bug. Could you
 elaborate?

OK, I've looked and could not find a reference to this in policy.  But I
felt it was common sense, and policy shouldn't need to exist for such
things:

A minimal no-desktop install should not include ~70 additional packages
with ~30MB download size and taking ~100MB space, if they add no extra
functionality for that use case.  (Also curious, what will gnupg2 do if
only pinentry-gtk2 is installed but no desktop is running?)

I'm not saying it's gnupg-agent's fault this situation occurred, but it
is IMHO a serious enough problem affecting d-i/tasksel, to need to fix
it before release?

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Re: partman-zfs upload?

2014-10-09 Thread Steven Chamberlain

Hi,

On 09/10/14 07:44, Christian PERRIER wrote:
 Is it OK to upload partman-zfs with this:
 
   * Team upload.
   * When creating new zpools, disable optional features including
 lz4_compress, because that will require grub-pc  2.02~, which is
 in sid but not jessie yet. (Closes: #764287)
 Please revert this after grub-pc  2.02~ migrates.
   * Bump zfsutils dependency to upstream version 9.2 or later, with
 support for the zpool create -d flag required for the above

Yes, please upload that for me;  thank you!

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#764505: kFreeBSD-amd64: Beta installer 2 seems to work

2014-10-08 Thread Steven Chamberlain

Hi,

Many thanks for testing!

On 08/10/14 17:45, Dan Greene wrote:
 When selecting a mirror, I chose the default choice, and it gave an
 error about the architecture not being found on the mirror.
 I tried again with the same choice, and it worked.

I've seen this in the past, and thought it was due to unreliable
connectivity.  I more often see problems fetching the base system
packages, and sometimes have to manually retry that step.  Perhaps there
is a bug causing it, or it's just not trying hard enough.

 I noticed there was an option to install GNOME, even though it is
 not available for kFreeBSD jessie.

I think tasksel is already supposed to hide uninstallable tasks, and
indeed, meta-gnome-desktop should be uninstallable in testing:
| Package has a Depends on gnome-core which cannot be satisfied on
kfreebsd-amd64.
| Package has a Depends on gnome-core which cannot be satisfied on
kfreebsd-i386.

 Someone should test the installer with fancier options enabled,
 like ZFS and iSCSI. (ZFS on iSCSI? May not be a typical use case,
 but it still should be tested in csse it reveals other bugs.)

iSCSI isn't implemented yet, and shouldn't be listed there.  Only
showing because it's an arch:all udeb and unsatisfied dependencies are
not being considered by anna.  I should try to put partman-iscsi in the
anna excludes list.

 kldstat:  61 0x86167000 88e3 fuse.ko

It's interesting to see that module loaded.  I guess os-prober loads it
in order to scan for ntfs partitions, and that's good.  Someone said
that feature wasn't working, but seems like it may start to work as soon
as fuse4bsd/0.3.9~pre1.20080208-9 migrates (of already if installing the
sid distribution).  Needs to be tested on a machine having Windows...

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/543586d3.9060...@pyro.eu.org

Bug#764287: GRUB before 2.02 doesn't support ZFS lz4 compression

Package: partman-zfs
Version: 39
Severity: critical
Tags: pending
User: debian-...@lists.debian.org
Usertags: kfreebsd

Hi,

If d-i is running kfreebsd 10.1, lz4 will be active by default for
newly-created ZFS pools.  GRUB 2.02 (in sid) can boot from these,
but GRUB 2.00 (in jessie) cannot.

So, installing sid from sid d-i is fine.

Installing jessie from sid d-i will fail to install a bootloader,
if /boot or /lib/modules are in a newly-created ZFS pool.

Installing jessie from jessie d-i Beta 2 is also fine, but new
installs to ZFS could fail as above if kfreebsd 10.1 udebs migrate
before GRUB 2.00 does.

To avoid that happening, I'm changing the default to not enable
lz4 in newly-created zpools.  This change can be reverted as soon as
GRUB migrates to jessie.

Old zpools created by wheezy are not affected because lz4 is already
disabled for them.

Thanks.

-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'stable')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 9.0-2-amd64-xenhvm-ipsec
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20141006212800.39709.29945.report...@sid.kfreebsd-amd64.pyro.eu.org

Bug#764292: gnupg2: brings too many dependencies into standard task

Package: gnupg2
Version: 2.0.26-2
Severity: serious
Tags: d-i

Hi,

Installing a sid system from d-i Beta 2 (kfreebsd-amd64, but probably
also Linux), tasksel task 'standard' brings in many unwanted
dependencies via:

mutt - libgpgme11 - gnupg2 - gnupg-agent - pinentry-gtk2 - ...

I suggest maybe pinentry-curses should be the first-listed provider
of pinentry, but pinentry-gtk2 could be added to the desktop tasks?

Demonstrated in /target/ from d-i shell:

# apt-get -o Debug::pkgDepCache::AutoInstall=1 install mutt
Reading package lists... Done
Building dependency tree
Reading state information... Done
[...]
  Installing libgpgme11 as Depends of mutt
Installing gnupg2 as Depends of libgpgme11
  Installing gnupg-agent as Depends of gnupg2
Installing pinentry-gtk2 as Depends of gnupg-agent
[...]
The following extra packages will be installed:
  bsd-mailx ca-certificates dbus exim4-base exim4-config exim4-daemon-light 
file fontconfig fontconfig-config fonts-dejavu-core gnupg-agent gnupg2
  hicolor-icon-theme libassuan0 libatk1.0-0 libatk1.0-data libavahi-client3 
libavahi-common-data libavahi-common3 libcairo2 libcups2 libcurl3-gnutls
  libdatrie1 libdbus-1-3 libfontconfig1 libfreetype6 libgcrypt11 libgcrypt20 
libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libglib2.0-0 libglib2.0-data
  libgnutls26 libgpg-error0 libgpgme11 libgraphite2-3 libgtk2.0-0 libgtk2.0-bin 
libgtk2.0-common libharfbuzz0b libjasper1 libjbig0 libjpeg62 libksba8
  libldap-2.4-2 liblockfile-bin liblockfile1 libmagic1 libpango-1.0-0 
libpangocairo-1.0-0 libpangoft2-1.0-0 libpixman-1-0 libpng12-0 libpth20 librtmp1
  libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 libthai-data 
libthai0 libtiff5 libtokyocabinet9 libx11-6 libx11-data libxau6 libxcb-render0
  libxcb-shm0 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxdmcp6 libxext6 
libxfixes3 libxi6 libxinerama1 libxml2 libxrandr2 libxrender1 mime-support
  openssl pinentry-gtk2 psmisc sgml-base shared-mime-info ucf xdg-user-dirs 
xml-core
Suggested packages:
  dbus-x11 mail-reader eximon4 exim4-doc-html exim4-doc-info spf-tools-perl 
swaks gnupg-doc parcimonie xloadimage cups-common rng-tools gpgsm
  librsvg2-common gvfs libjasper-runtime libsasl2-modules-otp 
libsasl2-modules-ldap libsasl2-modules-sql libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-heimdal urlview aspell ispell mixmaster pinentry-doc 
sgml-base-doc debhelper
Recommended packages:
  mailx default-mta mail-transport-agent
The following NEW packages will be installed:
  bsd-mailx ca-certificates dbus exim4-base exim4-config exim4-daemon-light 
file fontconfig fontconfig-config fonts-dejavu-core gnupg-agent gnupg2
  hicolor-icon-theme libassuan0 libatk1.0-0 libatk1.0-data libavahi-client3 
libavahi-common-data libavahi-common3 libcairo2 libcups2 libcurl3-gnutls
  libdatrie1 libdbus-1-3 libfontconfig1 libfreetype6 libgcrypt11 libgcrypt20 
libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libglib2.0-0 libglib2.0-data
  libgnutls26 libgpg-error0 libgpgme11 libgraphite2-3 libgtk2.0-0 libgtk2.0-bin 
libgtk2.0-common libharfbuzz0b libjasper1 libjbig0 libjpeg62 libksba8
  libldap-2.4-2 liblockfile-bin liblockfile1 libmagic1 libpango-1.0-0 
libpangocairo-1.0-0 libpangoft2-1.0-0 libpixman-1-0 libpng12-0 libpth20 librtmp1
  libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1 libthai-data 
libthai0 libtiff5 libtokyocabinet9 libx11-6 libx11-data libxau6 libxcb-render0
  libxcb-shm0 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxdmcp6 libxext6 
libxfixes3 libxi6 libxinerama1 libxml2 libxrandr2 libxrender1 mime-support
  mutt openssl pinentry-gtk2 psmisc sgml-base shared-mime-info ucf 
xdg-user-dirs xml-core
0 upgraded, 90 newly installed, 0 to remove and 0 not upgraded.
Need to get 30.8 MB of archives.
After this operation, 101 MB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

Thanks!

-- System Information:
Debian Release: jessie/sid
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'testing'), (500, 'stable'), (1, 
'experimental')
Architecture: kfreebsd-amd64 (x86_64)

Kernel: kFreeBSD 10.1-0-amd64
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages gnupg2 depends on:
ii  dpkg 1.17.13
ii  gnupg-agent  2.0.26-2
ii  install-info 5.2.0.dfsg.1-2
ii  libassuan0   2.1.1-1
ii  libbz2-1.0   1.0.6-5
ii  libc0.1  2.19-11
ii  libcurl3-gnutls  7.33.0-1
ii  libgcrypt20  1.6.2-3
ii  libgpg-error01.13-0.2
ii  libksba8 1.3.0-2
ii  libreadline6 6.2+dfsg-0.1
ii  zlib1g   1:1.2.8.dfsg-2

Versions of packages gnupg2 recommends:
ii  libldap-2.4-2  2.4.31-1+nmu2+b1

Versions of packages gnupg2 suggests:
pn  gnupg-doc   none
pn  parcimonie  none
pn  xloadimage  none

-- no debconf information


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Re: Debian Installer Jessie Beta 2 release

Hi!

On 16:06, Baptiste Jammet wrote:
 Testing debian-jessie-DI-b2-kfreebsd-amd64-netinst.iso under qemu,
 if I only choose
 « Graphical desktop environment », d-i don't install any DE, [...]

There should have been several DEs listed in that menu to choose from;
seems okay to me, or is that choice only shown for Expert-mode installs?

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141006234418.gf25...@squeeze.pyro.eu.org

Re: Bug#762762: Updating isc-dhcp udeb to dynamically link bind (was: Bug#762762: nmu fixing bind issues)

Michael Gilbert wrote:
 On Sun, Oct 5, 2014 at 9:59 PM, Cyril Brulebois wrote:
  I'm not going to go through building this on a kfreebsd porterbox to try
  and figure out how isc-dhcp would look if rebuilt against such packages,
  but that looks a saner base for porters to build upon.
 
  That doesn't make the timing issues I've mentioned disappear though. I'm
  OK with thinking about it again if porters endorse/welcome/successfully
  test the resulting packages and installation images.

I will try to find time to do this.  Probably it is desirable, because it
should reduce the overall size of udebs in the installer ramdisks?

 Would it be ok to stage the changes in unstable to make it somewhat
 easy for porters to test?

I don't particularly need that as I can build the udebs and d-i image
from them myself.  But doing so would allow others to be testing it
meanwhile, so I'm okay with it.

Thanks,
Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org


-- 
To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20141006235202.gg25...@squeeze.pyro.eu.org

Re: Bug#762762: Updating isc-dhcp udeb to dynamically link bind (was: Bug#762762: nmu fixing bind issues)