Bug#814027: [pkg-gnupg-maint] Bug#814027: "ts209" d-i image failed to build due to size

[Martin Michlmayr]

* Werner Koch  [2016-02-16 08:33]:
> What about using a smaller version of libgcrypt for d-i?  My current 1.7
> version using standard options has a stripped size of 1.1 MiB.  By using
> these options:
> 
>   ./configure --enable-maintainer-mode \
>   --enable-ciphers=cast5,des,aes \
>   --enable-digests=sha1,sha256,sha512 \
>   --enable-kfds=s2k,pkdf2 \
>   --disable-padlock-support --disable-aesni-support \
>   --disable-drng-support \
>   --disable-avx-support --disable-avx2-support \
>   --disable-pclmul-support 
> 
> I get down to a stripped size of the SO of 551 KiB on amd64.
> 
> We would need to tweak GnuPG a bit to work with that version.  Right now
> it complains about missing MD5 at runtime.  But that should be fixed
> anyway.
> 
> Would that be a way forward?

I'm not sure if it will help in my particular case (I can try,
though), but I think this should be done anyway.  Do you (or Daniel)
want to file a bug on libgcrypt once GnuPG has been tweaked?

-- 
Martin Michlmayr
http://www.cyrius.com/

Bug#814027: [pkg-gnupg-maint] Bug#814027: "ts209" d-i image failed to build due to size

[Daniel Kahn Gillmor]

On Mon 2016-02-15 22:47:11 -0500, Martin Michlmayr wrote:

> I had an idea on how to solve this problem: in addition to building
> gpgv-udeb from gnupg2, you could build a gpgv1.4-udeb on armel.  I
> could then use that gpgv1.4-udeb on the armel subarch with the size
> issues.
>
> Would that work for you?  I've attached a proposed patch.

I've uploaded this patch for now so that the stretch installers can be
rebuilt for the tighter armel platforms.  But i'd really like to pursue
Werner's suggestion about minimizing the libgcrypt udeb where possible,
so we can drop gpgv1.4-udeb entirely.

   --dkg

Bug#814027: [pkg-gnupg-maint] Bug#814027: "ts209" d-i image failed to build due to size

[Werner Koch]

Hi!

What about using a smaller version of libgcrypt for d-i?  My current 1.7
version using standard options has a stripped size of 1.1 MiB.  By using
these options:

  ./configure --enable-maintainer-mode \
  --enable-ciphers=cast5,des,aes \
  --enable-digests=sha1,sha256,sha512 \
  --enable-kfds=s2k,pkdf2 \
  --disable-padlock-support --disable-aesni-support \
  --disable-drng-support \
  --disable-avx-support --disable-avx2-support \
  --disable-pclmul-support 

I get down to a stripped size of the SO of 551 KiB on amd64.

We would need to tweak GnuPG a bit to work with that version.  Right now
it complains about missing MD5 at runtime.  But that should be fixed
anyway.

Would that be a way forward?


Salam-Shalom,

   Werner

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

Bug#814027: "ts209" d-i image failed to build due to size

[Martin Michlmayr]

Hi Daniel,

I had an idea on how to solve this problem: in addition to building
gpgv-udeb from gnupg2, you could build a gpgv1.4-udeb on armel.  I
could then use that gpgv1.4-udeb on the armel subarch with the size
issues.

Would that work for you?  I've attached a proposed patch.

-- 
Martin Michlmayr
http://www.cyrius.com/

diff --git a/debian/control b/debian/control
index 39ce3d2..d3860ed 100644
--- a/debian/control
+++ b/debian/control
@@ -70,6 +70,22 @@ Description: GNU privacy guard - signature verification tool
  different (and simpler) way to check that the public keys used to
  make the signature are trustworthy.
 
+Package: gpgv1.4-udeb
+Package-Type: udeb
+Section: debian-installer
+Priority: extra
+Architecture: armel
+Provides: gpgv-udeb
+Depends: ${misc:Depends}, ${shlibs:Depends}
+Description: minimal signature verification tool
+ GnuPG is GNU's tool for secure communication and data storage.
+ It can be used to encrypt data and to create digital signatures.
+ It includes an advanced key management facility and is compliant
+ with the proposed OpenPGP Internet standard as described in RFC 4880.
+ .
+ This is GnuPG's signature verification tool, gpgv, packaged in minimal
+ form for use in debian-installer.
+
 Package: gpgv-win32
 Architecture: all
 Priority: extra
diff --git a/debian/gpgv1.4-udeb.install b/debian/gpgv1.4-udeb.install
new file mode 100644
index 000..dc83591
--- /dev/null
+++ b/debian/gpgv1.4-udeb.install
@@ -0,0 +1 @@
+build-udeb/g10/gpgv usr/bin/
diff --git a/debian/rules b/debian/rules
index e51d3f0..29c2516 100755
--- a/debian/rules
+++ b/debian/rules
@@ -50,6 +50,11 @@ build-deb-curl/config.status:
 	(mkdir -p $(@D); cd $(@D); \
 	../configure LDFLAGS="$(LDFLAGS)" CPPFLAGS="$(CPPFLAGS_DEF)" CFLAGS="$(CFLAGS_DEF)" $(CONFARGS) $(CONFARGS_FULL) $(HOSTARG))
 
+build-udeb/config.status:
+	dh_testdir
+	(mkdir -p $(@D); cd $(@D); CFLAGS="-g -Os" \
+	../configure $(CONFARGS) $(CONFARGS_MINIMAL) $(HOSTARG) )
+
 build-win32/config.status:
 	dh_testdir
 	(mkdir -p $(@D); cd $(@D); CFLAGS="-g -Os" \
@@ -74,13 +79,21 @@ ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
 endif
 	touch $@
 
+build-udeb-stamp: build-udeb/config.status
+	dh_testdir
+	$(MAKE) -C build-udeb/
+ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
+	make -C build-udeb/checks check || exit 1
+endif
+	touch $@
+
 build-win32-stamp: build-win32/config.status
 	dh_testdir
 	$(MAKE) -C build-win32/
 	touch $@
 
 build: build-arch
-build-arch: build-deb-stamp build-deb-curl-stamp
+build-arch: build-deb-stamp build-deb-curl-stamp build-udeb-stamp
 build-indep: build-win32-stamp
 
 # clean ##
@@ -88,9 +101,9 @@ build-indep: build-win32-stamp
 clean: 
 	dh_testdir
 	dh_testroot
-	rm -rf build-deb-curl/ build-deb/ build-win32/
+	rm -rf build-udeb/ build-deb-curl/ build-deb/ build-win32/
 	# find . -name \*~ | xargs rm -vf
-	dh_clean build-deb-stamp build-deb-curl-stamp build-win32-stamp
+	dh_clean build-deb-stamp build-deb-curl-stamp build-udeb-stamp build-win32-stamp
 
  install #
 

Bug#814027: "ts209" d-i image failed to build due to size

[Martin Michlmayr]

* Ben Hutchings  [2016-02-14 18:18]:
> > For now, I pushed my change to the benh/armel-marvell branch.
> 
> After discussing this with Martin Michlmayr, we agreed they should not
> be combined, so I've deleted that branch.

Yes, I believe it doesn't make sense to combine them for a number of
reasons.  One is that if we make that change, we should also update
archdetect which would require a number of other updates, such as
the flash-kernel udeb.

I think it's easier to just change the line to use the -marvell kernel
rather than -orion5x/kirkwood.

Another is that I'd like to keep the existing directory layout,
otherwise the links change between jessie and stretch.

I'll make the changes when 4.4 hits unstable.

-- 
Martin Michlmayr
http://www.cyrius.com/

Bug#814027: "ts209" d-i image failed to build due to size

[Ben Hutchings]

On Sun, 2016-02-14 at 17:50 +, Ben Hutchings wrote:
> On Mon, 2016-02-15 at 01:13 +0900, Roger Shimizu wrote:
> > [remove CC 814...@bugs.debian.org due to my post has nothing to do with 
> > gnupg]
> > 
> > Dear Martin,
> > 
> > On Thu, Feb 11, 2016 at 4:36 AM, Martin Michlmayr  wrote:
> > > * Roger Shimizu  [2016-02-11 00:12]:
> > > > As you may already know, "ts209" d-i image failed to build due to
> > > > size, since two day ago.
> > > > It was OK on Feb. 6th [0], but got failed since Feb. 7th [1].
> > > 
> > > gpgv-udeb is now provided by gnupg2 rather than gnupg 1.4 and this
> > > brought in a number of new dependencies, in particular libgcrypt20
> > > which is huge.  I filed #814027 but I'm not sure how realistic it is
> > > that this change will be reverted.
> > > 
> > > Since GnuPG is only used for verification, I wonder if there's a
> > > smaller tool that only does signature verification but I don't know.
> > > 
> > > If we find no solution, I'll see if I can provide unofficial images
> > > with the old gpgv-udeb.
> > 
> > Thanks for the info!
> > 
> > I tried to hack debian-installer, and split orion5x flavour into
> > orion5x and orion5x-qnap.
> > For orion5x-qnap, I removed a few qnap unused stuff in "pkg-lists" folder.
> > I pushed my change to branch "split_orion5x"
> >   - 
> > https://anonscm.debian.org/cgit/d-i/debian-installer.git/log/?h=split_orion5x
> [...]
> 
> I was looking at combining the kirkwood and orion5x flavours, matching
> the kernel.  Should I not do that?  Or is it OK to have a combined
> flavour for everything but those old QNAP devices?
> 
> For now, I pushed my change to the benh/armel-marvell branch.

After discussing this with Martin Michlmayr, we agreed they should not
be combined, so I've deleted that branch.

Ben.

-- 
Ben Hutchings
Sturgeon's Law: Ninety percent of everything is crap.

signature.asc
Description: This is a digitally signed message part

Bug#814027: "ts209" d-i image failed to build due to size

[Roger Shimizu]

[remove CC 814...@bugs.debian.org due to my post has nothing to do with gnupg]

Dear Martin,

On Thu, Feb 11, 2016 at 4:36 AM, Martin Michlmayr  wrote:
> * Roger Shimizu  [2016-02-11 00:12]:
>> As you may already know, "ts209" d-i image failed to build due to
>> size, since two day ago.
>> It was OK on Feb. 6th [0], but got failed since Feb. 7th [1].
>
> gpgv-udeb is now provided by gnupg2 rather than gnupg 1.4 and this
> brought in a number of new dependencies, in particular libgcrypt20
> which is huge.  I filed #814027 but I'm not sure how realistic it is
> that this change will be reverted.
>
> Since GnuPG is only used for verification, I wonder if there's a
> smaller tool that only does signature verification but I don't know.
>
> If we find no solution, I'll see if I can provide unofficial images
> with the old gpgv-udeb.

Thanks for the info!

I tried to hack debian-installer, and split orion5x flavour into
orion5x and orion5x-qnap.
For orion5x-qnap, I removed a few qnap unused stuff in "pkg-lists" folder.
I pushed my change to branch "split_orion5x"
  - 
https://anonscm.debian.org/cgit/d-i/debian-installer.git/log/?h=split_orion5x

Here's comparison of error build log from debian-installer:

  branch master:
  File size 4465504 is larger than total, padded size 4194304.

  branch orion-qnap (after my change):
  File size 4394588 is larger than total, padded size 4194304.

so the qnap d-i image has reduced 70k, and need another 200k reducement.

I have no idea whether this kinda change valid.
Maybe it's just proof of concept that size can be reduced.

Cheers,
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 17B3ACB1

Bug#814027: "ts209" d-i image failed to build due to size

[Ben Hutchings]

On Mon, 2016-02-15 at 01:13 +0900, Roger Shimizu wrote:
> [remove CC 814...@bugs.debian.org due to my post has nothing to do with gnupg]
> 
> Dear Martin,
> 
> On Thu, Feb 11, 2016 at 4:36 AM, Martin Michlmayr  wrote:
> > * Roger Shimizu  [2016-02-11 00:12]:
> > > As you may already know, "ts209" d-i image failed to build due to
> > > size, since two day ago.
> > > It was OK on Feb. 6th [0], but got failed since Feb. 7th [1].
> > 
> > gpgv-udeb is now provided by gnupg2 rather than gnupg 1.4 and this
> > brought in a number of new dependencies, in particular libgcrypt20
> > which is huge.  I filed #814027 but I'm not sure how realistic it is
> > that this change will be reverted.
> > 
> > Since GnuPG is only used for verification, I wonder if there's a
> > smaller tool that only does signature verification but I don't know.
> > 
> > If we find no solution, I'll see if I can provide unofficial images
> > with the old gpgv-udeb.
> 
> Thanks for the info!
> 
> I tried to hack debian-installer, and split orion5x flavour into
> orion5x and orion5x-qnap.
> For orion5x-qnap, I removed a few qnap unused stuff in "pkg-lists" folder.
> I pushed my change to branch "split_orion5x"
>   - 
> https://anonscm.debian.org/cgit/d-i/debian-installer.git/log/?h=split_orion5x
[...]

I was looking at combining the kirkwood and orion5x flavours, matching
the kernel.  Should I not do that?  Or is it OK to have a combined
flavour for everything but those old QNAP devices?

For now, I pushed my change to the benh/armel-marvell branch.

Ben.

-- 
Ben Hutchings
Sturgeon's Law: Ninety percent of everything is crap.

signature.asc
Description: This is a digitally signed message part

Bug#814027: "ts209" d-i image failed to build due to size

[Martin Michlmayr]

* Roger Shimizu  [2016-02-11 00:12]:
> As you may already know, "ts209" d-i image failed to build due to
> size, since two day ago.
> It was OK on Feb. 6th [0], but got failed since Feb. 7th [1].

gpgv-udeb is now provided by gnupg2 rather than gnupg 1.4 and this
brought in a number of new dependencies, in particular libgcrypt20
which is huge.  I filed #814027 but I'm not sure how realistic it is
that this change will be reverted.

Since GnuPG is only used for verification, I wonder if there's a
smaller tool that only does signature verification but I don't know.

If we find no solution, I'll see if I can provide unofficial images
with the old gpgv-udeb.

-- 
Martin Michlmayr
http://www.cyrius.com/