Bug#772983: kirkwood kernel image is too big
On Sun, 2014-12-14 at 19:26 +, Ben Hutchings wrote:
I think it would be better to add an unconditional warning, rather than
an error, when there is 1% free space left. I realise this will be
easy to ignore but it's still better than an unnecessary failure.
OK, makes sense. Here's what I'm currently running with, I'll push it
along with the size reduction stuff.
Ian.
commit 42c4d12d02edbf8ca065d4d21f8fdc668ec095cc
Author: Ian Campbell i...@debian.org
Date: Mon Dec 15 21:25:38 2014 +
[armel] Warn if image size leaves less than 1% spare capacity in the flash.
This allows some slack for growth over the lifetime of a stable release.
diff --git a/debian/bin/buildcheck.py b/debian/bin/buildcheck.py
index 38241df..8cad299 100755
--- a/debian/bin/buildcheck.py
+++ b/debian/bin/buildcheck.py
@@ -172,6 +172,8 @@ class CheckImage(object):
self.dir = dir
self.arch, self.featureset, self.flavour = arch, featureset, flavour
+self.changelog = Changelog(version=VersionLinux)[0]
+
self.config_entry_build = config.merge('build', arch, featureset,
flavour)
self.config_entry_image = config.merge('image', arch, featureset,
flavour)
@@ -204,7 +206,20 @@ class CheckImage(object):
out.write('Image too large (%d %d)! Refusing to continue.\n' %
(size, value))
return 1
-out.write('Image fits (%d = %d). Continuing.\n' % (size, value))
+# 1% overhead is desirable in order to cope with growth
+# through the lifetime of a stable release. Warn if this is
+# not the case.
+usage = (float(size)/value) * 100.0
+out.write('Image size %d/%d, using %.2f%%. ' % (size, value, usage))
+if size value:
+sys.write('Too large. Refusing to continue.\n')
+return 1
+elif usage = 99.0:
+out.write('Under 1%% space in %s. ' % self.changelog.distribution)
+else:
+out.write('Image fits. ')
+out.write('Continuing.\n')
+
return 0
diff --git a/debian/changelog b/debian/changelog
index 6492cda..ae58576 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -14,6 +14,9 @@ linux (3.16.7-ckt2-2) UNRELEASED; urgency=medium
OMAP5_DSS_HDMI, DISPLAY_ENCODER_TPD12S015, DISPLAY_CONNECTOR_HDMI,
USB_DWC3_OMAP, EXTCON_PALMAS, TI_EMIF and DDR.
Based on a patch from Chen Baozi (Closes: #772953)
+ * [armel] Change configuration to reduce kernel image size
+- Warn if image size leaves less than 1% spare capacity in the flash. This
+ allows some slack for growth over the lifetime of a stable release.
-- Ben Hutchings b...@decadent.org.uk Sat, 13 Dec 2014 11:45:48 +
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772983: kirkwood kernel image is too big
On Sun, 2014-12-14 at 17:56 +, Ian Campbell wrote:
On Fri, 2014-12-12 at 20:59 +, Ben Hutchings wrote:
That implies we want to allow for about 1% growth
from the size in the .0 release.
I'm considering something like this. What do you think?
diff --git a/debian/bin/buildcheck.py b/debian/bin/buildcheck.py
index a6f6f06..5bb815c 100755
--- a/debian/bin/buildcheck.py
+++ b/debian/bin/buildcheck.py
@@ -174,6 +174,8 @@ class CheckImage(object):
self.dir = dir
self.arch, self.featureset, self.flavour = arch, featureset, flavour
+self.changelog = Changelog(version=VersionLinux)[0]
+
self.config_entry_build = config.merge('build', arch, featureset,
flavour)
self.config_entry_image = config.merge('image', arch, featureset,
flavour)
@@ -206,7 +208,18 @@ class CheckImage(object):
out.write('Image too large (%d %d)! Refusing to continue.\n'
% (size, value))
return 1
-out.write('Image fits (%d = %d). Continuing.\n' % (size, value))
+# 1% overhead is desirable in order to cope with growth
+# through the lifetime of a stable release. Enforce that
+# development releases leave sufficient overhead.
+soft_value = value * 0.99
+if size soft_value:
+out.write('Image has 1%% overhead (%d %d).' % (size,
soft_value))
+if self.changelog.distribution in [unstable, experimental,
UNRELEASED]:
+out.write(' Refusing to continue.\n')
+return 1
+out.write(' Continuing.\n')
I don't think this distribution condition is right:
- If we're nowhere near a release, then it doesn't matter whether images
in unstable or experimental nearly fill the space available. The size
may go up or down a long way before we release.
- If I'm preparing a stable update but don't think it's ready to upload
yet then my test build may have distribution=UNRELEASED.
I think it would be better to add an unconditional warning, rather than
an error, when there is 1% free space left. I realise this will be
easy to ignore but it's still better than an unnecessary failure.
Ben.
+out.write('Image fits (%d = %d, soft %d). Continuing.\n' % (size,
value, soft_value))
return 0
--
Ben Hutchings
The two most common things in the universe are hydrogen and stupidity.
signature.asc
Description: This is a digitally signed message part
Bug#772983: kirkwood kernel image is too big
On Sat, 2014-12-13 at 18:09 +, Ben Hutchings wrote: On Sat, 2014-12-13 at 07:57 +, Ian Campbell wrote: On Fri, 2014-12-12 at 20:59 +, Ben Hutchings wrote: [...] I misread earlier - kirkwood is about 2.5 KB below the limit, not 1. Anyway, both kirkwood and orion5x have much less than 1% of growth room and ixp4xx has slightly less. I think that some of the config changes I made in trunk should be applied to jessie/sid as well. I agree. I suppose you have some candidates in mind? Any of the things I disabled on trunk could reasonably be disabled on sid, except that I want to keep CONFIG_DEBUG_BUGVERBOSE enabled if possible. In addition to what's in trunk I'll look at whether CONFIG_PCI_QUIRKS is relevant on kirkwood, since according to the comment in config-reduced it offers good bang for buck if it isn't needed (17K saving). USER_NS might be a candidate too, especially if MEMCG and CHECPOINT_RESTORE is going. kirkwood: 1606512 1613040 2097080 0.41 30.54 I think we are looking at limit of around 2076109 to give ourselves room for 1% growth. orion5x: 1475936 1483632 1572792 0.52 6.56 Desired size is 1557064. Note that on sid, config-reduced applies to both ixp4xx and orion5x whereas on trunk it now applies to kirkwood and orion5x. Noted. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772983: kirkwood kernel image is too big
On Fri, 2014-12-12 at 20:59 +, Ben Hutchings wrote:
That implies we want to allow for about 1% growth
from the size in the .0 release.
I'm considering something like this. What do you think?
diff --git a/debian/bin/buildcheck.py b/debian/bin/buildcheck.py
index a6f6f06..5bb815c 100755
--- a/debian/bin/buildcheck.py
+++ b/debian/bin/buildcheck.py
@@ -174,6 +174,8 @@ class CheckImage(object):
self.dir = dir
self.arch, self.featureset, self.flavour = arch, featureset, flavour
+self.changelog = Changelog(version=VersionLinux)[0]
+
self.config_entry_build = config.merge('build', arch, featureset,
flavour)
self.config_entry_image = config.merge('image', arch, featureset,
flavour)
@@ -206,7 +208,18 @@ class CheckImage(object):
out.write('Image too large (%d %d)! Refusing to continue.\n' %
(size, value))
return 1
-out.write('Image fits (%d = %d). Continuing.\n' % (size, value))
+# 1% overhead is desirable in order to cope with growth
+# through the lifetime of a stable release. Enforce that
+# development releases leave sufficient overhead.
+soft_value = value * 0.99
+if size soft_value:
+out.write('Image has 1%% overhead (%d %d).' % (size,
soft_value))
+if self.changelog.distribution in [unstable, experimental,
UNRELEASED]:
+out.write(' Refusing to continue.\n')
+return 1
+out.write(' Continuing.\n')
+
+out.write('Image fits (%d = %d, soft %d). Continuing.\n' % (size,
value, soft_value))
return 0
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772983: kirkwood kernel image is too big
On Fri, 2014-12-12 at 20:59 +, Ben Hutchings wrote: I had originally planned to switch QNAP over for 3.16 but it wasn't quite ready upstream (I've forgotten why). The board files went away in 3.17 so in experimental (v3.18) appending is necessary. Once I've worked out some kinks with kirkwood in v3.18 I was planning to upload a corresponding flash-kernel which does appending for those platforms. In case you hadn't noticed, I've changed the size check in trunk so it can optionally include the size of the largest DTB. That is somewhat pessimistic in that the largest DTB probably won't be used in conjunction with the smallest partition. I had seen, I think it's a good idea despite the short comings. The size difference between the smallest and largest DTB is only 14K so it's pretty close either way. To be more accurate we'd need to store the name of the dtb corresponding to the smallest partition in the kernel source, making the (reasonable) assumption that the second smallest partition is at least 14K larger than the smallest. I don't know that it's worth it though. I enabled this for both kirkwood and orion5x, though now I think I should not have done so for orion5x. I think you were right to do so even for orion5x (in trunk at least). Right now there are only a small numbers of dtbs built on orion5x, and they are all 8K. As upstream transitions from board files to dtb that number (and the sizes) will grow, but it's not skewing our results too much to check now. At some point upstream will remove the board file support for orion5x, so I think it is fine to start checking the sizes now even while we are still on board files, so we are prewarned. [...] 2094680 2097080 2094680 + 10394 = 2105074 2097080 The orion5x machine with the smallest known kernel partition is D-Link DNS-323, with 1572792 bytes space. We currently have less than 1 KB to spare here. Thankfully this machine is still supported by board code and doesn't need a DTB. But if any of the other orion5x machines we intend to support have a similarly small kernel partition and require a DTB they will not work with this version. I don't know much about orion5x, but the flash-kernel db tells me that we don't currently append a dtb on any platform there. 1k is rather tight though, even if appending isn't needed. A security update adding 1k of binary size wouldn't be totally out of the question and it would be unfortunate to have to start disabling features in a security update. Yes, you're right. For comparison, this is what's happened over the course of wheezy stable updates: 3.2.41-2 3.2.63-2 limit growth% growth limit% iop32x:1427968 1434632 1441784 0.47 0.97 ixp4xx:1424696 1428920 1441760 0.30 1.20 kirkwood: 1606512 1613040 2097080 0.41 30.54 orion5x: 1475936 1483632 1572792 0.52 6.56 They've grown by up to about 0.5% over the course of 20 months of a ~36 month support period. That implies we want to allow for about 1% growth from the size in the .0 release. Thankfully they did all start with this much space. Currently in sid we have: 3.16.7-ckt2-1 limit growth limit% ixp4xx:14297121441760 0.84% kirkwood: 20944882097080 0.12% orion5x: 15682481572792 0.29% I misread earlier - kirkwood is about 2.5 KB below the limit, not 1. Anyway, both kirkwood and orion5x have much less than 1% of growth room and ixp4xx has slightly less. I think that some of the config changes I made in trunk should be applied to jessie/sid as well. I agree. I suppose you have some candidates in mind? Ian. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772983: kirkwood kernel image is too big
On Sat, 2014-12-13 at 07:57 +, Ian Campbell wrote: On Fri, 2014-12-12 at 20:59 +, Ben Hutchings wrote: [...] I misread earlier - kirkwood is about 2.5 KB below the limit, not 1. Anyway, both kirkwood and orion5x have much less than 1% of growth room and ixp4xx has slightly less. I think that some of the config changes I made in trunk should be applied to jessie/sid as well. I agree. I suppose you have some candidates in mind? Any of the things I disabled on trunk could reasonably be disabled on sid, except that I want to keep CONFIG_DEBUG_BUGVERBOSE enabled if possible. Note that on sid, config-reduced applies to both ixp4xx and orion5x whereas on trunk it now applies to kirkwood and orion5x. Ben. -- Ben Hutchings Unix is many things to many people, but it's never been everything to anybody. signature.asc Description: This is a digitally signed message part
Bug#772983: kirkwood kernel image is too big
Package: src:linux Version: 3.16.7-2 Severity: serious The kirkwood and orion5x kernel images generally have to be installed in flash partitions with a fixed size. Currently we check at build time that vmlinuz is small enough to fit. However, these platforms now require Device Tree blobs, and the appropriate DTB must be appended to the kernel image in flash since the boot loader won't load it separately. The kirkwood machines with the smallest known kernel partition are QNAP TS-119/TS-219, with 2097080 bytes space. We need to fit: -rw-r--r-- 1 ben ben 2094680 Nov 7 03:37 vmlinuz-3.16.0-4-kirkwood -rw-r--r-- 1 ben ben 10394 Dec 9 04:57 kirkwood-ts219-6281.dtb 2094680 2097080 2094680 + 10394 = 2105074 2097080 The orion5x machine with the smallest known kernel partition is D-Link DNS-323, with 1572792 bytes space. We currently have less than 1 KB to spare here. Thankfully this machine is still supported by board code and doesn't need a DTB. But if any of the other orion5x machines we intend to support have a similarly small kernel partition and require a DTB they will not work with this version. Ben. -- System Information: Debian Release: 8.0 APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental') Architecture: i386 (x86_64) Foreign Architectures: amd64 Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772983: kirkwood kernel image is too big
On Fri, 2014-12-12 at 18:12 +, Ben Hutchings wrote: Package: src:linux Version: 3.16.7-2 Severity: serious The kirkwood and orion5x kernel images generally have to be installed in flash partitions with a fixed size. Currently we check at build time that vmlinuz is small enough to fit. However, these platforms now require Device Tree blobs, and the appropriate DTB must be appended to the kernel image in flash since the boot loader won't load it separately. The kirkwood machines with the smallest known kernel partition are QNAP TS-119/TS-219, with 2097080 bytes space. We need to fit: -rw-r--r-- 1 ben ben 2094680 Nov 7 03:37 vmlinuz-3.16.0-4-kirkwood -rw-r--r-- 1 ben ben 10394 Dec 9 04:57 kirkwood-ts219-6281.dtb We have not switched to dtb appending for the QNAP TS* platforms in Jessie. The board support still existed in v3.16 and we still use it. I had originally planned to switch QNAP over for 3.16 but it wasn't quite ready upstream (I've forgotten why). The board files went away in 3.17 so in experimental (v3.18) appending is necessary. Once I've worked out some kinks with kirkwood in v3.18 I was planning to upload a corresponding flash-kernel which does appending for those platforms. There are other kirkwood platforms with appending enabled in the flash kernel db in Jessie. They are: * Buffalo Linkstation LS-CHLv2 (kirkwood-lschlv2.dtb) * Buffalo Linkstation LS-XHL (kirkwood-lsxhl.dtb) * D-Link DNS-320 NAS (Rev A1) (kirkwood-dns320.dtb) * LaCie Internet Space v2 (kirkwood-is2.dtb) * LaCie Network Space Max v2 (kirkwood-ns2max.dtb) * Globalscale Technologies Dreamplug (kirkwood-dreamplug.dtb) * Marvell GuruPlug Reference Board (kirkwood-guruplug-server-plus.dtb) * (AKA Globalscale Technologies Guruplug Server Plus) * Marvell SheevaPlug Reference Board (kirkwood-sheevaplug.dtb) * (AKA Globalscale Technologies SheevaPlug) * Marvell eSATA SheevaPlug Reference Board (kirkwood-sheevaplug-esata.dtb) * (AKA Globalscale Technologies eSATA SheevaPlug) * Plat'Home OpenBlocksA6 (kirkwood-openblocks_a6.dtb) * Seagate FreeAgent Dockstar (kirkwood-dockstar.dtb) Of all those only D-Link DNS-320 NAS (Rev A1) has the Mtd-kernel field, so all the others must boot from a filesystem not an raw MTD partition. Sadly the db doesn't record the mtd sizes for platforms, so I don't know how much space that model has. kirkwood-dns320.dtb is 13199 bytes though. 2094680 2097080 2094680 + 10394 = 2105074 2097080 The orion5x machine with the smallest known kernel partition is D-Link DNS-323, with 1572792 bytes space. We currently have less than 1 KB to spare here. Thankfully this machine is still supported by board code and doesn't need a DTB. But if any of the other orion5x machines we intend to support have a similarly small kernel partition and require a DTB they will not work with this version. I don't know much about orion5x, but the flash-kernel db tells me that we don't currently append a dtb on any platform there. 1k is rather tight though, even if appending isn't needed. A security update adding 1k of binary size wouldn't be totally out of the question and it would be unfortunate to have to start disabling features in a security update. Ben. -- System Information: Debian Release: 8.0 APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental') Architecture: i386 (x86_64) Foreign Architectures: amd64 Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#772983: kirkwood kernel image is too big
Control: severity -1 important Control: retitle -1 armel kernel images are close to size limits On Fri, 2014-12-12 at 19:36 +, Ian Campbell wrote: On Fri, 2014-12-12 at 18:12 +, Ben Hutchings wrote: Package: src:linux Version: 3.16.7-2 Severity: serious The kirkwood and orion5x kernel images generally have to be installed in flash partitions with a fixed size. Currently we check at build time that vmlinuz is small enough to fit. However, these platforms now require Device Tree blobs, and the appropriate DTB must be appended to the kernel image in flash since the boot loader won't load it separately. The kirkwood machines with the smallest known kernel partition are QNAP TS-119/TS-219, with 2097080 bytes space. We need to fit: -rw-r--r-- 1 ben ben 2094680 Nov 7 03:37 vmlinuz-3.16.0-4-kirkwood -rw-r--r-- 1 ben ben 10394 Dec 9 04:57 kirkwood-ts219-6281.dtb We have not switched to dtb appending for the QNAP TS* platforms in Jessie. The board support still existed in v3.16 and we still use it. OK. I had originally planned to switch QNAP over for 3.16 but it wasn't quite ready upstream (I've forgotten why). The board files went away in 3.17 so in experimental (v3.18) appending is necessary. Once I've worked out some kinks with kirkwood in v3.18 I was planning to upload a corresponding flash-kernel which does appending for those platforms. In case you hadn't noticed, I've changed the size check in trunk so it can optionally include the size of the largest DTB. That is somewhat pessimistic in that the largest DTB probably won't be used in conjunction with the smallest partition. I enabled this for both kirkwood and orion5x, though now I think I should not have done so for orion5x. [...] 2094680 2097080 2094680 + 10394 = 2105074 2097080 The orion5x machine with the smallest known kernel partition is D-Link DNS-323, with 1572792 bytes space. We currently have less than 1 KB to spare here. Thankfully this machine is still supported by board code and doesn't need a DTB. But if any of the other orion5x machines we intend to support have a similarly small kernel partition and require a DTB they will not work with this version. I don't know much about orion5x, but the flash-kernel db tells me that we don't currently append a dtb on any platform there. 1k is rather tight though, even if appending isn't needed. A security update adding 1k of binary size wouldn't be totally out of the question and it would be unfortunate to have to start disabling features in a security update. Yes, you're right. For comparison, this is what's happened over the course of wheezy stable updates: 3.2.41-2 3.2.63-2 limit growth% growth limit% iop32x:1427968 1434632 1441784 0.47 0.97 ixp4xx:1424696 1428920 1441760 0.30 1.20 kirkwood: 1606512 1613040 2097080 0.41 30.54 orion5x: 1475936 1483632 1572792 0.52 6.56 They've grown by up to about 0.5% over the course of 20 months of a ~36 month support period. That implies we want to allow for about 1% growth from the size in the .0 release. Thankfully they did all start with this much space. Currently in sid we have: 3.16.7-ckt2-1 limit growth limit% ixp4xx:14297121441760 0.84% kirkwood: 20944882097080 0.12% orion5x: 15682481572792 0.29% I misread earlier - kirkwood is about 2.5 KB below the limit, not 1. Anyway, both kirkwood and orion5x have much less than 1% of growth room and ixp4xx has slightly less. I think that some of the config changes I made in trunk should be applied to jessie/sid as well. Ben. -- Ben Hutchings Kids! Bringing about Armageddon can be dangerous. Do not attempt it in your own home. - Terry Pratchett and Neil Gaiman, `Good Omens' signature.asc Description: This is a digitally signed message part
