Re: [OE-core] [PATCH 1/5] RFC image_types.bbclass: add image size limit for tar image type
On Thu, 2018-11-29 at 14:17 +, mikko.rap...@bmw.de wrote: > On Thu, Nov 29, 2018 at 02:04:14PM +, Richard Purdie wrote: > > On Thu, 2018-11-29 at 14:21 +0200, Mikko Rapeli wrote: > > > If IMAGE_ROOTFS_SIZE_LIMIT is defined in image configuration, > > > then > > > build will fail if for tar image type the uncompressed tar ball > > > size > > > exceeds the value, as reported by 'du -ks'. > > > > > > This check could be extended to other image types as well. > > > > > > There already exists a check with IMAGE_ROOTFS_SIZE variable > > > but I could not figure out how to actually use it. It does > > > some quite complex overhead calculations which did not seem > > > to work for me on sumo. > > > > > > When the image size is exceeded, build fails like this: > > > > > > ERROR: image-1.0-r0 do_image_tar: Image size 170712 of > > > /home/builder/src/base/build/tmp/work/linux/image/1.0-r0/deploy- > > > image-complete/image.rootfs.tar reported by 'du -ks' is larger > > > than > > > limit IMAGE_ROOTFS_SIZE_LIMIT 17 > > > > What about IMAGE_ROOTFS_MAXSIZE? > > > > I think IMAGE_ROOTFS_SIZE is about adding extra space to the image, > > not > > limiting its size... > > Yea, forgot to mention that I tried that too but got inconsisten > results. > I know it's bad but it was easier for me to add this new test than to > figure out what's wrong with current yocto image size checks. Hence > RFC. How would we document this new variable? Recommend users to set both and hope for the best? :) Seriously, we need one good way of doing this which works. That means either you debug the IMAGE_ROOTFS_MAXSIZE option or at least file a bug with as much information as you can about the problems you're seeing... Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/5] RFC image_types.bbclass: add image size limit for tar image type
This seems like it’d make a good general image qa check instead, rather than being part of do_image_tar. On Thu, Nov 29, 2018 at 7:29 AM wrote: > On Thu, Nov 29, 2018 at 02:04:14PM +, Richard Purdie wrote: > > On Thu, 2018-11-29 at 14:21 +0200, Mikko Rapeli wrote: > > > If IMAGE_ROOTFS_SIZE_LIMIT is defined in image configuration, then > > > build will fail if for tar image type the uncompressed tar ball size > > > exceeds the value, as reported by 'du -ks'. > > > > > > This check could be extended to other image types as well. > > > > > > There already exists a check with IMAGE_ROOTFS_SIZE variable > > > but I could not figure out how to actually use it. It does > > > some quite complex overhead calculations which did not seem > > > to work for me on sumo. > > > > > > When the image size is exceeded, build fails like this: > > > > > > ERROR: image-1.0-r0 do_image_tar: Image size 170712 of > > > /home/builder/src/base/build/tmp/work/linux/image/1.0-r0/deploy- > > > image-complete/image.rootfs.tar reported by 'du -ks' is larger than > > > limit IMAGE_ROOTFS_SIZE_LIMIT 17 > > > > What about IMAGE_ROOTFS_MAXSIZE? > > > > I think IMAGE_ROOTFS_SIZE is about adding extra space to the image, not > > limiting its size... > > Yea, forgot to mention that I tried that too but got inconsisten results. > I know it's bad but it was easier for me to add this new test than to > figure > out what's wrong with current yocto image size checks. Hence RFC. > > -Mikko > > > Cheers, > > > > Richard > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > -- Christopher Larson kergoth at gmail dot com Founder - BitBake, OpenEmbedded, OpenZaurus Senior Software Engineer, Mentor Graphics -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/5] RFC image_types.bbclass: add image size limit for tar image type
On Thu, Nov 29, 2018 at 02:04:14PM +, Richard Purdie wrote: > On Thu, 2018-11-29 at 14:21 +0200, Mikko Rapeli wrote: > > If IMAGE_ROOTFS_SIZE_LIMIT is defined in image configuration, then > > build will fail if for tar image type the uncompressed tar ball size > > exceeds the value, as reported by 'du -ks'. > > > > This check could be extended to other image types as well. > > > > There already exists a check with IMAGE_ROOTFS_SIZE variable > > but I could not figure out how to actually use it. It does > > some quite complex overhead calculations which did not seem > > to work for me on sumo. > > > > When the image size is exceeded, build fails like this: > > > > ERROR: image-1.0-r0 do_image_tar: Image size 170712 of > > /home/builder/src/base/build/tmp/work/linux/image/1.0-r0/deploy- > > image-complete/image.rootfs.tar reported by 'du -ks' is larger than > > limit IMAGE_ROOTFS_SIZE_LIMIT 17 > > What about IMAGE_ROOTFS_MAXSIZE? > > I think IMAGE_ROOTFS_SIZE is about adding extra space to the image, not > limiting its size... Yea, forgot to mention that I tried that too but got inconsisten results. I know it's bad but it was easier for me to add this new test than to figure out what's wrong with current yocto image size checks. Hence RFC. -Mikko > Cheers, > > Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/5] RFC image_types.bbclass: add image size limit for tar image type
On Thu, 2018-11-29 at 14:21 +0200, Mikko Rapeli wrote: > If IMAGE_ROOTFS_SIZE_LIMIT is defined in image configuration, then > build will fail if for tar image type the uncompressed tar ball size > exceeds the value, as reported by 'du -ks'. > > This check could be extended to other image types as well. > > There already exists a check with IMAGE_ROOTFS_SIZE variable > but I could not figure out how to actually use it. It does > some quite complex overhead calculations which did not seem > to work for me on sumo. > > When the image size is exceeded, build fails like this: > > ERROR: image-1.0-r0 do_image_tar: Image size 170712 of > /home/builder/src/base/build/tmp/work/linux/image/1.0-r0/deploy- > image-complete/image.rootfs.tar reported by 'du -ks' is larger than > limit IMAGE_ROOTFS_SIZE_LIMIT 17 What about IMAGE_ROOTFS_MAXSIZE? I think IMAGE_ROOTFS_SIZE is about adding extra space to the image, not limiting its size... Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/5] RFC image_types.bbclass: add image size limit for tar image type
If IMAGE_ROOTFS_SIZE_LIMIT is defined in image configuration, then
build will fail if for tar image type the uncompressed tar ball size
exceeds the value, as reported by 'du -ks'.
This check could be extended to other image types as well.
There already exists a check with IMAGE_ROOTFS_SIZE variable
but I could not figure out how to actually use it. It does
some quite complex overhead calculations which did not seem
to work for me on sumo.
When the image size is exceeded, build fails like this:
ERROR: image-1.0-r0 do_image_tar: Image size 170712 of
/home/builder/src/base/build/tmp/work/linux/image/1.0-r0/deploy-image-complete/image.rootfs.tar
reported by 'du -ks' is larger than limit IMAGE_ROOTFS_SIZE_LIMIT 17
Signed-off-by: Mikko Rapeli
---
meta/classes/image.bbclass | 2 +-
meta/classes/image_types.bbclass | 11 ++-
2 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 276d0d3..34a567f 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -127,7 +127,7 @@ def rootfs_variables(d):
'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS',
'MULTILIBRE_ALLOW_REP','MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS',
'PACKAGE_ARCHS','PACKAGE_CLASSES','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','USE_DEVFS',
- 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED',
'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY',
'REPRODUCIBLE_TIMESTAMP_ROOTFS', 'IMAGE_INSTALL_DEBUGFS']
+ 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED',
'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY',
'REPRODUCIBLE_TIMESTAMP_ROOTFS', 'IMAGE_INSTALL_DEBUGFS',
'IMAGE_ROOTFS_SIZE_LIMIT']
variables.extend(rootfs_command_variables(d))
variables.extend(variable_depends(d))
return " ".join(variables)
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 5c40648..0481703 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -125,7 +125,16 @@ IMAGE_CMD_squashfs-lz4 = "mksquashfs ${IMAGE_ROOTFS}
${IMGDEPLOYDIR}/${IMAGE_NAM
# required when extracting, but it seems prudent to use it in both cases.
IMAGE_CMD_TAR ?= "tar"
# ignore return code 1 "file changed as we read it" as other tasks(e.g.
do_image_wic) may be hardlinking rootfs
-IMAGE_CMD_tar = "${IMAGE_CMD_TAR} --numeric-owner -cf
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar -C ${IMAGE_ROOTFS} . || [
$? -eq 1 ]"
+IMAGE_CMD_tar() {
+ set -ex
+ "${IMAGE_CMD_TAR}" --numeric-owner -cf
"${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar" -C "${IMAGE_ROOTFS}" .
|| [ $? -eq 1 ]
+ # fail build if IMAGE_ROOTFS_SIZE_LIMIT is exceeded
+ if [ -n "${IMAGE_ROOTFS_SIZE_LIMIT}" ]; then
+ imagesize=$( du -ks
"${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar" | awk '{ print $1 }' )
+ [ "${imagesize}" -gt "${IMAGE_ROOTFS_SIZE_LIMIT}" ] && \
+ bberror "Image size ${imagesize} of
${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.tar reported by 'du -ks' is
larger than limit IMAGE_ROOTFS_SIZE_LIMIT ${IMAGE_ROOTFS_SIZE_LIMIT}"
+ fi
+}
do_image_cpio[cleandirs] += "${WORKDIR}/cpio_append"
IMAGE_CMD_cpio () {
--
1.9.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
