On Fri, Feb 8, 2019 at 7:27 PM Andre McCurdy <[email protected]> wrote: > > On Fri, Feb 8, 2019 at 5:29 PM Khem Raj <[email protected]> wrote: > > > > when building with opkg backend and huge packages e.g. chromium/llvm all > > going in parallel, memory pressure causes xz to catapult with > > > > do_package_write_ipk: Failed to create package, opkg-build failed with: xz: > > (stdin): Cannot allocate memory > > > > since there are many tasks going on in parallel, xz adds to memory pressure > > and it wants it all, put an upper limit for memory xz can use > > > > We add a variable XZ_MAXRAM with 70% of RAM limit and can be customized > > if builders have more memory one can set it like > > > > XZMAX_RAM = "-M 0" > > This needs a little more explanation. How does a change to command > used to create .xz images help when running out of memory in > do_package_write_ipk? Doesn't the image get created a lot later? >
its not related to image alone. xz is used in various places e.g. opkgs switched to using xz as default compression format recently, so every ipk creation task involves xz, and you might be in multiple of ipk creation tasks based on task parallelism hope that clarifies. If not feel free to ask more > > Signed-off-by: Khem Raj <[email protected]> > > Cc: Alejandro del Castillo <[email protected]> > > --- > > meta/classes/image_types.bbclass | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/meta/classes/image_types.bbclass > > b/meta/classes/image_types.bbclass > > index 70bd315306..9c8ea0643c 100644 > > --- a/meta/classes/image_types.bbclass > > +++ b/meta/classes/image_types.bbclass > > @@ -57,6 +57,7 @@ def imagetypes_getdepends(d): > > XZ_COMPRESSION_LEVEL ?= "-3" > > XZ_INTEGRITY_CHECK ?= "crc32" > > XZ_THREADS ?= "-T 0" > > +XZ_MAXRAM ?= "-M 70%" > > > > ZIP_COMPRESSION_LEVEL ?= "-9" > > > > @@ -284,7 +285,7 @@ CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip sum > > md5sum sha1sum sha224sum sha25 > > CONVERSION_CMD_lzma = "lzma -k -f -7 > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}" > > CONVERSION_CMD_gz = "pigz -f -9 -n -c > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.gz" > > CONVERSION_CMD_bz2 = "pbzip2 -f -k > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}" > > -CONVERSION_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} > > --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.xz" > > +CONVERSION_CMD_xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} ${XZ_THREADS} > > ${XZ_MAXRAM} --check=${XZ_INTEGRITY_CHECK} > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.xz" > > CONVERSION_CMD_lz4 = "lz4 -9 -z -l > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.lz4" > > CONVERSION_CMD_lzo = "lzop -9 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}" > > CONVERSION_CMD_zip = "zip ${ZIP_COMPRESSION_LEVEL} > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.zip > > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}" > > -- > > 2.20.1 > > > > -- > > _______________________________________________ > > Openembedded-core mailing list > > [email protected] > > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
