On 9 April 2018 at 22:20, Khem Raj <raj.k...@gmail.com> wrote:
> On Mon, Apr 9, 2018 at 1:44 PM, Burton, Ross <ross.bur...@intel.com> wrote:
>> On 9 April 2018 at 21:13, Andre McCurdy <armccu...@gmail.com> wrote:
>>> On Mon, Apr 9, 2018 at 8:19 AM, Ross Burton <ross.bur...@intel.com> wrote:
>>>> xz has native support for threaded compression now and SDK creation was 
>>>> the only
>>>> part of oe-core which is using pixz instead of xz.
>>>>
>>>> Not only does this remove pixz-native from the SDK dependencies, but in my
>>>> limited testing xz -T0 is slightly faster and produces smaller archives 
>>>> than
>>>> pixz for the same input.
>>>>
>>>> Signed-off-by: Ross Burton <ross.bur...@intel.com>
>>>> ---
>>>>
>>>>  # We want the MULTIARCH_TARGET_SYS to point to the TUNE_PKGARCH, not 
>>>> PACKAGE_ARCH as it
>>>> @@ -225,7 +225,7 @@ fakeroot tar_sdk() {
>>>>         # Package it up
>>>>         mkdir -p ${SDKDEPLOYDIR}
>>>>         cd ${SDK_OUTPUT}/${SDKPATH}
>>>> -       tar ${SDKTAROPTS} -cf - . | pixz > 
>>>> ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.tar.xz
>>>> +       tar ${SDKTAROPTS} -cf - . | xz -T 0 > 
>>>> ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.tar.xz
>>>
>>> Since -T 0 will use all available CPUs (ie any attempts the user may
>>> have made to limit parallelism via BB_NUMBER_THREADS or PARALLEL_MAKE
>>> will be ignored), perhaps it's worth adding something like
>>> "--memlimit=70%" to try to give some protection for environments with
>>> lots of CPUs but not so much DRAM?
>>
>> There's a few places where -T0 is passed already: tar.xz image
>> creation and opkg creation, so I guess we need to centralise this
>> somewhere too.
>>
>
> May be use XZ_OPT env bariable and param to -T could use the bitbake
> calculated number of CPUs

-T0 is use as many threads as there are cores, which is the same as
the bitbake function.  Re-using BB_NUMBER_THREADS might work.

Ross
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Reply via email to