On 10/15/21 10:29 AM, Richard Purdie wrote:
On Fri, 2021-10-15 at 19:06 +0200, Alexander Kanavin wrote:
This allows significantly faster do_populate_sdk() (5 min -> 3 min for
the sato sdk on my machine) and decompression, at the cost of SDK file
taking up more disk space.
Can you quantify "more"?
I'd be interested to know if a higher compression helps and what the time/cost
tradeoff is too. Not sure I'm keep to have it increase in size. This likely
increases the eSDK size a lot too?
since eSDKs are generate once use many times compression time is not as
important, decompression time is important, however, they are also
transmitted via network which means we might lose out the decompression
gain towards network transfer time. Perhaps it will be good to get a
wholesome picture.
If that's a significant concern, it's possible to increase the zstd
compression level or even go back to xz compression.
Signed-off-by: Alexander Kanavin <a...@linutronix.de>
---
meta/classes/populate_sdk_base.bbclass | 10 ++++++++--
meta/files/toolchain-shar-extract.sh | 9 ++++-----
2 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/meta/classes/populate_sdk_base.bbclass
b/meta/classes/populate_sdk_base.bbclass
index 49e166e697..92a23e2325 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -50,17 +50,23 @@ TOOLCHAIN_TARGET_TASK_ATTEMPTONLY ?= ""
TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-toolchain-${SDK_VERSION}"
# Default archived SDK's suffix
-SDK_ARCHIVE_TYPE ?= "tar.xz"
+SDK_ARCHIVE_TYPE ?= "tar.zst"
SDK_XZ_COMPRESSION_LEVEL ?= "-9"
SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${SDK_XZ_COMPRESSION_LEVEL}"
-# To support different sdk type according to SDK_ARCHIVE_TYPE, now support zip and tar.xz
+SDK_ZSTD_COMPRESSION_LEVEL ?= ""
+SDK_ZSTD_OPTIONS ?= "-T${ZSTD_THREADS} ${SDK_ZSTD_COMPRESSION_LEVEL}"
+
+# To support different sdk type according to SDK_ARCHIVE_TYPE, now support
zip, tar.zst and tar.xz
python () {
if d.getVar('SDK_ARCHIVE_TYPE') == 'zip':
d.setVar('SDK_ARCHIVE_DEPENDS', 'zip-native')
# SDK_ARCHIVE_CMD used to generate archived sdk
${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE} from input dir
${SDK_OUTPUT}/${SDKPATH} to output dir ${SDKDEPLOYDIR}
# recommand to cd into input dir first to avoid archive with buildpath
d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; zip -r -y
${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE} .')
+ elif d.getVar('SDK_ARCHIVE_TYPE') == 'tar.zst':
+ d.setVar('SDK_ARCHIVE_DEPENDS', 'zstd-native')
+ d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; tar
${SDKTAROPTS} -cf - . | zstd ${SDK_ZSTD_OPTIONS} >
${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}')
else:
d.setVar('SDK_ARCHIVE_DEPENDS', 'xz-native')
d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; tar
${SDKTAROPTS} -cf - . | xz ${SDK_XZ_OPTIONS} >
${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}')
diff --git a/meta/files/toolchain-shar-extract.sh
b/meta/files/toolchain-shar-extract.sh
index 4386b985bb..4b8892af79 100644
--- a/meta/files/toolchain-shar-extract.sh
+++ b/meta/files/toolchain-shar-extract.sh
@@ -51,11 +51,6 @@ if [ "$INST_ARCH" != "$SDK_ARCH" ]; then
fi
fi
-if ! xz -V > /dev/null 2>&1; then
- echo "Error: xz is required for installation of this SDK, please install it
first"
- exit 1
-fi
-
Deleting that is fine for testing but I think the final patch needs to do
better! ;-)
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157007):
https://lists.openembedded.org/g/openembedded-core/message/157007
Mute This Topic: https://lists.openembedded.org/mt/86353954/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
