Re: [OE-core] [master][PATCH] image_types: add 7-Zip support in conversion types and commands

2022-11-06 Thread Livius 
I resend this patch via " git send-email".
https://lists.openembedded.org/g/openembedded-core/message/172765

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172767): 
https://lists.openembedded.org/g/openembedded-core/message/172767
Mute This Topic: https://lists.openembedded.org/mt/93256288/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [master][PATCH] image_types: add 7-Zip support in conversion types and commands

2022-08-26 Thread Livius 
I can not use 'git send-email' because my workstation is not configured to be 
able to send any mail. You can use my patch, the content is available under 
'--' what is need to for it, or I can copy and paste it and re-send to this 
thread. But in 2022, I really recommend to you to improve your contributing 
processes more. It could be much better in a modern pull-request ticket in 
GitHub, GitLab or Bitbucket portal then this historical mode. That kind of 
process is too legacy, and it can not help in developing speed and to get 
more-more users/contributors who want to help in daily developing.

My goal was replacing wic.zip conversion, because it is extremely slowly for an 
image which is several GB sizes. Problem is that normal zip can operate only in 
single-thread mode, but 7-zip can do it in multi-thread mode in much faster if 
I use BZip2 method for a zip archive. I liked to provide .zip extension because 
BZip2 support it, and it is fine for multiplatform usage, in 100% you can unzip 
it in Windows, Linux and OSX too. So, my new wic.7zip can operate in 100% CPU 
load in each 8 cores of my CPU, therefore this is the best for making a .zip 
file for wic image. But if someone wants to use any other mode, they can do 
with variables overriding.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169918): 
https://lists.openembedded.org/g/openembedded-core/message/169918
Mute This Topic: https://lists.openembedded.org/mt/93256288/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [master][PATCH] image_types: add 7-Zip support in conversion types and commands

2022-08-26 Thread Luca Ceresoli via lists.openembedded.org 
Hello Livius,

thank you for your patch!

I have a few comments, see below.

On Thu, 25 Aug 2022 12:59:34 -0700
"Livius"  wrote:

> I added to support 7-Zip in conversion types/commands. It is fully 
> configurable in compression level, method and file extension.
> 
> From: "Benjamin Szőke" 
> Date: Thu, 25 Aug 2022 21:45:55 +0200
> Subject: [PATCH] image_types: add 7-Zip support in conversion types and 
> commands

These 3 lines should be in the mail header, not in the body. All the
lines that appear in the body before the '---' line will be part of the
commit message, and of course we don't want these 3 lines in the commit
message.

I recommend you to use 'git send-email' to send patches.

See this link  (from README.md in the meta-yocto sources) for more info:
https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded

> ---
> meta/classes-recipe/image_types.bbclass | 8 +++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/image_types.bbclass 
> b/meta/classes-recipe/image_types.bbclass
> index a731e585b2..94aa1d9510 100644
> --- a/meta/classes-recipe/image_types.bbclass
> +++ b/meta/classes-recipe/image_types.bbclass
> @@ -59,6 +59,10 @@ XZ_INTEGRITY_CHECK ?= "crc32"
> 
> ZIP_COMPRESSION_LEVEL ?= "-9"
> 
> +7ZIP_COMPRESSION_LEVEL ?= "9"
> +7ZIP_COMPRESSION_METHOD ?= "BZip2"

Did you run some performance test to check that this is actually a good
balance between compression ratio and compression time?

> +7ZIP_EXTENSION ?= "zip"

I think the 7zip default extension is '7z', why defaulting to 'zip'?
Seems like generating confusion for no reason.

-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169907): 
https://lists.openembedded.org/g/openembedded-core/message/169907
Mute This Topic: https://lists.openembedded.org/mt/93256288/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [master][PATCH] image_types: add 7-Zip support in conversion types and commands

2022-08-25 Thread Livius 
I added to support 7-Zip in conversion types/commands. It is fully configurable 
in compression level, method and file extension.

From: "Benjamin Szőke" 
Date: Thu, 25 Aug 2022 21:45:55 +0200
Subject: [PATCH] image_types: add 7-Zip support in conversion types and commands

---
meta/classes-recipe/image_types.bbclass | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/image_types.bbclass 
b/meta/classes-recipe/image_types.bbclass
index a731e585b2..94aa1d9510 100644
--- a/meta/classes-recipe/image_types.bbclass
+++ b/meta/classes-recipe/image_types.bbclass
@@ -59,6 +59,10 @@ XZ_INTEGRITY_CHECK ?= "crc32"

ZIP_COMPRESSION_LEVEL ?= "-9"

+7ZIP_COMPRESSION_LEVEL ?= "9"
+7ZIP_COMPRESSION_METHOD ?= "BZip2"
+7ZIP_EXTENSION ?= "zip"
+
ZSTD_COMPRESSION_LEVEL ?= "-3"

JFFS2_SUM_EXTRA_ARGS ?= ""
@@ -296,7 +300,7 @@ IMAGE_TYPES:append:x86-64 = " hddimg iso"
# CONVERSION_CMD/DEPENDS.
COMPRESSIONTYPES ?= ""

-CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip zst sum md5sum sha1sum sha224sum 
sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync 
zsync ${COMPRESSIONTYPES}"
+CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1sum 
sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 
base64 gzsync zsync ${COMPRESSIONTYPES}"
CONVERSION_CMD:lzma = "lzma -k -f -7 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
CONVERSION_CMD:gz = "gzip -f -9 -n -c --rsyncable 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.gz"
CONVERSION_CMD:bz2 = "pbzip2 -f -k ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
@@ -304,6 +308,7 @@ CONVERSION_CMD:xz = "xz -f -k -c ${XZ_COMPRESSION_LEVEL} 
${XZ_DEFAULTS} --check=
CONVERSION_CMD:lz4 = "lz4 -9 -z -l ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.lz4"
CONVERSION_CMD:lzo = "lzop -9 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
CONVERSION_CMD:zip = "zip ${ZIP_COMPRESSION_LEVEL} 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.zip 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
+CONVERSION_CMD:7zip = "7za a -mx=${7ZIP_COMPRESSION_LEVEL} 
-mm=${7ZIP_COMPRESSION_METHOD} 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.${7ZIP_EXTENSION} 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
CONVERSION_CMD:zst = "zstd -f -k -T0 -c ${ZSTD_COMPRESSION_LEVEL} 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.zst"
CONVERSION_CMD:sum = "sumtool -i ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} -o 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sum ${JFFS2_SUM_EXTRA_ARGS}"
CONVERSION_CMD:md5sum = "md5sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > 
${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.md5sum"
@@ -329,6 +334,7 @@ CONVERSION_DEPENDS_xz = "xz-native"
CONVERSION_DEPENDS_lz4 = "lz4-native"
CONVERSION_DEPENDS_lzo = "lzop-native"
CONVERSION_DEPENDS_zip = "zip-native"
+CONVERSION_DEPENDS_7zip = "p7zip-native"
CONVERSION_DEPENDS_zst = "zstd-native"
CONVERSION_DEPENDS_sum = "mtd-utils-native"
CONVERSION_DEPENDS_bmap = "bmap-tools-native"
--
2.35.1.windows.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169887): 
https://lists.openembedded.org/g/openembedded-core/message/169887
Mute This Topic: https://lists.openembedded.org/mt/93256288/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-