Re: [PATCH 1/2] lib: decompress_unzstd: Limit output size

> On Aug 21, 2020, at 9:29 AM, Paul Cercueil wrote: > > The zstd decompression code, as it is right now, will have internal > values overflow on 32-bit systems when the output size is LONG_MAX. > > Until someone smarter than me can figure out how to fix the zstd code > properly, limit the

Re: [PATCH 2/2] MIPS: Add support for ZSTD-compressed kernels

> On Aug 21, 2020, at 9:29 AM, Paul Cercueil wrote: > > Add support for self-extracting kernels with a ZSTD compression. > > Tested on a kernel for the GCW-Zero, it allows to reduce the size of the > kernel file from 4.1 MiB with gzip to 3.5 MiB with ZSTD, and boots just > as fast. > >

Re: [PATCH] lz4: Fix kernel decompression speed

> On Aug 3, 2020, at 6:56 PM, Arvind Sankar wrote: > > On Mon, Aug 03, 2020 at 10:55:01PM +0000, Nick Terrell wrote: >> >> >>> On Aug 3, 2020, at 2:57 PM, Arvind Sankar wrote: >>> >>> On Mon, Aug 03, 2020 at 12:40:22PM -0

Re: [PATCH] lz4: Fix kernel decompression speed

> On Aug 4, 2020, at 8:16 AM, Arvind Sankar wrote: > > On Tue, Aug 04, 2020 at 10:32:36AM +0200, Pavel Machek wrote: >> Hi! >> > I've measured the kernel decompression speed using QEMU before and after > this patch for the x86_64 and i386 architectures. The speed-up is about > 10x

Re: [PATCH] lz4: Fix kernel decompression speed

> On Aug 3, 2020, at 6:56 PM, Arvind Sankar wrote: > > On Mon, Aug 03, 2020 at 10:55:01PM +0000, Nick Terrell wrote: >> >> >>> On Aug 3, 2020, at 2:57 PM, Arvind Sankar wrote: >>> >>> On Mon, Aug 03, 2020 at 12:40:22PM -0

Re: [PATCH] lz4: Fix kernel decompression speed

> On Aug 3, 2020, at 2:57 PM, Arvind Sankar wrote: > > On Mon, Aug 03, 2020 at 12:40:22PM -0700, Nick Terrell wrote: >> From: Nick Terrell >> >> This patch replaces all memcpy() calls with LZ4_memcpy() which calls >> __builtin_memcpy() so the compiler can inl

[PATCH] lz4: Fix kernel decompression speed

From: Nick Terrell This patch replaces all memcpy() calls with LZ4_memcpy() which calls __builtin_memcpy() so the compiler can inline it. LZ4 relies heavily on memcpy() with a constant size being inlined. In x86 and i386 pre-boot environments memcpy() cannot be inlined because memcpy() doesn't

Re: [PATCH v10 7/8] .gitignore: add ZSTD-compressed files

> On Jul 31, 2020, at 2:50 AM, Ingo Molnar wrote: > > > * Nick Terrell wrote: > >> From: Adam Borowski >> >> For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more >> will come, thus let's be consistent with all other com

[PATCH v10 7/8] .gitignore: add ZSTD-compressed files

From: Adam Borowski For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more will come, thus let's be consistent with all other compressors. Tested-by: Sedat Dilek Reviewed-by: Kees Cook Signed-off-by: Nick Terrell Signed-off-by: Adam Borowski --- .gitignore | 1 + 1 file

[PATCH v10 8/8] Documentation: dontdiff: Add zstd compressed files

From: Nick Terrell For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more will come, thus let's be consistent with all other compressors. Signed-off-by: Nick Terrell --- Documentation/dontdiff | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/dontdiff b

[PATCH v10 6/8] x86: Add support for ZSTD compressed kernel

From: Nick Terrell * Add support for zstd compressed kernel * Define __DISABLE_EXPORTS in Makefile * Remove __DISABLE_EXPORTS definition from kaslr.c * Bump the heap size for zstd. * Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires

[PATCH v10 1/8] lib: prepare zstd for preboot environment

From: Nick Terrell * Remove a double definition of the CHECK_F macro when the zstd library is amalgamated. * Switch ZSTD_copy8() to __builtin_memcpy(), because in the preboot environment on x86 gcc can't inline `memcpy()` otherwise. * Limit the gcc hack in ZSTD_wildcopy() to the broken gcc

[PATCH v10 2/8] lib: add zstd support to decompress

From: Nick Terrell * Add unzstd() and the zstd decompress interface. * Add zstd support to decompress_method(). The decompress_method() and unzstd() functions are used to decompress the initramfs and the initrd. The __decompress() function is used in the preboot environment to decompress a zstd

[GIT PULL][PATCH v10 0/8] Add support for ZSTD-compressed kernel and initramfs

From: Nick Terrell Please pull from g...@github.com:terrelln/linux.git tags/v10-zstd to get these changes. Alternatively the patchset is included. Hi all, This patch set adds support for a ZSTD-compressed kernel, ramdisk, and initramfs in the kernel boot process. ZSTD-compressed ramdisk

[PATCH v10 3/8] init: add support for zstd compressed kernel

From: Nick Terrell * Adds the zstd and zstd22 cmds to scripts/Makefile.lib * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- Makefile

[PATCH v10 4/8] usr: add support for zstd compressed initramfs

From: Nick Terrell * Add support for a zstd compressed initramfs. * Add compression for compressing built-in initramfs with zstd. I have tested this patch by boot testing with buildroot and QEMU. Specifically, I booted the kernel with both a zstd and gzip compressed initramfs, both built

[PATCH v10 5/8] x86: bump ZO_z_extra_bytes margin for zstd

From: Nick Terrell Bump the ZO_z_extra_bytes margin for zstd. Zstd needs 3 bytes per 128 KB, and has a 22 byte fixed overhead. Zstd needs to maintain 128 KB of space at all times, since that is the maximum block size. See the comments regarding in-place decompression added in lib

Re: [PATCH v9 7/7] .gitignore: add ZSTD-compressed files

> On Jul 27, 2020, at 6:53 PM, Masahiro Yamada wrote: > > On Tue, Jul 28, 2020 at 8:11 AM Nick Terrell wrote: >> >> From: Adam Borowski >> >> For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more >> will come, thus let's b

Re: [PATCH v9 6/7] x86: Add support for ZSTD compressed kernel

> On Jul 27, 2020, at 7:24 PM, Arvind Sankar wrote: > > On Mon, Jul 27, 2020 at 04:06:05PM -0700, Nick Terrell wrote: >> From: Nick Terrell >> >> * Define __DISABLE_EXPORTS in misc.c > ... >> __DISABLE_EXPORTS is defined in misc.c instead of the

[PATCH v9 5/7] x86: bump ZO_z_extra_bytes margin for zstd

From: Nick Terrell Bump the ZO_z_extra_bytes margin for zstd. Zstd needs 3 bytes per 128 KB, and has a 22 byte fixed overhead. Zstd needs to maintain 128 KB of space at all times, since that is the maximum block size. See the comments regarding in-place decompression added in lib

[PATCH v9 7/7] .gitignore: add ZSTD-compressed files

From: Adam Borowski For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more will come, thus let's be consistent with all other compressors. Tested-by: Sedat Dilek Reviewed-by: Kees Cook Signed-off-by: Nick Terrell Signed-off-by: Adam Borowski --- .gitignore | 1 + 1 file

[GIT PULL][PATCH v9 0/7] Add support for ZSTD-compressed kernel and initramfs

From: Nick Terrell Please pull from g...@github.com:terrelln/linux.git tags/v9-zstd to get these changes. Alternatively the patchset is included. Hi all, This patch set adds support for a ZSTD-compressed kernel, ramdisk, and initramfs in the kernel boot process. ZSTD-compressed ramdisk

[PATCH v9 6/7] x86: Add support for ZSTD compressed kernel

From: Nick Terrell * Add support for zstd compressed kernel * Define __DISABLE_EXPORTS in misc.c * Bump the heap size for zstd. * Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires slightly more memory during the kernel decompression

[PATCH v9 3/7] init: add support for zstd compressed kernel

From: Nick Terrell * Adds the zstd and zstd22 cmds to scripts/Makefile.lib * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- Makefile

[PATCH v9 4/7] usr: add support for zstd compressed initramfs

From: Nick Terrell * Add support for a zstd compressed initramfs. * Add compression for compressing built-in initramfs with zstd. I have tested this patch by boot testing with buildroot and QEMU. Specifically, I booted the kernel with both a zstd and gzip compressed initramfs, both built

[PATCH v9 2/7] lib: add zstd support to decompress

From: Nick Terrell * Add unzstd() and the zstd decompress interface. * Add zstd support to decompress_method(). The decompress_method() and unzstd() functions are used to decompress the initramfs and the initrd. The __decompress() function is used in the preboot environment to decompress a zstd

[PATCH v9 1/7] lib: prepare zstd for preboot environment

From: Nick Terrell * Remove a double definition of the CHECK_F macro when the zstd library is amalgamated. * Switch ZSTD_copy8() to __builtin_memcpy(), because in the preboot environment on x86 gcc can't inline `memcpy()` otherwise. * Limit the gcc hack in ZSTD_wildcopy() to the broken gcc

Re: [PATCH v8 6/7] x86: Add support for ZSTD compressed kernel

> On Jul 24, 2020, at 7:30 AM, Arvind Sankar wrote: > > On Fri, Jul 24, 2020 at 02:50:34AM +0200, Sedat Dilek wrote: >> On Thu, Jul 23, 2020 at 9:30 PM Nick Terrell wrote: >>> >>> From: Nick Terrell >>> >>> * Add support for zstd compres

Re: [PATCH v8 6/7] x86: Add support for ZSTD compressed kernel

> On Jul 24, 2020, at 5:26 AM, Ingo Molnar wrote: > > > * Nick Terrell wrote: > >> --- a/arch/x86/boot/compressed/misc.c >> +++ b/arch/x86/boot/compressed/misc.c >> @@ -12,6 +12,11 @@ >> * High loaded stuff by Hans Lermen & Werner Almesberger

[PATCH v8 5/7] x86: bump ZO_z_extra_bytes margin for zstd

From: Nick Terrell Bump the ZO_z_extra_bytes margin for zstd. Zstd needs 3 bytes per 128 KB, and has a 22 byte fixed overhead. Zstd needs to maintain 128 KB of space at all times, since that is the maximum block size. See the comments regarding in-place decompression added in lib

[PATCH v8 1/7] lib: prepare zstd for preboot environment

From: Nick Terrell * Remove a double definition of the CHECK_F macro when the zstd library is amalgamated. * Switch ZSTD_copy8() to __builtin_memcpy(), because in the preboot environment on x86 gcc can't inline `memcpy()` otherwise. * Limit the gcc hack in ZSTD_wildcopy() to the broken gcc

[PATCH v8 6/7] x86: Add support for ZSTD compressed kernel

From: Nick Terrell * Add support for zstd compressed kernel * Define __DISABLE_EXPORTS in misc.c * Bump the heap size for zstd. * Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires slightly more memory during the kernel decompression

[PATCH v8 2/7] lib: add zstd support to decompress

From: Nick Terrell * Add unzstd() and the zstd decompress interface. * Add zstd support to decompress_method(). The decompress_method() and unzstd() functions are used to decompress the initramfs and the initrd. The __decompress() function is used in the preboot environment to decompress a zstd

[PATCH v8 4/7] usr: add support for zstd compressed initramfs

From: Nick Terrell * Add support for a zstd compressed initramfs. * Add compression for compressing built-in initramfs with zstd. I have tested this patch by boot testing with buildroot and QEMU. Specifically, I booted the kernel with both a zstd and gzip compressed initramfs, both built

[PATCH v8 7/7] .gitignore: add ZSTD-compressed files

From: Adam Borowski For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more will come, thus let's be consistent with all other compressors. Tested-by: Sedat Dilek Reviewed-by: Kees Cook Signed-off-by: Nick Terrell Signed-off-by: Adam Borowski --- .gitignore | 1 + 1 file

[PATCH v8 3/7] init: add support for zstd compressed kernel

From: Nick Terrell * Adds the zstd and zstd22 cmds to scripts/Makefile.lib * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- Makefile

[GIT PULL][PATCH v8 0/7] Add support for ZSTD-compressed kernel and initramfs

From: Nick Terrell Please pull from g...@github.com:terrelln/linux.git tags/v8-zstd to get these changes. Alternatively the patchset is included. Hi all, This patch set adds support for a ZSTD-compressed kernel, ramdisk, and initramfs in the kernel boot process. ZSTD-compressed ramdisk

Re: [PATCH v7 2/7] lib: add zstd support to decompress

> On Jul 14, 2020, at 5:42 PM, Arvind Sankar wrote: > > On Wed, Jul 08, 2020 at 11:50:19AM -0700, Nick Terrell wrote: >> From: Nick Terrell >> >> * Add unzstd() and the zstd decompress interface. >> * Add zstd support to decompress_method(). >>

[PATCH v7 7/7] .gitignore: add ZSTD-compressed files

From: Adam Borowski For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more will come, thus let's be consistent with all other compressors. Tested-by: Sedat Dilek Reviewed-by: Kees Cook Signed-off-by: Nick Terrell Signed-off-by: Adam Borowski --- .gitignore | 1 + 1 file

[PATCH v7 4/7] usr: add support for zstd compressed initramfs

From: Nick Terrell * Add support for a zstd compressed initramfs. * Add compression for compressing built-in initramfs with zstd. I have tested this patch by boot testing with buildroot and QEMU. Specifically, I booted the kernel with both a zstd and gzip compressed initramfs, both built

[PATCH v7 5/7] x86: bump ZO_z_extra_bytes margin for zstd

From: Nick Terrell Bump the ZO_z_extra_bytes margin for zstd. Zstd needs 3 bytes per 128 KB, and has a 22 byte fixed overhead. Zstd needs to maintain 128 KB of space at all times, since that is the maximum block size. See the comments regarding in-place decompression added in lib

[PATCH v7 6/7] x86: Add support for ZSTD compressed kernel

From: Nick Terrell * Add support for zstd compressed kernel * Bump the heap size for zstd. * Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires slightly more memory during the kernel decompression on x86 (192 KB vs 64 KB

[PATCH v7 2/7] lib: add zstd support to decompress

From: Nick Terrell * Add unzstd() and the zstd decompress interface. * Add zstd support to decompress_method(). The decompress_method() and unzstd() functions are used to decompress the initramfs and the initrd. The __decompress() function is used in the preboot environment to decompress a zstd

[PATCH v7 3/7] init: add support for zstd compressed kernel

From: Nick Terrell * Adds the zstd and zstd22 cmds to scripts/Makefile.lib * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- Makefile

[PATCH v7 1/7] lib: prepare zstd for preboot environment

From: Nick Terrell * Remove a double definition of the CHECK_F macro when the zstd library is amalgamated. * Switch ZSTD_copy8() to __builtin_memcpy(), because in the preboot environment on x86 gcc can't inline `memcpy()` otherwise. * Limit the gcc hack in ZSTD_wildcopy() to the broken gcc

[GIT PULL][PATCH v7 0/7] Add support for ZSTD-compressed kernel and initramfs

From: Nick Terrell Please pull from g...@github.com:terrelln/linux.git tags/v7-zstd to get these changes. Alternatively the patchset is included. Hi all, This patch set adds support for a ZSTD-compressed kernel, ramdisk, and initramfs in the kernel boot process. ZSTD-compressed ramdisk

Re: [PATCH v6 2/8] lib: prepare xxhash for preboot environment

> On Jul 7, 2020, at 5:59 PM, Arvind Sankar wrote: > > On Mon, Jul 06, 2020 at 08:45:58PM -0700, Nick Terrell wrote: >> From: Nick Terrell >> >> Don't export symbols if XXH_PREBOOT is defined. >> >> This change is necessary to get xxhash to work in a

Re: [PATCH v6 4/8] init: add support for zstd compressed kernel

> On Jul 7, 2020, at 6:11 PM, Arvind Sankar wrote: > > On Mon, Jul 06, 2020 at 08:46:00PM -0700, Nick Terrell wrote: >> From: Nick Terrell >> >> * Adds the zstd cmd to scripts/Makefile.lib >> * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options >> &g

Re: [PATCH v6 4/8] init: add support for zstd compressed kernel

> On Jul 7, 2020, at 3:19 AM, Norbert Lange wrote: > > Thanks for the respin. > > Am Di., 7. Juli 2020 um 05:51 Uhr schrieb Nick Terrell > : >> >> From: Nick Terrell >> >> * Adds the zstd cmd to scripts/Makefile.lib >> * A

[PATCH v6 8/8] .gitignore: add ZSTD-compressed files

From: Adam Borowski For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more will come, thus let's be consistent with all other compressors. Tested-by: Sedat Dilek Reviewed-by: Kees Cook Signed-off-by: Nick Terrell Signed-off-by: Adam Borowski --- .gitignore | 1 + 1 file

[PATCH v6 6/8] x86: bump ZO_z_extra_bytes margin for zstd

From: Nick Terrell Bump the ZO_z_extra_bytes margin for zstd. Zstd needs 3 bytes per 128 KB, and has a 22 byte fixed overhead. Zstd needs to maintain 128 KB of space at all times, since that is the maximum block size. See the comments regarding in-place decompression added in lib

[PATCH v6 2/8] lib: prepare xxhash for preboot environment

From: Nick Terrell Don't export symbols if XXH_PREBOOT is defined. This change is necessary to get xxhash to work in a preboot environment, which is needed to support zstd-compressed kernels. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- lib/xxhash.c | 21

[PATCH v6 1/8] lib: prepare zstd for preboot environment

From: Nick Terrell * Don't export symbols if ZSTD_PREBOOT is defined. * Remove a double definition of the CHECK_F macro when the zstd library is amalgamated. * Switch ZSTD_copy8() to __builtin_memcpy(), because in the preboot environment on x86 gcc can't inline `memcpy()` otherwise. * Limit

[PATCH v6 3/8] lib: add zstd support to decompress

From: Nick Terrell * Add unzstd() and the zstd decompress interface. * Add zstd support to decompress_method(). The decompress_method() and unzstd() functions are used to decompress the initramfs and the initrd. The __decompress() function is used in the preboot environment to decompress a zstd

[PATCH v6 4/8] init: add support for zstd compressed kernel

From: Nick Terrell * Adds the zstd cmd to scripts/Makefile.lib * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options Architecture specific support is still needed for decompression. Reviewed-by: Kees Cook Tested-by: Sedat Dilek Signed-off-by: Nick Terrell --- init/Kconfig | 15

[PATCH v6 7/8] x86: Add support for ZSTD compressed kernel

From: Nick Terrell * Add support for zstd compressed kernel * Bump the heap size for zstd. * Update the documentation. Integrates the ZSTD decompression code to the x86 pre-boot code. Zstandard requires slightly more memory during the kernel decompression on x86 (192 KB vs 64 KB

[PATCH v6 5/8] usr: add support for zstd compressed initramfs

From: Nick Terrell * Add support for a zstd compressed initramfs. * Add compression for compressing built-in initramfs with zstd. I have tested this patch by boot testing with buildroot and QEMU. Specifically, I booted the kernel with both a zstd and gzip compressed initramfs, both built

[GIT PULL][PATCH v6 0/8] Add support for ZSTD-compressed kernel and initramfs

From: Nick Terrell Please pull from g...@github.com:terrelln/linux.git tags/v6-zstd to get these changes. Alternatively the patchset is included. Hi all, This patch set adds support for a ZSTD-compressed kernel, ramdisk, and initramfs in the kernel boot process. ZSTD-compressed ramdisk

Re: Kernel compression benchmarks

> On Jul 3, 2020, at 12:06 PM, Kees Cook wrote: > > On Fri, Jul 03, 2020 at 10:15:20AM +0200, Sedat Dilek wrote: >> On Thu, Jul 2, 2020 at 5:18 PM Kees Cook wrote: >>> >>> On Wed, Jul 01, 2020 at 10:35:48AM -0400, Alex Xu (Hello71) wrote: ZSTD compression patches have been sent in a

Re: [GIT PULL][PATCH v5 0/8] Add support for ZSTD-compressed kernel and initramfs

> On Jun 1, 2020, at 2:58 PM, Norbert Lange wrote: > > The series seems to be stuck in limbo, and I got the hint to bring > this to Andrew's attention [1]. > Hope this will finally end in upstream, been using these patches for ~2 years. Thanks for the CC!

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

> On Jun 7, 2019, at 4:06 PM, Adam Borowski wrote: > > On Fri, Jun 07, 2019 at 07:20:46PM +0000, Nick Terrell wrote: >> We'd love to get this mainlined as well! >> >> We're using these patches internally as well. We're seeing an improvement on >> an >

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

> On Jun 5, 2019, at 7:43 AM, René Rebe wrote: > > Hey there, > > just wanted to check about the status of zstd-compressed kernel. > It works great for us, and the initrd part I tested on PPC, PPC64, SPARC64, > HPPA, MIPS64 and probably other random things that I may forgot about in the >

Re: Kernel LZO compressor

> On Mar 25, 2019, at 6:13 AM, Dave Rodgman wrote: > > On 19/03/2019 8:15 pm, Nick Terrell wrote: >> Hi Dave, >> >> I just saw you patches adding LZO-RLE, so I decided to fuzz the LZO >> compressor and decompressor. I didn't find any crashe

Kernel LZO compressor

gned)out[i]); fprintf(stderr, "\nret = %d\n", ret); } assert(ret == LZO_E_OK); assert(rtsize == size); assert(memcmp(data, rt, size) == 0); free(out); free(rt); return 0; } ``` [1] https://github.com/terrelln/lzo-userspace-fuzz Best, Nick Terrell

Re: [PATCH 00/11] btrfs: add zstd compression level support

> On Jan 29, 2019, at 9:18 AM, David Sterba wrote: > > On Mon, Jan 28, 2019 at 04:24:26PM -0500, Dennis Zhou wrote: >> As mentioned above, a requirement that differs zstd from zlib is that >> higher levels of compression require more memory. To manage this, each >> compression level has its

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

time? I've been reverting back to gzip recently >> because I care very much about that. > > Too lazy to benchmark a kernel image (IIRC Nick Terrell posted that a while > ago), here's copypasta of a random 16824672 byte executable, in userspace, > with default level setting: &

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

time? I've been reverting back to gzip recently >> because I care very much about that. > > Too lazy to benchmark a kernel image (IIRC Nick Terrell posted that a while > ago), here's copypasta of a random 16824672 byte executable, in userspace, > with default level setting: &

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

> On Mar 21, 2018, at 6:29 PM, Nick Terrell wrote: > This patch set adds support for a ZSTD-compressed kernel and ramdisk > images in the kernel boot process. It only integrates the support with > x86, though the first patch is generic to all architectures. Hi all, Is there anyth

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

> On Mar 21, 2018, at 6:29 PM, Nick Terrell wrote: > This patch set adds support for a ZSTD-compressed kernel and ramdisk > images in the kernel boot process. It only integrates the support with > x86, though the first patch is generic to all architectures. Hi all, Is there anyth

Re: [RESEND PATCH] crypto: Add zstd support

> On Mar 30, 2018, at 10:14 AM, Herbert Xu <herb...@gondor.apana.org.au> wrote: > > On Thu, Mar 22, 2018 at 01:32:30PM +0900, Sergey Senozhatsky wrote: >> On (03/21/18 15:49), Nick Terrell wrote: >>> depends on CONFIG_CRYPTO_ZSTD, which isn't defined until this patc

Re: [RESEND PATCH] crypto: Add zstd support

> On Mar 30, 2018, at 10:14 AM, Herbert Xu wrote: > > On Thu, Mar 22, 2018 at 01:32:30PM +0900, Sergey Senozhatsky wrote: >> On (03/21/18 15:49), Nick Terrell wrote: >>> depends on CONFIG_CRYPTO_ZSTD, which isn't defined until this patch is in >>

[RESEND PATCH] crypto: Add zstd support

to pstore [0], which depends on crypto zstd. [0] lkml.kernel.org/r/9c9416b2dff19f05fb4c35879aaa83d11ff72c92.1521626182.git.geliangt...@gmail.com Signed-off-by: Nick Terrell <terre...@fb.com> --- crypto/Kconfig | 9 ++ crypto/Makefile | 1 + crypto/testmgr.c | 10 +++ crypto/testmgr.h

[RESEND PATCH] crypto: Add zstd support

to pstore [0], which depends on crypto zstd. [0] lkml.kernel.org/r/9c9416b2dff19f05fb4c35879aaa83d11ff72c92.1521626182.git.geliangt...@gmail.com Signed-off-by: Nick Terrell --- crypto/Kconfig | 9 ++ crypto/Makefile | 1 + crypto/testmgr.c | 10 +++ crypto/testmgr.h | 71

[RESEND PATCH] crypto: Add zstd support

Adds zstd support to crypto and scompress. Only supports the default level. Previously we held off on this patch, since there weren't any users. Now zram is ready for zstd support, but depends on CONFIG_CRYPTO_ZSTD, which isn't defined until this patch is in. I also see a patch adding zstd to

[RESEND PATCH] crypto: Add zstd support

Adds zstd support to crypto and scompress. Only supports the default level. Previously we held off on this patch, since there weren't any users. Now zram is ready for zstd support, but depends on CONFIG_CRYPTO_ZSTD, which isn't defined until this patch is in. I also see a patch adding zstd to

Re: [PATCH 1/1] lz4: Implement lz4 with dynamic offset length.

On (03/21/18 10:10), Maninder Singh wrote: > diff --git a/lib/lz4/lz4_compress.c b/lib/lz4/lz4_compress.c > index cc7b6d4..185c358 100644 > --- a/lib/lz4/lz4_compress.c > +++ b/lib/lz4/lz4_compress.c > @@ -183,7 +183,8 @@ static FORCE_INLINE int LZ4_compress_generic( > const tableType_t

Re: [PATCH 1/1] lz4: Implement lz4 with dynamic offset length.

On (03/21/18 10:10), Maninder Singh wrote: > diff --git a/lib/lz4/lz4_compress.c b/lib/lz4/lz4_compress.c > index cc7b6d4..185c358 100644 > --- a/lib/lz4/lz4_compress.c > +++ b/lib/lz4/lz4_compress.c > @@ -183,7 +183,8 @@ static FORCE_INLINE int LZ4_compress_generic( > const tableType_t

Re: [PATCH 0/1] cover-letter/lz4: Implement lz4 with dynamic offset length.

On (03/21/18 10:10), Maninder Singh wrote: > LZ4 specification defines 2 byte offset length for 64 KB data. > But in case of ZRAM we compress data per page and in most of > architecture PAGE_SIZE is 4KB. So we can decide offset length based > on actual offset value. For this we can reserve 1 bit

Re: [PATCH 0/1] cover-letter/lz4: Implement lz4 with dynamic offset length.

On (03/21/18 10:10), Maninder Singh wrote: > LZ4 specification defines 2 byte offset length for 64 KB data. > But in case of ZRAM we compress data per page and in most of > architecture PAGE_SIZE is 4KB. So we can decide offset length based > on actual offset value. For this we can reserve 1 bit

Re: [btrfs_mount] general protection fault: 0000 [#1] SMP

> On Nov 29, 2017, at 6:21 PM, Fengguang Wu wrote: > > Hello, > > FYI this happens in mainline kernel 4.15.0-rc1. > It looks like a new regression. Bisect is in progress. > > It occurs in 11 out of 11 xfstests run. > > [ 1456.361614] > [ 1456.918942] BTRFS info

Re: [btrfs_mount] general protection fault: 0000 [#1] SMP

> On Nov 29, 2017, at 6:21 PM, Fengguang Wu wrote: > > Hello, > > FYI this happens in mainline kernel 4.15.0-rc1. > It looks like a new regression. Bisect is in progress. > > It occurs in 11 out of 11 xfstests run. > > [ 1456.361614] > [ 1456.918942] BTRFS info (device vdb): disk space

Re: [PATCH] lib: zstd: bitstream: Mark expected switch fall-throughs

com> Looks good to me. We had the same patch in upstream PR#690 [1]. I believe that PR#695 [2] was also necessary to quiet the warning. [1] https://github.com/facebook/zstd/pull/690/files [2] https://github.com/facebook/zstd/pull/695/files Reviewed-by: Nick Terrell <terre...@fb.com>

Re: [PATCH] lib: zstd: bitstream: Mark expected switch fall-throughs

PR#690 [1]. I believe that PR#695 [2] was also necessary to quiet the warning. [1] https://github.com/facebook/zstd/pull/690/files [2] https://github.com/facebook/zstd/pull/695/files Reviewed-by: Nick Terrell

[PATCH v2 2/2] x86: Add support for ZSTD-compressed kernel

compression for decompressing the ramdisk image, since streaming mode is used. Newer versions of zstd (1.3.2+) list the window size of a file with `zstd -lv '. The absolute maximum amount of memory required is just over 8 MB. Signed-off-by: Nick Terrell <terre...@fb.com> --- Documentation/x86/bo

[PATCH v2 2/2] x86: Add support for ZSTD-compressed kernel

compression for decompressing the ramdisk image, since streaming mode is used. Newer versions of zstd (1.3.2+) list the window size of a file with `zstd -lv '. The absolute maximum amount of memory required is just over 8 MB. Signed-off-by: Nick Terrell --- Documentation/x86/boot.txt| 6

[PATCH v2 1/2] lib: Add support for ZSTD-compressed kernel

to be a __builtin_memcpy() for performance. The gcc-7.1 bug in ZSTD_wildcopy() was fixed in gcc-7.2, so it can be gated, since it hurts performance. Signed-off-by: Nick Terrell <terre...@fb.com> --- v1 -> v2: - Fix performance bug in zstd decompression - Gate gcc bug workaround to buggy versions incl

[PATCH v2 1/2] lib: Add support for ZSTD-compressed kernel

to be a __builtin_memcpy() for performance. The gcc-7.1 bug in ZSTD_wildcopy() was fixed in gcc-7.2, so it can be gated, since it hurts performance. Signed-off-by: Nick Terrell --- v1 -> v2: - Fix performance bug in zstd decompression - Gate gcc bug workaround to buggy versions include/linux/decompress/unzst

[PATCH v2 0/2] Add support for ZSTD-compressed kernel

. Thanks, Nick Terrell Changelog: v1 -> v2: - Fix performance bug in zstd decompression - Gate gcc bug workaround to buggy versions - Benchmark kernel compression Nick Terrell (2): lib: Add support for ZSTD-compressed kernel x86: Add support for ZSTD-compressed kernel Documentation/x86/boot.

[PATCH v2 0/2] Add support for ZSTD-compressed kernel

. Thanks, Nick Terrell Changelog: v1 -> v2: - Fix performance bug in zstd decompression - Gate gcc bug workaround to buggy versions - Benchmark kernel compression Nick Terrell (2): lib: Add support for ZSTD-compressed kernel x86: Add support for ZSTD-compressed kernel Documentation/x86/boot.

Re: [PATCH 0/2] Add support for ZSTD-compressed kernel

On 10/10/17, 5:08 PM, "Adam Borowski" <kilob...@angband.pl> wrote: > On Tue, Oct 10, 2017 at 10:40:13PM +, Nick Terrell wrote: > > On 10/10/17, 2:56 PM, "h...@zytor.com" <h...@zytor.com> wrote: > > >On October 10, 2017 2:22:42 PM PDT, Nick Ter

Re: [PATCH 0/2] Add support for ZSTD-compressed kernel

On 10/10/17, 5:08 PM, "Adam Borowski" wrote: > On Tue, Oct 10, 2017 at 10:40:13PM +, Nick Terrell wrote: > > On 10/10/17, 2:56 PM, "h...@zytor.com" wrote: > > >On October 10, 2017 2:22:42 PM PDT, Nick Terrell wrote: > > >>This patch set add

Re: [PATCH 0/2] Add support for ZSTD-compressed kernel

On 10/10/17, 2:56 PM, "h...@zytor.com" <h...@zytor.com> wrote: >On October 10, 2017 2:22:42 PM PDT, Nick Terrell <terre...@fb.com> wrote: >>Hi, >> >>This patch set adds support for a ZSTD-compressed kernel and ramdisk >>images in the kernel boot p

Re: [PATCH 0/2] Add support for ZSTD-compressed kernel

On 10/10/17, 2:56 PM, "h...@zytor.com" wrote: >On October 10, 2017 2:22:42 PM PDT, Nick Terrell wrote: >>Hi, >> >>This patch set adds support for a ZSTD-compressed kernel and ramdisk >>images in the kernel boot process. It only integrates the support with >

[PATCH 2/2] x86: Add support for ZSTD-compressed kernel

compression for decompressing the ramdisk image, since streaming mode is used. Newer versions of zstd (1.3.2+) list the window size of a file with `zstd -lv '. The absolute maximum amount of memory required is just over 8 MB. Signed-off-by: Nick Terrell <terre...@fb.com> --- Documentation/x86/bo

[PATCH 2/2] x86: Add support for ZSTD-compressed kernel

compression for decompressing the ramdisk image, since streaming mode is used. Newer versions of zstd (1.3.2+) list the window size of a file with `zstd -lv '. The absolute maximum amount of memory required is just over 8 MB. Signed-off-by: Nick Terrell --- Documentation/x86/boot.txt| 6

[PATCH 1/2] lib: Add support for ZSTD-compressed kernel

function requires memory proportional to the window size used during compression, which is limited to 8 MB. The maximum memory usage is just over 8 MB. Signed-off-by: Nick Terrell <terre...@fb.com> --- include/linux/decompress/unzstd.h | 26 +++ init/Kconfig | 14 +

[PATCH 1/2] lib: Add support for ZSTD-compressed kernel

function requires memory proportional to the window size used during compression, which is limited to 8 MB. The maximum memory usage is just over 8 MB. Signed-off-by: Nick Terrell --- include/linux/decompress/unzstd.h | 26 +++ init/Kconfig | 14 +- lib/Kconfig

[PATCH 0/2] Add support for ZSTD-compressed kernel

'. The absolute maximum amount of memory required is just over 8 MB. Thanks, Nick Terrell Nick Terrell (2): lib: Add support for ZSTD-compressed kernel x86: Add support for ZSTD-compressed kernel Documentation/x86/boot.txt| 6 +- arch/x86/Kconfig | 1 + arch/x86/boot

[PATCH 0/2] Add support for ZSTD-compressed kernel

'. The absolute maximum amount of memory required is just over 8 MB. Thanks, Nick Terrell Nick Terrell (2): lib: Add support for ZSTD-compressed kernel x86: Add support for ZSTD-compressed kernel Documentation/x86/boot.txt| 6 +- arch/x86/Kconfig | 1 + arch/x86/boot

[PREVIEW] Add x86 zstd kernel compression

On 10/5/17, 1:18 AM, "René Rebe" <r...@exactcode.com> wrote: > > Hi Nick, > > On Oct 5, 2017, at 1:29, Nick Terrell <terre...@fb.com> wrote: > >> On 10/4/17, 3:01 AM, "linux-kernel-ow...@vger.kernel.org on behalf of René >> Rebe" <linu

<    1   2   3   >