On Fri, 11 Dec 2009 01:34:08 +0000
Phillip Lougher <phil...@lougher.demon.co.uk> wrote:
> Add a config option DECOMPRESS_LZMA_NEEDED which allows subsystems to
> specify they need the unlzma code. Normally decompress_unlzma.c is
> compiled with __init and unlzma is not exported to modules.
>
> Move INIT definition into separate header files for bzip2/lzma/inflate
> so it can be defined differently for each decompressor.
>
This patch (which is in linux-next) breaks
lib-add-support-for-lzo-compressed-kernels.patch, below. The
definition of INIT is no longer available in lib/decompress_unlzo.c, and
lib/decompress_unlzo.c: In function 'unlzo':
lib/decompress_unlzo.c:106: error: 'error' undeclared (first use in this
function)
lib/decompress_unlzo.c:106: error: (Each undeclared identifier is reported only
once
lib/decompress_unlzo.c:106: error: for each function it appears in.)
lib/decompress_unlzo.c:111: error: implicit declaration of function 'error'
I'm planning on merging
zlib-optimize-inffast-when-copying-direct-from-output.patch
lib-add-support-for-lzo-compressed-kernels.patch
arm-add-support-for-lzo-compressed-kernels.patch
x86-add-support-for-lzo-compressed-kernels.patch
add-lzo-compression-support-for-initramfs-and-old-style-initrd.patch
into 2.6.33. I don't immediately remember why I decided that - perhaps
because the patches did arrive in time for .33, but got stalled because
people were screwing around in other trees.
So if I go ahead with that merge, linux-next will need fixing. And I
didn't get down and work what the appropriate fix is, and I don't want
to break linux-next in serious ways.
So what to do? I guess I could go ahead with the mainline merge, and
Stephen drops <whatever that tree was> from linux-next until it has
been fixed up?
If we decide to hold the above five patches off until 2.6.34 then
they'll need to be fixed up to work against linux-next, and they will
be dependent upon your patch series.
From: Albin Tonnerre <albin.tonne...@free-electrons.com>
This patch series adds generic support for creating and extracting
LZO-compressed kernel images, as well as support for using such images on
the x86 and ARM architectures, and support for creating and using
LZO-compressed initrd and initramfs images.
Russell King said:
: Testing on a Cortex A9 model:
: - lzo decompressor is 65% of the time gzip takes to decompress a kernel
: - lzo kernel is 9% larger than a gzip kernel
:
: which I'm happy to say confirms your figures when comparing the two.
:
: However, when comparing your new gzip code to the old gzip code:
: - new is 99% of the size of the old code
: - new takes 42% of the time to decompress than the old code
:
: What this means is that for a proper comparison, the results get even better:
: - lzo is 7.5% larger than the old gzip'd kernel image
: - lzo takes 28% of the time that the old gzip code took
:
: So the expense seems definitely worth the effort. The only reason I
: can think of ever using gzip would be if you needed the additional
: compression (eg, because you have limited flash to store the image.)
:
: I would argue that the default for ARM should therefore be LZO.
This patch:
The lzo compressor is worse than gzip at compression, but faster at
extraction. Here are some figures for an ARM board I'm working on:
Uncompressed size: 3.24Mo
gzip 1.61Mo 0.72s
lzo 1.75Mo 0.48s
So for a compression ratio that is still relatively close to gzip, it's
much faster to extract, at least in that case.
This part contains:
- Makefile routine to support lzo compression
- Fixes to the existing lzo compressor so that it can be used in
compressed kernels
- wrapper around the existing lzo1x_decompress, as it only extracts one
block at a time, while we need to extract a whole file here
- config dialog for kernel compression
[a...@linux-foundation.org: coding-style fixes]
[a...@linux-foundation.org: cleanup]
Signed-off-by: Albin Tonnerre <albin.tonne...@free-electrons.com>
Tested-by: Wu Zhangjin <wuzhang...@gmail.com>
Acked-by: "H. Peter Anvin" <h...@zytor.com>
Cc: Ingo Molnar <mi...@elte.hu>
Cc: Thomas Gleixner <t...@linutronix.de>
Tested-by: Russell King <r...@arm.linux.org.uk>
Acked-by: Russell King <r...@arm.linux.org.uk>
Cc: Ralf Baechle <r...@linux-mips.org>
Signed-off-by: Andrew Morton <a...@linux-foundation.org>
---
include/linux/decompress/unlzo.h | 10 +
init/Kconfig | 18 +-
lib/decompress_unlzo.c | 209 +++++++++++++++++++++++++++++
lib/lzo/lzo1x_decompress.c | 9 -
scripts/Makefile.lib | 5
5 files changed, 244 insertions(+), 7 deletions(-)
diff -puN /dev/null include/linux/decompress/unlzo.h
--- /dev/null
+++ a/include/linux/decompress/unlzo.h
@@ -0,0 +1,10 @@
+#ifndef DECOMPRESS_UNLZO_H
+#define DECOMPRESS_UNLZO_H
+
+int unlzo(unsigned char *inbuf, int len,
+ int(*fill)(void*, unsigned int),
+ int(*flush)(void*, unsigned int),
+ unsigned char *output,
+ int *pos,
+ void(*error)(char *x));
+#endif
diff -puN init/Kconfig~lib-add-support-for-lzo-compressed-kernels init/Kconfig
--- a/init/Kconfig~lib-add-support-for-lzo-compressed-kernels
+++ a/init/Kconfig
@@ -123,10 +123,13 @@ config HAVE_KERNEL_BZIP2
config HAVE_KERNEL_LZMA
bool
+config HAVE_KERNEL_LZO
+ bool
+
choice
prompt "Kernel compression mode"
default KERNEL_GZIP
- depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA
+ depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_LZO
help
The linux kernel is a kind of self-extracting executable.
Several compression algorithms are available, which differ
@@ -149,9 +152,8 @@ config KERNEL_GZIP
bool "Gzip"
depends on HAVE_KERNEL_GZIP
help
- The old and tried gzip compression. Its compression ratio is
- the poorest among the 3 choices; however its speed (both
- compression and decompression) is the fastest.
+ The old and tried gzip compression. It provides a good balance
+ between compression ratio and decompression speed.
config KERNEL_BZIP2
bool "Bzip2"
@@ -172,6 +174,14 @@ config KERNEL_LZMA
two. Compression is slowest. The kernel size is about 33%
smaller with LZMA in comparison to gzip.
+config KERNEL_LZO
+ bool "LZO"
+ depends on HAVE_KERNEL_LZO
+ help
+ Its compression ratio is the poorest among the 4. The kernel
+ size is about about 10% bigger than gzip; however its speed
+ (both compression and decompression) is the fastest.
+
endchoice
config SWAP
diff -puN /dev/null lib/decompress_unlzo.c
--- /dev/null
+++ a/lib/decompress_unlzo.c
@@ -0,0 +1,209 @@
+/*
+ * LZO decompressor for the Linux kernel. Code borrowed from the lzo
+ * implementation by Markus Franz Xaver Johannes Oberhumer.
+ *
+ * Linux kernel adaptation:
+ * Copyright (C) 2009
+ * Albin Tonnerre, Free Electrons <albin.tonne...@free-electrons.com>
+ *
+ * Original code:
+ * Copyright (C) 1996-2005 Markus Franz Xaver Johannes Oberhumer
+ * All Rights Reserved.
+ *
+ * lzop and the LZO library are free software; you can redistribute them
+ * and/or modify them under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING.
+ * If not, write to the Free Software Foundation, Inc.,
+ * 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Markus F.X.J. Oberhumer
+ * <mar...@oberhumer.com>
+ * http://www.oberhumer.com/opensource/lzop/
+ */
+
+#ifdef STATIC
+#include "lzo/lzo1x_decompress.c"
+#else
+#include <linux/slab.h>
+#include <linux/decompress/unlzo.h>
+#endif
+
+#include <linux/types.h>
+#include <linux/lzo.h>
+#include <linux/decompress/mm.h>
+
+#include <linux/compiler.h>
+#include <asm/unaligned.h>
+
+static const unsigned char lzop_magic[] = {
+ 0x89, 0x4c, 0x5a, 0x4f, 0x00, 0x0d, 0x0a, 0x1a, 0x0a };
+
+#define LZO_BLOCK_SIZE (256*1024l)
+#define HEADER_HAS_FILTER 0x00000800L
+
+STATIC inline int INIT parse_header(u8 *input, u8 *skip)
+{
+ int l;
+ u8 *parse = input;
+ u8 level = 0;
+ u16 version;
+
+ /* read magic: 9 first bits */
+ for (l = 0; l < 9; l++) {
+ if (*parse++ != lzop_magic[l])
+ return 0;
+ }
+ /* get version (2bytes), skip library version (2),
+ * 'need to be extracted' version (2) and
+ * method (1) */
+ version = get_unaligned_be16(parse);
+ parse += 7;
+ if (version >= 0x0940)
+ level = *parse++;
+ if (get_unaligned_be32(parse) & HEADER_HAS_FILTER)
+ parse += 8; /* flags + filter info */
+ else
+ parse += 4; /* flags */
+
+ /* skip mode and mtime_low */
+ parse += 8;
+ if (version >= 0x0940)
+ parse += 4; /* skip mtime_high */
+
+ l = *parse++;
+ /* don't care about the file name, and skip checksum */
+ parse += l + 4;
+
+ *skip = parse - input;
+ return 1;
+}
+
+STATIC inline int INIT unlzo(u8 *input, int in_len,
+ int (*fill) (void *, unsigned int),
+ int (*flush) (void *, unsigned int),
+ u8 *output, int *posp,
+ void (*error_fn) (char *x))
+{
+ u8 skip = 0, r = 0;
+ u32 src_len, dst_len;
+ size_t tmp;
+ u8 *in_buf, *in_buf_save, *out_buf;
+ int obytes_processed = 0;
+
+ set_error_fn(error_fn);
+
+ if (output) {
+ out_buf = output;
+ } else if (!flush) {
+ error("NULL output pointer and no flush function provided");
+ goto exit;
+ } else {
+ out_buf = malloc(LZO_BLOCK_SIZE);
+ if (!out_buf) {
+ error("Could not allocate output buffer");
+ goto exit;
+ }
+ }
+
+ if (input && fill) {
+ error("Both input pointer and fill function provided, don't
know what to do");
+ goto exit_1;
+ } else if (input) {
+ in_buf = input;
+ } else if (!fill || !posp) {
+ error("NULL input pointer and missing position pointer or fill
function");
+ goto exit_1;
+ } else {
+ in_buf = malloc(lzo1x_worst_compress(LZO_BLOCK_SIZE));
+ if (!in_buf) {
+ error("Could not allocate input buffer");
+ goto exit_1;
+ }
+ }
+ in_buf_save = in_buf;
+
+ if (posp)
+ *posp = 0;
+
+ if (fill)
+ fill(in_buf, lzo1x_worst_compress(LZO_BLOCK_SIZE));
+
+ if (!parse_header(input, &skip)) {
+ error("invalid header");
+ goto exit_2;
+ }
+ in_buf += skip;
+
+ if (posp)
+ *posp = skip;
+
+ for (;;) {
+ /* read uncompressed block size */
+ dst_len = get_unaligned_be32(in_buf);
+ in_buf += 4;
+
+ /* exit if last block */
+ if (dst_len == 0) {
+ if (posp)
+ *posp += 4;
+ break;
+ }
+
+ if (dst_len > LZO_BLOCK_SIZE) {
+ error("dest len longer than block size");
+ goto exit_2;
+ }
+
+ /* read compressed block size, and skip block checksum info */
+ src_len = get_unaligned_be32(in_buf);
+ in_buf += 8;
+
+ if (src_len <= 0 || src_len > dst_len) {
+ error("file corrupted");
+ goto exit_2;
+ }
+
+ /* decompress */
+ tmp = dst_len;
+ r = lzo1x_decompress_safe((u8 *) in_buf, src_len,
+ out_buf, &tmp);
+
+ if (r != LZO_E_OK || dst_len != tmp) {
+ error("Compressed data violation");
+ goto exit_2;
+ }
+
+ obytes_processed += dst_len;
+ if (flush)
+ flush(out_buf, dst_len);
+ if (output)
+ out_buf += dst_len;
+ if (posp)
+ *posp += src_len + 12;
+ if (fill) {
+ in_buf = in_buf_save;
+ fill(in_buf, lzo1x_worst_compress(LZO_BLOCK_SIZE));
+ } else
+ in_buf += src_len;
+ }
+
+exit_2:
+ if (!input)
+ free(in_buf);
+exit_1:
+ if (!output)
+ free(out_buf);
+exit:
+ return obytes_processed;
+}
+
+#define decompress unlzo
diff -puN lib/lzo/lzo1x_decompress.c~lib-add-support-for-lzo-compressed-kernels
lib/lzo/lzo1x_decompress.c
--- a/lib/lzo/lzo1x_decompress.c~lib-add-support-for-lzo-compressed-kernels
+++ a/lib/lzo/lzo1x_decompress.c
@@ -11,11 +11,13 @@
* Richard Purdie <rpur...@openedhand.com>
*/
+#ifndef STATIC
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/lzo.h>
-#include <asm/byteorder.h>
+#endif
+
#include <asm/unaligned.h>
+#include <linux/lzo.h>
#include "lzodefs.h"
#define HAVE_IP(x, ip_end, ip) ((size_t)(ip_end - ip) < (x))
@@ -244,9 +246,10 @@ lookbehind_overrun:
*out_len = op - out;
return LZO_E_LOOKBEHIND_OVERRUN;
}
-
+#ifndef STATIC
EXPORT_SYMBOL_GPL(lzo1x_decompress_safe);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("LZO1X Decompressor");
+#endif
diff -puN scripts/Makefile.lib~lib-add-support-for-lzo-compressed-kernels
scripts/Makefile.lib
--- a/scripts/Makefile.lib~lib-add-support-for-lzo-compressed-kernels
+++ a/scripts/Makefile.lib
@@ -235,3 +235,8 @@ quiet_cmd_lzma = LZMA $@
cmd_lzma = (cat $(filter-out FORCE,$^) | \
lzma -9 && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
(rm -f $@ ; false)
+
+quiet_cmd_lzo = LZO $@
+cmd_lzo = (cat $(filter-out FORCE,$^) | \
+ lzop -9 && $(call size_append, $(filter-out FORCE,$^))) > $@ || \
+ (rm -f $@ ; false)
_
--
To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
