Re: [PATCH 08/16 v4] pramfs: headers
On Sat, Nov 20, 2010 at 11:00:15AM +0100, Marco Stornelli wrote:
+/*
+ * Debug code
+ */
+#define pram_dbg(s, args...) pr_debug(PRAMFS: s, ## args)
+#define pram_err(s, args...) pr_err(PRAMFS: s, ## args)
+#define pram_warn(s, args...)pr_warning(PRAMFS: s, ## args)
+#define pram_info(s, args...)pr_info(PRAMFS: s, ## args)
+
Please kill off all of this and just use KBUILD_MODNAME centrally.
+#ifdef CONFIG_PRAMFS_WRITE_PROTECT
+extern void pram_writeable(void *vaddr, unsigned long size, int rw);
+
+#define wrprotect(addr, size) pram_writeable(addr, size, 0)
+
+#else
+
+#define wrprotect(addr, size) do {} while (0)
+
+#endif /* CONFIG PRAMFS_WRITE_PROTECT */
+
Perhaps this should be pram_wrprotect()? Does this really benefit from
being a config option instead of a mount option? Will this handle
multiple mounts with some write protected and others not?
+#ifdef CONFIG_PRAMFS_WRITE_PROTECT
+static inline void pram_memunlock_range(void *p, unsigned long len)
+{
+#ifndef CONFIG_X86
+ local_irq_disable();
+#endif
+ preempt_disable();
+ pram_writeable(p, len, 1);
+}
+
This needs some explaining, or killing. While the latter is preferable,
we can also work with the former.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 04/16 v4] pramfs: file operations
2010/11/24 Paul Mundt let...@linux-sh.org:
On Sat, Nov 20, 2010 at 10:58:40AM +0100, Marco Stornelli wrote:
diff -Nurp linux-2.6.36-orig/fs/pramfs/file.c linux-2.6.36/fs/pramfs/file.c
--- linux-2.6.36-orig/fs/pramfs/file.c 1970-01-01 01:00:00.0
+0100
+++ linux-2.6.36/fs/pramfs/file.c 2010-09-24 18:34:03.0 +0200
@@ -0,0 +1,166 @@
+/*
+ * FILE NAME fs/pramfs/file.c
+ *
+ * BRIEF DESCRIPTION
+ *
+ * File operations for files.
+ *
This FILE NAME / BRIEF DESCRIPTION thing should probably die, it's also
not used consistently throughout the series.
Maybe yes, it's more a problem then a advantage :)
+static int pram_open_file(struct inode *inode, struct file *filp)
+{
+#ifndef CONFIG_PRAMFS_XIP
+ /* Without XIP we force to use Direct IO */
+ filp-f_flags |= O_DIRECT;
+#endif
+ return generic_file_open(inode, filp);
+}
+
Relying on a config option for this is in violently poor taste. Rely on
the config option to enable/disable XIP support as you like, but whether
it's actually mounted XIP or not should really depend be determined by a
mount option. Presently you have no way of dealing with the file system
being mounted multiple times with mixed XIP and non-XIP, which doesn't
seem like a particularly exotic configuration. As you seem to have copied
Maybe on embedded, it could be.
most of this from ext2, I'm curious why you opted to hardcode this
instead of maintaining the flexibility that ext2 XIP has over this.
First of all because it was simpler :) In addition there was some
design problem to use it in combination with the memory protection.
The difference with ext2 is that we aren't talking about a general
purpose fs used (mainly) on normal desktop/server systems, but a
specific fs for embedded world, so I think some little constraints are
ok.
Marco
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 09/16 v4] pramfs: dir operations
On Sat, Nov 20, 2010 at 11:00:45AM +0100, Marco Stornelli wrote:
+int pram_add_link(struct dentry *dentry, struct inode *inode)
+{
+ struct inode *dir = dentry-d_parent-d_inode;
+ struct pram_inode *pidir, *pi, *pitail = NULL;
+ u64 tail_ino, prev_ino;
+
+ const char *name = dentry-d_name.name;
+
+ int namelen = dentry-d_name.len PRAM_NAME_LEN ?
+ PRAM_NAME_LEN : dentry-d_name.len;
+
namelen = min_t(unsigned int, dentry-d_name.len, PRAM_NAME_LEN);
?
+#define S_SHIFT 12
+static unsigned int dtype_by_mode[S_IFMT S_SHIFT] = {
+ [S_IFREG S_SHIFT]DT_REG,
+ [S_IFDIR S_SHIFT]DT_DIR,
+ [S_IFCHR S_SHIFT]DT_CHR,
+ [S_IFBLK S_SHIFT]DT_BLK,
+ [S_IFIFO S_SHIFT]DT_FIFO,
+ [S_IFSOCK S_SHIFT] DT_SOCK,
+ [S_IFLNK S_SHIFT]DT_LNK,
+};
+
+static int pram_readdir(struct file *filp, void *dirent, filldir_t filldir)
+{
...
+ ret = filldir(dirent, name, namelen,
+ filp-f_pos, ino,
+ dtype_by_mode[(be16_to_cpu(pi-i_mode)
S_IFMT)S_SHIFT]);
+ filp-f_pos = pi-i_d.d_next ? be64_to_cpu(pi-i_d.d_next) : 3;
You might try to provide some generic helpers for this, at least GFS2
also does the same thing:
include/linux/gfs2_ondisk.h:#define DT2IF(dt) (((dt) 12) S_IFMT)
include/linux/gfs2_ondisk.h:#define IF2DT(sif) (((sif) S_IFMT) 12)
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 04/16 v4] pramfs: file operations
On Wed, Nov 24, 2010 at 09:11:13AM +0100, Marco Stornelli wrote: 2010/11/24 Paul Mundt let...@linux-sh.org: most of this from ext2, I'm curious why you opted to hardcode this instead of maintaining the flexibility that ext2 XIP has over this. First of all because it was simpler :) In addition there was some design problem to use it in combination with the memory protection. Do you have more details on this? You can easily check for unsupportable configurations with mount options and bail out accordingly. The difference with ext2 is that we aren't talking about a general purpose fs used (mainly) on normal desktop/server systems, but a specific fs for embedded world, so I think some little constraints are ok. I'm not sure what your point is. It's not a general purpose file system, but that's not an excuse for taking shortcuts. Out of the boards on my desk, I have at least 3 that could make use of this file system where I could use both XIP and non-XIP for different stores out of the box. I wouldn't exactly call it a corner case. Also, as Tony's patch set demonstrates, these sorts of non-volatile data stores are common enough in the server space to make pramfs an option there, too. Please lose this mentality that because something was originally tasked for embedded it's perfectly acceptable to ship a crippled interface. As it is, this is something that will have to be rewritten one way or the other, but whether that happens in or out of staging/ is not such a big concern. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 08/16 v4] pramfs: headers
On Wed, Nov 24, 2010 at 09:23:02AM +0100, Marco Stornelli wrote:
2010/11/24 Paul Mundt let...@linux-sh.org:
+#ifdef CONFIG_PRAMFS_WRITE_PROTECT
+extern void pram_writeable(void *vaddr, unsigned long size, int rw);
+
+#define wrprotect(addr, size) pram_writeable(addr, size, 0)
+
+#else
+
+#define wrprotect(addr, size) do {} while (0)
+
+#endif /* CONFIG PRAMFS_WRITE_PROTECT */
+
Perhaps this should be pram_wrprotect()? Does this really benefit from
being a config option instead of a mount option? Will this handle
multiple mounts with some write protected and others not?
See my previous response.
Your previous response only alludes to why you didn't feel like making it
a mount option, and doesn't address any of the other questions.
+#ifdef CONFIG_PRAMFS_WRITE_PROTECT
+static inline void pram_memunlock_range(void *p, unsigned long len)
+{
+#ifndef CONFIG_X86
+ ? ? local_irq_disable();
+#endif
+ ? ? preempt_disable();
+ ? ? pram_writeable(p, len, 1);
+}
+
This needs some explaining, or killing. While the latter is preferable,
we can also work with the former.
Maybe I didn't understand, you mean preemt_disable() without disabling
the interrupt?
I mean what exactly is this supposed to be doing? It looks pretty
questionable for SMP for starters, it doesn't explain why x86 is special,
etc. It looks like you wanted a spinlock with IRQs disabled but probably
opted not to use it because you were throwing this around interfaces that
could sleep, which looks like a really scary thing for latency. It's also
making architecture assumptions without any explanation of why. This
needs to be explained, and in some amount of detail, as it's not entirely
obvious.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH RFC 2/3] Decompressors: Add boot-time XZ support
From: Lasse Collin lasse.col...@tukaani.org This implements the API defined in linux/decompress/generic.h which is used for kernel, initramfs, and initrd decompression. This patch together with the first patch is enough for XZ-compressed initramfs and initrd; XZ-compressed kernel will need arch-specific changes. In contrast to other initramfs compression methods, support for XZ-compressed initramfs is not enabled by default in usr/Kconfig. This is primarily due to the Kconfig options of the xz_dec module. It can be good to require that xz_dec is enabled separately so the user can select only the BCJ filters he needs when EMBEDDED=y. The buffering requirements described in decompress_unxz.c are stricter than with gzip, so the relevant changes should be done to the arch-specific code when adding support for XZ-compressed kernel. Similarly, the heap size in arch-specific pre-boot code may need to be increased (30 KiB is enough). The XZ decompressor needs memmove(), memeq() (memcmp() == 0), and memzero() (memset(ptr, 0, size)), which aren't available in all arch-specific pre-boot environments. I'm including simple versions in decompress_unxz.c, but a cleaner solution would naturally be nicer. Signed-off-by: Lasse Collin lasse.col...@tukaani.org --- include/linux/decompress/unxz.h | 20 ++ init/Kconfig| 21 +- lib/Kconfig |3 lib/Makefile|1 lib/decompress.c|5 lib/decompress_unxz.c | 390 scripts/gen_initramfs_list.sh |2 usr/Kconfig | 25 ++ usr/Makefile|5 9 files changed, 464 insertions(+), 8 deletions(-) diff -uprN linux-2.6.37-rc3.orig/include/linux/decompress/unxz.h linux-2.6.37-rc3/include/linux/decompress/unxz.h --- linux-2.6.37-rc3.orig/include/linux/decompress/unxz.h 1970-01-01 02:00:00.0 +0200 +++ linux-2.6.37-rc3/include/linux/decompress/unxz.h2010-11-24 08:10:27.0 +0200 @@ -0,0 +1,20 @@ +/* + * Wrapper for XZ decompressor to make it usable for kernel and initramfs + * decompression + * + * Author: Lasse Collin lasse.col...@tukaani.org + * + * This file has been put into the public domain. + * You can do whatever you want with this file. + */ + +#ifndef DECOMPRESS_UNXZ_H +#define DECOMPRESS_UNXZ_H + +int unxz(unsigned char *in, int in_size, + int (*fill)(void *dest, unsigned int size), + int (*flush)(void *src, unsigned int size), + unsigned char *out, int *in_used, + void (*error)(char *x)); + +#endif diff -uprN linux-2.6.37-rc3.orig/init/Kconfig linux-2.6.37-rc3/init/Kconfig --- linux-2.6.37-rc3.orig/init/Kconfig 2010-11-22 10:51:46.0 +0200 +++ linux-2.6.37-rc3/init/Kconfig 2010-11-24 08:04:29.0 +0200 @@ -130,13 +130,16 @@ config HAVE_KERNEL_BZIP2 config HAVE_KERNEL_LZMA bool +config HAVE_KERNEL_XZ + bool + config HAVE_KERNEL_LZO bool choice prompt Kernel compression mode default KERNEL_GZIP - depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_LZO + depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO help The linux kernel is a kind of self-extracting executable. Several compression algorithms are available, which differ @@ -176,10 +179,20 @@ config KERNEL_LZMA bool LZMA depends on HAVE_KERNEL_LZMA help + This is the predecessor of XZ. + +config KERNEL_XZ + bool XZ + depends on HAVE_KERNEL_XZ + help The most recent compression algorithm. - Its ratio is best, decompression speed is between the other - two. Compression is slowest. The kernel size is about 33% - smaller with LZMA in comparison to gzip. + Its compression ratio is the best. Decompression speed is + better than that of bzip2 but worse than gzip and LZO. + Compression is the slowest. The kernel size is about 30% + smaller with XZ in comparison to gzip. On architectures + for which there is a BCJ filter in XZ (i386, x86_64, ARM, + IA-64, PowerPC, and SPARC), XZ will create a few percent + smaller kernel than LZMA. config KERNEL_LZO bool LZO diff -uprN linux-2.6.37-rc3.orig/lib/Kconfig linux-2.6.37-rc3/lib/Kconfig --- linux-2.6.37-rc3.orig/lib/Kconfig 2010-10-20 23:30:22.0 +0300 +++ linux-2.6.37-rc3/lib/Kconfig2010-11-21 10:42:11.0 +0200 @@ -120,6 +120,9 @@ config DECOMPRESS_BZIP2 config DECOMPRESS_LZMA tristate +config DECOMPRESS_XZ + tristate + config DECOMPRESS_LZO select LZO_DECOMPRESS tristate diff -uprN linux-2.6.37-rc3.orig/lib/Makefile linux-2.6.37-rc3/lib/Makefile --- linux-2.6.37-rc3.orig/lib/Makefile 2010-10-20 23:30:22.0 +0300
[PATCH RFC 3/3] x86: Support XZ-compressed kernel
From: Lasse Collin lasse.col...@tukaani.org This integrates the XZ decompression code to the x86 pre-boot code. mkpiggy.c is updated to reserve about 32 KiB more buffer safety margin for kernel decompression. It is done unconditionally for all decompressors to keep the code simpler. The XZ decompressor needs around 30 KiB of heap, so the heap size is increased to 32 KiB on both x86-32 and x86-64. Documentation/x86/boot.txt is updated to list the XZ magic number. With the x86 BCJ filter in XZ, XZ-compressed x86 kernel tends to be a few percent smaller than the equivalent LZMA-compressed kernel. Signed-off-by: Lasse Collin lasse.col...@tukaani.org --- Even though the buffer safety margin has been increased, the comments in misc.c about the safety margin calculation were not touched. This is because the misc.c comments describe the calculation of the margin for the .gz format. Documentation/x86/boot.txt |6 +++--- arch/x86/Kconfig |1 + arch/x86/boot/compressed/Makefile |5 - arch/x86/boot/compressed/misc.c|4 arch/x86/boot/compressed/mkpiggy.c |2 +- arch/x86/include/asm/boot.h|6 +- 6 files changed, 14 insertions(+), 10 deletions(-) diff -uprN linux-2.6.37-rc3.orig/Documentation/x86/boot.txt linux-2.6.37-rc3/Documentation/x86/boot.txt --- linux-2.6.37-rc3.orig/Documentation/x86/boot.txt2010-10-20 23:30:22.0 +0300 +++ linux-2.6.37-rc3/Documentation/x86/boot.txt 2010-11-18 19:58:57.0 +0200 @@ -621,9 +621,9 @@ Protocol: 2.08+ The payload may be compressed. The format of both the compressed and uncompressed data should be determined using the standard magic numbers. The currently supported compression formats are gzip - (magic numbers 1F 8B or 1F 9E), bzip2 (magic number 42 5A) and LZMA - (magic number 5D 00). The uncompressed payload is currently always ELF - (magic number 7F 45 4C 46). + (magic numbers 1F 8B or 1F 9E), bzip2 (magic number 42 5A), LZMA + (magic number 5D 00), and XZ (magic number FD 37). The uncompressed + payload is currently always ELF (magic number 7F 45 4C 46). Field name:payload_length Type: read diff -uprN linux-2.6.37-rc3.orig/arch/x86/Kconfig linux-2.6.37-rc3/arch/x86/Kconfig --- linux-2.6.37-rc3.orig/arch/x86/Kconfig 2010-11-22 10:51:37.0 +0200 +++ linux-2.6.37-rc3/arch/x86/Kconfig 2010-11-24 08:18:18.0 +0200 @@ -51,6 +51,7 @@ config X86 select HAVE_KERNEL_GZIP select HAVE_KERNEL_BZIP2 select HAVE_KERNEL_LZMA + select HAVE_KERNEL_XZ select HAVE_KERNEL_LZO select HAVE_HW_BREAKPOINT select HAVE_MIXED_BREAKPOINTS_REGS diff -uprN linux-2.6.37-rc3.orig/arch/x86/boot/compressed/Makefile linux-2.6.37-rc3/arch/x86/boot/compressed/Makefile --- linux-2.6.37-rc3.orig/arch/x86/boot/compressed/Makefile 2010-10-20 23:30:22.0 +0300 +++ linux-2.6.37-rc3/arch/x86/boot/compressed/Makefile 2010-11-16 20:36:23.0 +0200 @@ -4,7 +4,7 @@ # create a compressed vmlinux image from the original vmlinux # -targets := vmlinux.lds vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma vmlinux.bin.lzo head_$(BITS).o misc.o string.o cmdline.o early_serial_console.o piggy.o +targets := vmlinux.lds vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma vmlinux.bin.xz vmlinux.bin.lzo head_$(BITS).o misc.o string.o cmdline.o early_serial_console.o piggy.o KBUILD_CFLAGS := -m$(BITS) -D__KERNEL__ $(LINUX_INCLUDE) -O2 KBUILD_CFLAGS += -fno-strict-aliasing -fPIC @@ -49,12 +49,15 @@ $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.al $(call if_changed,bzip2) $(obj)/vmlinux.bin.lzma: $(vmlinux.bin.all-y) FORCE $(call if_changed,lzma) +$(obj)/vmlinux.bin.xz: $(vmlinux.bin.all-y) FORCE + $(call if_changed,xzkern) $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE $(call if_changed,lzo) suffix-$(CONFIG_KERNEL_GZIP) := gz suffix-$(CONFIG_KERNEL_BZIP2) := bz2 suffix-$(CONFIG_KERNEL_LZMA) := lzma +suffix-$(CONFIG_KERNEL_XZ) := xz suffix-$(CONFIG_KERNEL_LZO):= lzo quiet_cmd_mkpiggy = MKPIGGY $@ diff -uprN linux-2.6.37-rc3.orig/arch/x86/boot/compressed/misc.c linux-2.6.37-rc3/arch/x86/boot/compressed/misc.c --- linux-2.6.37-rc3.orig/arch/x86/boot/compressed/misc.c 2010-11-22 10:51:37.0 +0200 +++ linux-2.6.37-rc3/arch/x86/boot/compressed/misc.c2010-11-24 08:18:18.0 +0200 @@ -139,6 +139,10 @@ static int lines, cols; #include ../../../../lib/decompress_unlzma.c #endif +#ifdef CONFIG_KERNEL_XZ +#include ../../../../lib/decompress_unxz.c +#endif + #ifdef CONFIG_KERNEL_LZO #include ../../../../lib/decompress_unlzo.c #endif diff -uprN linux-2.6.37-rc3.orig/arch/x86/boot/compressed/mkpiggy.c linux-2.6.37-rc3/arch/x86/boot/compressed/mkpiggy.c --- linux-2.6.37-rc3.orig/arch/x86/boot/compressed/mkpiggy.c2010-10-20 23:30:22.0 +0300 +++
Re: [PATCH RFC 1/3] Decompressors: Add XZ decompressor module
On Wed, 24 Nov 2010 22:51:52 +0200 Lasse Collin lasse.col...@tukaani.org wrote: From: Lasse Collin lasse.col...@tukaani.org In userspace, the .lzma format has become mostly a legacy file format that got superseded by the .xz format. Similarly, LZMA Utils was superseded by XZ Utils. These patches add support for XZ decompression into the kernel. Most of the code is as is from XZ Embedded http://tukaani.org/xz/embedded.html. It was written for the Linux kernel but is usable in other projects too. Advantages of XZ over the current LZMA code in the kernel: - Nice API that can be used by other kernel modules; it's not limited to kernel, initramfs, and initrd decompression. - Integrity check support (CRC32) - BCJ filters improve compression of executable code on certain architectures. These together with LZMA2 can produce a few percent smaller kernel or Squashfs images than plain LZMA without making the decompression slower. This patch: Add the main decompression code (xz_dec), testing module (xz_dec_test), wrapper script (xz_wrap.sh) for the xz command line tool, and documentation. The xz_dec module is enough to have a usable XZ decompressor e.g. for Squashfs. I'm not seeing any documentation which tells me how to create, install and execute xs-compressed kernels. There are new makefile targets? ... +#define bcj_x86_test_msbyte(b) ((b) == 0x00 || (b) == 0xFF) This should be written in C. It looks nicer, and so bcj_x86_test_msbyte(*p++) won't explode. +static noinline_for_stack size_t bcj_x86( hm, but it uses little stack space. +static noinline_for_stack size_t bcj_x86( + struct xz_dec_bcj *s, uint8_t *buf, size_t size) The preferred style is static noinline_for_stack size_t bcj_x86(struct xz_dec_bcj *s, uint8_t *buf, size_t size) or static noinline_for_stack size_t bcj_x86(struct xz_dec_bcj *s, uint8_t *buf, size_t size) (lots of dittoes) -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC 1/3] Decompressors: Add XZ decompressor module
On 11/24/2010 02:50 PM, H. Peter Anvin wrote: Logically it should be an additional CONFIG option like we currently have for the other compression formats. And so it is (it's in the latter patches). -hpa -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC 2/3] Decompressors: Add boot-time XZ support
On 25/11/10 06:32, Phillip Lougher wrote:
+
+ for (i = 0; i size; ++i)
The kernel uses either i size or isize ...
Disregard these, looks like my email client (Mozilla Thunderbird)
is eating spaces... Very odd
+ if (fill == NULL flush == NULL) {
and this.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
