Re: [PATCH 6/7] Squashfs: Directly decompress into the page cache for file data
On 22/11/13 07:40, Roman Peniaev wrote: Hello, Phillip. one remark below: +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page) +{ + struct inode *i = target_page->mapping->host; + struct squashfs_cache_entry *buffer = squashfs_get_datablock(i- i_sb, +block, bsize); + int bytes = buffer->length, res = buffer->error, n, offset = 0; + void *pageaddr; + + if (res) { + ERROR("Unable to read page, block %llx, size %x\n", block, + bsize); + goto out; have you forgotten to unlock the pages on error path? in case of error squashfs_readpage will unlock only target page. Yup, a fix for -rc1 is on its way. This error path failed to trigger in my stress tests because it is pretty hard to hit, because it relies on decompress failure at the same time as we've raced against another process also grabbing the pages. Phillip + } + + for (n = 0; n < pages && bytes > 0; n++, + bytes -= PAGE_CACHE_SIZE, offset += PAGE_CACHE_SIZE) { + int avail = min_t(int, bytes, PAGE_CACHE_SIZE); -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/7] Squashfs: Directly decompress into the page cache for file data
On 22/11/13 07:40, Roman Peniaev wrote: Hello, Phillip. one remark below: +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page) +{ + struct inode *i = target_page-mapping-host; + struct squashfs_cache_entry *buffer = squashfs_get_datablock(i- i_sb, +block, bsize); + int bytes = buffer-length, res = buffer-error, n, offset = 0; + void *pageaddr; + + if (res) { + ERROR(Unable to read page, block %llx, size %x\n, block, + bsize); + goto out; have you forgotten to unlock the pages on error path? in case of error squashfs_readpage will unlock only target page. Yup, a fix for -rc1 is on its way. This error path failed to trigger in my stress tests because it is pretty hard to hit, because it relies on decompress failure at the same time as we've raced against another process also grabbing the pages. Phillip + } + + for (n = 0; n pages bytes 0; n++, + bytes -= PAGE_CACHE_SIZE, offset += PAGE_CACHE_SIZE) { + int avail = min_t(int, bytes, PAGE_CACHE_SIZE); -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/7] Squashfs: Directly decompress into the page cache for file data
Hello, Phillip. one remark below: > > +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, > + int pages, struct page **page) > +{ > + struct inode *i = target_page->mapping->host; > + struct squashfs_cache_entry *buffer = squashfs_get_datablock(i- >i_sb, > + block, bsize); > + int bytes = buffer->length, res = buffer->error, n, offset = 0; > + void *pageaddr; > + > + if (res) { > + ERROR("Unable to read page, block %llx, size %x\n", block, > + bsize); > + goto out; have you forgotten to unlock the pages on error path? in case of error squashfs_readpage will unlock only target page. > + } > + > + for (n = 0; n < pages && bytes > 0; n++, > + bytes -= PAGE_CACHE_SIZE, offset += PAGE_CACHE_SIZE) { > + int avail = min_t(int, bytes, PAGE_CACHE_SIZE); -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/7] Squashfs: Directly decompress into the page cache for file data
Hello, Phillip. one remark below: +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page) +{ + struct inode *i = target_page-mapping-host; + struct squashfs_cache_entry *buffer = squashfs_get_datablock(i- i_sb, + block, bsize); + int bytes = buffer-length, res = buffer-error, n, offset = 0; + void *pageaddr; + + if (res) { + ERROR(Unable to read page, block %llx, size %x\n, block, + bsize); + goto out; have you forgotten to unlock the pages on error path? in case of error squashfs_readpage will unlock only target page. + } + + for (n = 0; n pages bytes 0; n++, + bytes -= PAGE_CACHE_SIZE, offset += PAGE_CACHE_SIZE) { + int avail = min_t(int, bytes, PAGE_CACHE_SIZE); -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 6/7] Squashfs: Directly decompress into the page cache for file data
This introduces an implementation of squashfs_readpage_block() that directly decompresses into the page cache. This uses the previously added page handler abstraction to push down the necessary kmap_atomic/kunmap_atomic operations on the page cache buffers into the decompressors. This enables direct copying into the page cache without using the slow kmap/kunmap calls. The code detects when multiple threads are racing in squashfs_readpage() to decompress the same block, and avoids this regression by falling back to using an intermediate buffer. This patch enhances the performance of Squashfs significantly when multiple processes are accessing the filesystem simultaneously because it not only reduces memcopying, but it more importantly eliminates the lock contention on the intermediate buffer. Using single-thread decompression. dd if=file1 of=/dev/null bs=4096 & dd if=file2 of=/dev/null bs=4096 & dd if=file3 of=/dev/null bs=4096 & dd if=file4 of=/dev/null bs=4096 Before: 629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s After: 629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s Signed-off-by: Phillip Lougher Reviewed-by: Minchan Kim --- fs/squashfs/Kconfig | 28 fs/squashfs/Makefile | 4 +- fs/squashfs/file_direct.c | 173 ++ fs/squashfs/page_actor.c | 100 +++ fs/squashfs/page_actor.h | 32 + 5 files changed, 336 insertions(+), 1 deletion(-) create mode 100644 fs/squashfs/file_direct.c create mode 100644 fs/squashfs/page_actor.c diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig index 159bd66..b6fa865 100644 --- a/fs/squashfs/Kconfig +++ b/fs/squashfs/Kconfig @@ -26,6 +26,34 @@ config SQUASHFS If unsure, say N. choice + prompt "File decompression options" + depends on SQUASHFS + help + Squashfs now supports two options for decompressing file + data. Traditionally Squashfs has decompressed into an + intermediate buffer and then memcopied it into the page cache. + Squashfs now supports the ability to decompress directly into + the page cache. + + If unsure, select "Decompress file data into an intermediate buffer" + +config SQUASHFS_FILE_CACHE + bool "Decompress file data into an intermediate buffer" + help + Decompress file data into an intermediate buffer and then + memcopy it into the page cache. + +config SQUASHFS_FILE_DIRECT + bool "Decompress files directly into the page cache" + help + Directly decompress file data into the page cache. + Doing so can significantly improve performance because + it eliminates a memcpy and it also removes the lock contention + on the single buffer. + +endchoice + +choice prompt "Decompressor parallelisation options" depends on SQUASHFS help diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile index e01ba11..4132520 100644 --- a/fs/squashfs/Makefile +++ b/fs/squashfs/Makefile @@ -4,7 +4,9 @@ obj-$(CONFIG_SQUASHFS) += squashfs.o squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o -squashfs-y += namei.o super.o symlink.o decompressor.o file_cache.o +squashfs-y += namei.o super.o symlink.o decompressor.o +squashfs-$(CONFIG_SQUASHFS_FILE_CACHE) += file_cache.o +squashfs-$(CONFIG_SQUASHFS_FILE_DIRECT) += file_direct.o page_actor.o squashfs-$(CONFIG_SQUASHFS_DECOMP_SINGLE) += decompressor_single.o squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI) += decompressor_multi.o squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU) += decompressor_multi_percpu.o diff --git a/fs/squashfs/file_direct.c b/fs/squashfs/file_direct.c new file mode 100644 index 000..2943b2b --- /dev/null +++ b/fs/squashfs/file_direct.c @@ -0,0 +1,173 @@ +/* + * Copyright (c) 2013 + * Phillip Lougher + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "squashfs_fs.h" +#include "squashfs_fs_sb.h" +#include "squashfs_fs_i.h" +#include "squashfs.h" +#include "page_actor.h" + +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page); + +/* Read separately compressed datablock directly into page cache */ +int squashfs_readpage_block(struct page *target_page, u64 block, int bsize) + +{ + struct inode *inode = target_page->mapping->host; + struct squashfs_sb_info *msblk = inode->i_sb->s_fs_info; + + int file_end = (i_size_read(inode) - 1) >> PAGE_CACHE_SHIFT; + int mask = (1 << (msblk->block_log - PAGE_CACHE_SHIFT)) - 1; + int start_index = target_page->index & ~mask; + int end_index = start_index | mask; + int i, n, pages, missing_pages, bytes, res = -ENOMEM; + struct page **page; +
[PATCH 6/7] Squashfs: Directly decompress into the page cache for file data
This introduces an implementation of squashfs_readpage_block() that directly decompresses into the page cache. This uses the previously added page handler abstraction to push down the necessary kmap_atomic/kunmap_atomic operations on the page cache buffers into the decompressors. This enables direct copying into the page cache without using the slow kmap/kunmap calls. The code detects when multiple threads are racing in squashfs_readpage() to decompress the same block, and avoids this regression by falling back to using an intermediate buffer. This patch enhances the performance of Squashfs significantly when multiple processes are accessing the filesystem simultaneously because it not only reduces memcopying, but it more importantly eliminates the lock contention on the intermediate buffer. Using single-thread decompression. dd if=file1 of=/dev/null bs=4096 dd if=file2 of=/dev/null bs=4096 dd if=file3 of=/dev/null bs=4096 dd if=file4 of=/dev/null bs=4096 Before: 629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s After: 629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s Signed-off-by: Phillip Lougher phil...@squashfs.org.uk Reviewed-by: Minchan Kim minc...@kernel.org --- fs/squashfs/Kconfig | 28 fs/squashfs/Makefile | 4 +- fs/squashfs/file_direct.c | 173 ++ fs/squashfs/page_actor.c | 100 +++ fs/squashfs/page_actor.h | 32 + 5 files changed, 336 insertions(+), 1 deletion(-) create mode 100644 fs/squashfs/file_direct.c create mode 100644 fs/squashfs/page_actor.c diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig index 159bd66..b6fa865 100644 --- a/fs/squashfs/Kconfig +++ b/fs/squashfs/Kconfig @@ -26,6 +26,34 @@ config SQUASHFS If unsure, say N. choice + prompt File decompression options + depends on SQUASHFS + help + Squashfs now supports two options for decompressing file + data. Traditionally Squashfs has decompressed into an + intermediate buffer and then memcopied it into the page cache. + Squashfs now supports the ability to decompress directly into + the page cache. + + If unsure, select Decompress file data into an intermediate buffer + +config SQUASHFS_FILE_CACHE + bool Decompress file data into an intermediate buffer + help + Decompress file data into an intermediate buffer and then + memcopy it into the page cache. + +config SQUASHFS_FILE_DIRECT + bool Decompress files directly into the page cache + help + Directly decompress file data into the page cache. + Doing so can significantly improve performance because + it eliminates a memcpy and it also removes the lock contention + on the single buffer. + +endchoice + +choice prompt Decompressor parallelisation options depends on SQUASHFS help diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile index e01ba11..4132520 100644 --- a/fs/squashfs/Makefile +++ b/fs/squashfs/Makefile @@ -4,7 +4,9 @@ obj-$(CONFIG_SQUASHFS) += squashfs.o squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o -squashfs-y += namei.o super.o symlink.o decompressor.o file_cache.o +squashfs-y += namei.o super.o symlink.o decompressor.o +squashfs-$(CONFIG_SQUASHFS_FILE_CACHE) += file_cache.o +squashfs-$(CONFIG_SQUASHFS_FILE_DIRECT) += file_direct.o page_actor.o squashfs-$(CONFIG_SQUASHFS_DECOMP_SINGLE) += decompressor_single.o squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI) += decompressor_multi.o squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU) += decompressor_multi_percpu.o diff --git a/fs/squashfs/file_direct.c b/fs/squashfs/file_direct.c new file mode 100644 index 000..2943b2b --- /dev/null +++ b/fs/squashfs/file_direct.c @@ -0,0 +1,173 @@ +/* + * Copyright (c) 2013 + * Phillip Lougher phil...@squashfs.org.uk + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + */ + +#include linux/fs.h +#include linux/vfs.h +#include linux/kernel.h +#include linux/slab.h +#include linux/string.h +#include linux/pagemap.h +#include linux/mutex.h + +#include squashfs_fs.h +#include squashfs_fs_sb.h +#include squashfs_fs_i.h +#include squashfs.h +#include page_actor.h + +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page); + +/* Read separately compressed datablock directly into page cache */ +int squashfs_readpage_block(struct page *target_page, u64 block, int bsize) + +{ + struct inode *inode = target_page-mapping-host; + struct squashfs_sb_info *msblk = inode-i_sb-s_fs_info; + + int file_end = (i_size_read(inode) - 1) PAGE_CACHE_SHIFT; + int mask = (1 (msblk-block_log - PAGE_CACHE_SHIFT)) - 1; + int start_index = target_page-index ~mask; + int end_index