[PREVIEW] [RFC PATCH 5/5] erofs-mkfs: add miscellaneous files

2018-11-15 Thread bluce.liguifu
From: Li Guifu Add build environment, license and maintainers etc. Signed-off-by: Li Guifu Signed-off-by: Miao Xie Signed-off-by: Fang Wei --- .gitignore | 44 AUTHORS | 6 +++ COPYING | 113 +++ ChangeLog

[PREVIEW] [RFC PATCH 4/5] erofs-mkfs: add compression support

2018-11-15 Thread bluce.liguifu
From: Li Guifu This patch introduces a compression framework, so it can support different compression algorithm in principle, currently lz4hc is added right now due to the kernel implementation. Signed-off-by: Li Guifu Signed-off-by: Miao Xie Signed-off-by: Fang Wei --- erofs_compressor.c |

[PREVIEW] [RFC PATCH 2/5] erofs-mkfs: introduce mkfs basic framework

2018-11-15 Thread bluce.liguifu
From: Li Guifu This patch adds basic code to generate erofs file systems and empty images can be created properly. Some parameters are also introduced in this patch, used for setting the debugging level, whether to compress or not. Signed-off-by: Li Guifu Signed-off-by: Miao Xie

[PREVIEW] [RFC PATCH 3/5] erofs-mkfs: support to build a image from the specfic root directory

2018-11-15 Thread bluce.liguifu
From: Li Guifu Currently, regular files, directories, special files defined in POSIX are supported and the generated image can be mounted by kernel. Signed-off-by: Li Guifu Signed-off-by: Miao Xie Signed-off-by: Fang Wei --- erofs_cache.c | 211 ++ erofs_cache.h | 58

[PREVIEW] [RFC PATCH 1/5] erofs-mkfs: add erofs on-disk layout

2018-11-15 Thread bluce.liguifu
From: Li Guifu This patch adds a header file describing the erofs on-disk layout, which should be kept in line with the kernel implementation all the time. Signed-off-by: Li Guifu Signed-off-by: Miao Xie Signed-off-by: Fang Wei --- erofs_fs.h | 298

[PREVIEW] [RFC PATCH 0/5] erofs-mkfs: mkfs.erofs source code first preview

2018-11-15 Thread bluce.liguifu
From: Li Guifu This patchset introduces the source code for mkfs.erofs. It can build erofs image from scratch according to a specific directory. Currently, it can build lz4 compressed images and uncompressed images. Regular files, directories and special files are supported. However it can only

Re: [PATCH 3/3] staging: erofs: unzip_vle.c: Align parameter to the parentesis

2018-11-15 Thread Chao Yu
On 2018/11/13 4:43, Cristian Sicilia wrote: > Align parameters to the opened parentesis. > > Signed-off-by: Cristian Sicilia Reviewed-by: Chao Yu Thanks,

Re: [PATCH 2/3] staging: erofs: unzip_vle.c: Constant in comparison on right side

2018-11-15 Thread Chao Yu
On 2018/11/13 4:43, Cristian Sicilia wrote: > Comparisons should place the constant > on the right side of the test. > > Signed-off-by: Cristian Sicilia Reviewed-by: Chao Yu Thanks,

Re: [PATCH 1/3] staging: erofs: unzip_vle.c: Replace comparison to NULL.

2018-11-15 Thread Chao Yu
On 2018/11/13 4:43, Cristian Sicilia wrote: > Replace equal to NULL with logic unary operator, > and removing not equal to NULL comparison. > > Signed-off-by: Cristian Sicilia Reviewed-by: Chao Yu Thanks,

Re: [PATCH chao/erofs-dev rebased v2 01/12] staging: erofs: unzip_vle.c: Replace comparison to NULL.

2018-11-15 Thread Chao Yu
Hi Xiang, On 2018/11/16 10:55, Gao Xiang wrote: > Hi Chao, > > On 2018/11/16 10:53, Chao Yu wrote: >> On 2018/11/14 23:25, Gao Xiang wrote: >>> From: Cristian Sicilia >>> >>> Replace equal to NULL with logic unary operator, >>> and removing not equal to NULL comparison. >>> >>> Signed-off-by:

Re: [PATCH chao/erofs-dev rebased v2 04/12] staging: erofs: fix `trace_erofs_readpage' position

2018-11-15 Thread Chao Yu
On 2018/11/14 23:25, Gao Xiang wrote: > `trace_erofs_readpage' should be placed in .readpage() > rather than in the internal `z_erofs_do_read_page'. > > Fixes: 284db12cfda3 ("staging: erofs: add trace points for reading zipped > data") > Signed-off-by: Gao Xiang > Reviewed-by: Chen Gong

Re: [PATCH chao/erofs-dev rebased v2 01/12] staging: erofs: unzip_vle.c: Replace comparison to NULL.

2018-11-15 Thread Gao Xiang
Hi Chao, On 2018/11/16 10:53, Chao Yu wrote: > On 2018/11/14 23:25, Gao Xiang wrote: >> From: Cristian Sicilia >> >> Replace equal to NULL with logic unary operator, >> and removing not equal to NULL comparison. >> >> Signed-off-by: Cristian Sicilia > > Reviewed-by: Chao Yu > This patch is

Re: [PATCH chao/erofs-dev rebased v2 02/12] staging: erofs: unzip_vle.c: Constant in comparison on right side

2018-11-15 Thread Chao Yu
On 2018/11/14 23:25, Gao Xiang wrote: > From: Cristian Sicilia > > Comparisons should place the constant > on the right side of the test. > > Signed-off-by: Cristian Sicilia Reviewed-by: Chao Yu Thanks,

Re: [PATCH chao/erofs-dev rebased v2 03/12] staging: erofs: unzip_vle.c: Align parameter to the parentesis

2018-11-15 Thread Chao Yu
On 2018/11/14 23:25, Gao Xiang wrote: > From: Cristian Sicilia > > Align parameters to the opened parentesis. > > Signed-off-by: Cristian Sicilia Reviewed-by: Chao Yu Thanks,

Re: [PATCH chao/erofs-dev rebased v2 01/12] staging: erofs: unzip_vle.c: Replace comparison to NULL.

2018-11-15 Thread Chao Yu
On 2018/11/14 23:25, Gao Xiang wrote: > From: Cristian Sicilia > > Replace equal to NULL with logic unary operator, > and removing not equal to NULL comparison. > > Signed-off-by: Cristian Sicilia Reviewed-by: Chao Yu Thanks,

Re: [PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:05PM +0800, Ming Lei wrote: > Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), > physical segment number is mainly figured out in blk_queue_split() for > fast path, and the flag of BIO_SEG_VALID is set there too. > > Now only

Re: [PATCH V10 17/19] block: don't use bio->bi_vcnt to figure out segment number

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:04PM +0800, Ming Lei wrote: > It is wrong to use bio->bi_vcnt to figure out how many segments > there are in the bio even though CLONED flag isn't set on this bio, > because this bio may be splitted or advanced. > > So always use bio_segments() in

Re: [PATCH V10 16/19] block: document usage of bio iterator helpers

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:03PM +0800, Ming Lei wrote: > Now multi-page bvec is supported, some helpers may return page by > page, meantime some may return segment by segment, this patch > documents the usage. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc:

Re: [PATCH V10 14/19] block: enable multipage bvecs

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:01PM +0800, Ming Lei wrote: > This patch pulls the trigger for multi-page bvecs. > > Now any request queue which supports queue cluster will see multi-page > bvecs. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc:

Re: [PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:02PM +0800, Ming Lei wrote: > Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to > increase BIO_MAX_PAGES for it. You mentioned to it in the cover letter, but this needs more explanation in the commit message. Why did CONFIG_THP_SWAP require > 256?

Re: [PATCH V10 13/19] iomap & xfs: only account for new added page

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:53:00PM +0800, Ming Lei wrote: > After multi-page is enabled, one new page may be merged to a segment > even though it is a new added page. > > This patch deals with this issue by post-check in case of merge, and > only a freshly new added page need to be dealt with for

Re: [PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:56PM +0800, Ming Lei wrote: > There are still cases in which we need to use bio_bvecs() for get the > number of multi-page segment, so introduce it. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexander Viro > Cc:

Re: [PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:52PM +0800, Ming Lei wrote: > BTRFS and guard_bio_eod() need to get the last singlepage segment > from one multipage bvec, so introduce this helper to make them happy. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc:

Re: [PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:57PM +0800, Ming Lei wrote: > iov_iter is implemented with bvec itererator, so it is safe to pass > multipage bvec to it, and this way is much more efficient than > passing one page in each bvec. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc:

Re: [PATCH V10 06/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:53PM +0800, Ming Lei wrote: > Once multi-page bvec is enabled, the last bvec may include more than one > page, this patch use bvec_last_segment() to truncate the bio. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc:

Re: [PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:54PM +0800, Ming Lei wrote: > Preparing for supporting multi-page bvec. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexander Viro > Cc: linux-fsde...@vger.kernel.org > Cc: Shaohua Li > Cc:

Re: [PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:58PM +0800, Ming Lei wrote: > bch_bio_alloc_pages() is always called on one new bio, so it is safe > to access the bvec table directly. Given it is the only kind of this > case, open code the bvec table access since bio_for_each_segment_all() > will be changed to

Re: [PATCH V10 08/19] btrfs: move bio_pages_all() to btrfs

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:55PM +0800, Ming Lei wrote: > BTRFS is the only user of this helper, so move this helper into > BTRFS, and implement it via bio_for_each_segment_all(), since > bio->bi_vcnt may not equal to number of pages after multipage bvec > is enabled. Shouldn't you also get rid

Re: [PATCH V10 01/19] block: introduce multi-page page bvec helpers

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:48PM +0800, Ming Lei wrote: > This patch introduces helpers of 'mp_bvec_iter_*' for multipage > bvec support. > > The introduced helpers treate one bvec as real multi-page segment, > which may include more than one pages. > > The existed helpers of bvec_iter_* are

Re: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Mike Snitzer
On Thu, Nov 15 2018 at 3:20pm -0500, Omar Sandoval wrote: > On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > > First it is more efficient to use bio_for_each_bvec() in both > > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > > many multi-page bvecs there are

Re: [PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:51PM +0800, Ming Lei wrote: > It is more efficient to use bio_for_each_bvec() to map sg, meantime > we have to consider splitting multipage bvec as done in > blk_bio_segment_split(). > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc:

Re: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:50PM +0800, Ming Lei wrote: > First it is more efficient to use bio_for_each_bvec() in both > blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how > many multi-page bvecs there are in the bio. > > Secondly once bio_for_each_bvec() is used, the bvec

Re: [PATCH V10 02/19] block: introduce bio_for_each_bvec()

2018-11-15 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 04:52:49PM +0800, Ming Lei wrote: > This helper is used for iterating over multi-page bvec for bio > split & merge code. > > Cc: Dave Chinner > Cc: Kent Overstreet > Cc: Mike Snitzer > Cc: dm-de...@redhat.com > Cc: Alexander Viro > Cc: linux-fsde...@vger.kernel.org >

[RFC PATCH chao/erofs-dev rebased NOTEST 1/3] staging: erofs: rename strange variable names in z_erofs_vle_frontend

2018-11-15 Thread Gao Xiang
Previously, 2 members called `initial' and `cachedzone_la' are used for applying caching policy (whether the workgroup is at either end), which are hard to understand, rename them to `backmost' and `headoffset'. Signed-off-by: Gao Xiang --- drivers/staging/erofs/unzip_vle.c | 25

[RFC PATCH chao/erofs-dev rebased NOTEST 3/3] staging: erofs: fix compressed pages submission flow

2018-11-15 Thread Gao Xiang
This patch fully closes race between page reclaiming and compressed pages submitting, which could cause very low probability of reference leak and double free. Signed-off-by: Gao Xiang --- drivers/staging/erofs/unzip_vle.c | 331 ++

[RFC PATCH chao/erofs-dev rebased NOTEST 2/3] staging: erofs: introduce MNGD_MAPPING helper

2018-11-15 Thread Gao Xiang
This patch introduces MNGD_MAPPING to wrap up sbi->managed_cache->i_mapping. Signed-off-by: Gao Xiang --- drivers/staging/erofs/internal.h | 4 drivers/staging/erofs/unzip_vle.c | 29 + 2 files changed, 17 insertions(+), 16 deletions(-) diff --git

[PATCH V10 19/19] block: kill BLK_MQ_F_SG_MERGE

2018-11-15 Thread Ming Lei
QUEUE_FLAG_NO_SG_MERGE has been killed, so kill BLK_MQ_F_SG_MERGE too. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org Cc: David

[PATCH V10 18/19] block: kill QUEUE_FLAG_NO_SG_MERGE

2018-11-15 Thread Ming Lei
Since bdced438acd83ad83a6c ("block: setup bi_phys_segments after splitting"), physical segment number is mainly figured out in blk_queue_split() for fast path, and the flag of BIO_SEG_VALID is set there too. Now only blk_recount_segments() and blk_recalc_rq_segments() use this flag. Basically

[PATCH V10 16/19] block: document usage of bio iterator helpers

2018-11-15 Thread Ming Lei
Now multi-page bvec is supported, some helpers may return page by page, meantime some may return segment by segment, this patch documents the usage. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua

[PATCH V10 15/19] block: always define BIO_MAX_PAGES as 256

2018-11-15 Thread Ming Lei
Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to increase BIO_MAX_PAGES for it. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc:

[PATCH V10 11/19] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages()

2018-11-15 Thread Ming Lei
bch_bio_alloc_pages() is always called on one new bio, so it is safe to access the bvec table directly. Given it is the only kind of this case, open code the bvec table access since bio_for_each_segment_all() will be changed to support for iterating over multipage bvec. Cc: Dave Chinner Cc: Kent

[PATCH V10 10/19] block: loop: pass multi-page bvec to iov_iter

2018-11-15 Thread Ming Lei
iov_iter is implemented with bvec itererator, so it is safe to pass multipage bvec to it, and this way is much more efficient than passing one page in each bvec. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org

[PATCH V10 09/19] block: introduce bio_bvecs()

2018-11-15 Thread Ming Lei
There are still cases in which we need to use bio_bvecs() for get the number of multi-page segment, so introduce it. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc:

[PATCH V10 06/19] fs/buffer.c: use bvec iterator to truncate the bio

2018-11-15 Thread Ming Lei
Once multi-page bvec is enabled, the last bvec may include more than one page, this patch use bvec_last_segment() to truncate the bio. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc:

[PATCH V10 05/19] block: introduce bvec_last_segment()

2018-11-15 Thread Ming Lei
BTRFS and guard_bio_eod() need to get the last singlepage segment from one multipage bvec, so introduce this helper to make them happy. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc:

[PATCH V10 07/19] btrfs: use bvec_last_segment to get bio's last page

2018-11-15 Thread Ming Lei
Preparing for supporting multi-page bvec. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org Cc: David Sterba Cc:

[PATCH V10 02/19] block: introduce bio_for_each_bvec()

2018-11-15 Thread Ming Lei
This helper is used for iterating over multi-page bvec for bio split & merge code. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua Li Cc: linux-r...@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org

[PATCH V10 00/19] block: support multi-page bvec

2018-11-15 Thread Ming Lei
Hi, This patchset brings multi-page bvec into block layer: 1) what is multi-page bvec? Multipage bvecs means that one 'struct bio_bvec' can hold multiple pages which are physically contiguous instead of one single page used in linux kernel for long time. 2) why is multi-page bvec introduced?

[PATCH V10 04/19] block: use bio_for_each_bvec() to map sg

2018-11-15 Thread Ming Lei
It is more efficient to use bio_for_each_bvec() to map sg, meantime we have to consider splitting multipage bvec as done in blk_bio_segment_split(). Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-de...@redhat.com Cc: Alexander Viro Cc: linux-fsde...@vger.kernel.org Cc: Shaohua