Re: [f2fs-dev] [PATCH v3 00/13] fiemap extension for more physical information

2024-04-03 Thread Gao Xiang
ust fiemap report for iomap seems straight-forward. Thanks for your work! Thanks, Gao Xiang Sweet Tea ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH v3 00/13] fiemap extension for more physical information

2024-04-03 Thread Gao Xiang
physical extent information? since compressed EROFS uses iomap FIEMAP interface to report compressed extents ("z_erofs_iomap_report_ops") but there is no way to return correct compressed lengths, that is unexpected. Thanks, Gao Xiang ___ Linux-

Re: [f2fs-dev] Weird EROFS data corruption

2023-12-05 Thread Gao Xiang
Hi Juhyung, On 2023/12/5 22:43, Juhyung Park wrote: On Tue, Dec 5, 2023 at 11:34 PM Gao Xiang wrote: ... I'm still analyzing this behavior as well as the root cause and I will also try to get a recent cloud server with FSRM myself to find more clues. Down the rabbit hole we go... Let

Re: [f2fs-dev] Weird EROFS data corruption

2023-12-05 Thread Gao Xiang
On 2023/12/5 22:23, Juhyung Park wrote: Hi Gao, On Tue, Dec 5, 2023 at 4:32 PM Gao Xiang wrote: Hi Juhyung, On 2023/12/4 11:41, Juhyung Park wrote: ... - Could you share the full message about the output of `lscpu`? Sure: Architecture:x86_64 CPU op-mode(s

Re: [f2fs-dev] Weird EROFS data corruption

2023-12-04 Thread Gao Xiang
startup latency Thanks, Gao Xiang ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] Weird EROFS data corruption

2023-12-03 Thread Gao Xiang
t of `lscpu`? Thanks, Gao Xiang ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] Weird EROFS data corruption

2023-12-03 Thread Gao Xiang
On 2023/12/4 01:01, Juhyung Park wrote: Hi Gao, On Mon, Dec 4, 2023 at 1:52 AM Gao Xiang wrote: Hi Juhyung, On 2023/12/4 00:22, Juhyung Park wrote: (Cc'ing f2fs and crypto as I've noticed something similar with f2fs a while ago, which may mean that this is not specific to EROFS: https

Re: [f2fs-dev] Weird EROFS data corruption

2023-12-03 Thread Gao Xiang
nce like different compliers? Thanks, Gao Xiang ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 2/2] f2fs: handle decompress only post processing in softirq

2022-06-20 Thread Gao Xiang
you'd like to handle all: - decompression algorithms; - verity algorithms; - decrypt algorithms; in this way, regardless of slow decompression algorithms, that would be a long latency and CPU overhead of softirq context. This is my last words on this, I will not talk any

Re: [f2fs-dev] [PATCH] f2fs: handle decompress only post processing in softirq

2022-06-14 Thread Gao Xiang
todate(page) > > > >some callback to decompress in addition to kworker > > > >lock_page() > >->read_folio() > > > > If mm folks don't like it, I think RT thread is also fine after we > > analysed the root cause of the kworker delay I think. > &g

Re: [f2fs-dev] [PATCH] f2fs: handle decompress only post processing in softirq

2022-06-14 Thread Gao Xiang
ession before so the priority becomes low, but that is > >just a pure guess. May be we need to use RT scheduling policy instead. > > > >At least with WQ_HIGHPRI for dm-verity at least, but I don't find > >WQ_HIGHPRI mark for dm-verity. > > > > Thanks, &

Re: [f2fs-dev] [PATCH] f2fs: handle decompress only post processing in softirq

2022-06-14 Thread Gao Xiang
punishment due to a lot of CPU consuming due to decompression before so the priority becomes low, but that is just a pure guess. May be we need to use RT scheduling policy instead. At least with WQ_HIGHPRI for dm-verity at least, but I don't find WQ_HIGHPRI mark for dm-verity. Thank

[f2fs-dev] [PATCH] f2fs: fix up f2fs_lookup tracepoints

2021-09-21 Thread Gao Xiang
Fix up a misuse that the filename pointer isn't always valid in the ring buffer, and we should copy the content instead. Fixes: 0c5e36db17f5 ("f2fs: trace f2fs_lookup") Signed-off-by: Gao Xiang --- include/trace/events/f2fs.h | 12 ++-- 1 file changed, 6 insertions(+), 6

Re: [f2fs-dev] [PATCH] f2fs: replace ERANGE with ENAMETOOLONG in file name length check

2021-06-14 Thread Gao Xiang
On Tue, Jun 15, 2021 at 11:19:24AM +0800, wangxiaojun (N) wrote: > > 在 2021/6/15 10:31, Gao Xiang 写道: > > On Tue, Jun 15, 2021 at 09:35:09AM +0800, Wang Xiaojun wrote: > > > ERANGE indicates that the math result is not representative. Here, > > > ENAMETOO

Re: [f2fs-dev] [PATCH] f2fs: replace ERANGE with ENAMETOOLONG in file name length check

2021-06-14 Thread Gao Xiang
n7.org/linux/man-pages/man2/getxattr.2.html https://man7.org/linux/man-pages/man2/setxattr.2.html instead of ERANGE. please also see ext4 / xfs implementations. Thanks, Gao Xiang > --- > fs/f2fs/xattr.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git

Re: [f2fs-dev] 答复: [PATCH v4] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-11 Thread Gao Xiang
g, it smelled somewhat dangerous to me anyway. But considering some performance impact, I just leave some message here. Thanks, Gao Xiang > > > > > Thanks. > > > > -邮件原件- > > 发件人: Jaegeuk Kim > > 发送时间: 2021年5月10日 23:44 > > 收件人: Fengnan Ch

Re: [f2fs-dev] 答复: 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-06 Thread Gao Xiang via Linux-f2fs-devel
comments about this sounds better. Thanks, Gao Xiang > > > -邮件原件- > 发件人: Chao Yu > 发送时间: 2021年5月6日 18:38 > 收件人: Gao Xiang > 抄送: Gao Xiang ; changfeng...@vivo.com; > jaeg...@kernel.org; linux-f2fs-devel@lists.sourceforge.net > 主题: Re: [f2fs-dev] 答复: [PATC

Re: [f2fs-dev] 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-06 Thread Gao Xiang via Linux-f2fs-devel
On Thu, May 06, 2021 at 06:37:45PM +0800, Chao Yu wrote: > Hi Xiang, > > On 2021/5/6 17:58, Gao Xiang wrote: > > Hi Chao, > > > > On Thu, May 06, 2021 at 05:15:04PM +0800, Chao Yu wrote: > > > On 2021/4/26 17:00, Gao Xiang wrote: > > > > On

Re: [f2fs-dev] 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-06 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao, On Thu, May 06, 2021 at 05:15:04PM +0800, Chao Yu wrote: > On 2021/4/26 17:00, Gao Xiang wrote: > > On Mon, Apr 26, 2021 at 04:42:20PM +0800, changfeng...@vivo.com wrote: > > > Thank you for the reminder, I hadn't thought about fallocate before. > >

Re: [f2fs-dev] 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-04-26 Thread Gao Xiang
nges (e.g. supporting FALLOC_FL_ZERO_RANGE with keep size later), it may cause some potential regression. > > -邮件原件- > 发件人: Gao Xiang > 发送时间: 2021年4月26日 11:19 > 收件人: Fengnan Chang > 抄送: c...@kernel.org; jaeg...@kernel.org; > linux-f2fs-devel@lists.sourceforge.net > 主题:

Re: [f2fs-dev] [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-04-25 Thread Gao Xiang
On Mon, Apr 26, 2021 at 10:11:53AM +0800, Fengnan Chang wrote: > when write compressed file with O_TRUNC, there will be a lot of > unnecessary check valid blocks in f2fs_prepare_compress_overwrite, > especially when written in page size, remove it. > > Signed-off-by: Fengnan Chang Even though I

Re: [f2fs-dev] [PATCH] f2fs: introduce gc_merge mount option

2021-03-27 Thread Gao Xiang
, > it can eliminate the sluggish issue caused by slow foreground GC > operation when GC is triggered from a process with limited I/O > and CPU resources. > > Original idea is from Xiang. > > Signed-off-by: Gao Xiang > Signed-off-by: Chao Yu Ah, that was a quite old comm

Re: [f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-04 Thread Gao Xiang
10362 MB/s 10790 MB/s 100.00 lz4 1.9.2 737 MB/s4448 MB/s 47.60 lz4hc 1.9.2 -9 33 MB/s 4378 MB/s 36.75 So adding more IO time (due to storage device difference) could make CPU loading lower (also could make the whole process IO bound) but the overall

Re: [f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-03 Thread Gao Xiang
Hi Chao, On Fri, Dec 04, 2020 at 03:09:20PM +0800, Chao Yu wrote: > On 2020/12/4 8:31, Gao Xiang wrote: > > could make more sense), could you leave some CR numbers about these > > algorithms on typical datasets (enwik9, silisia.tar or else.) with 16k > > cluster size? >

Re: [f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-03 Thread Gao Xiang
On Fri, Dec 04, 2020 at 11:11:03AM +0800, Chao Yu wrote: > On 2020/12/4 10:47, Gao Xiang wrote: ... > > > future (and add more dependency to algorithms, you might see BWT-based bzip2 > > removal patch > > Oops, is that really allowed? I don't this is a goo

Re: [f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-03 Thread Gao Xiang
On Fri, Dec 04, 2020 at 10:38:08AM +0800, Chao Yu wrote: > On 2020/12/4 10:06, Gao Xiang wrote: > > On Fri, Dec 04, 2020 at 09:56:27AM +0800, Chao Yu wrote: ... > > > > > Keep lz4hc dirty data under writeback could block writeback, make kswapd > > busy, and direc

Re: [f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-03 Thread Gao Xiang
On Fri, Dec 04, 2020 at 09:56:27AM +0800, Chao Yu wrote: > Hi Xiang, > > On 2020/12/4 8:31, Gao Xiang wrote: > > Hi Chao, > > > > On Thu, Dec 03, 2020 at 11:32:34AM -0800, Eric Biggers wrote: > > > > ... > > > > > > > >

Re: [f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-03 Thread Gao Xiang
lzo-rle, initially it was only used for in-memory anonymous pages and it won't be kept on-disk so that's fine. I'm not sure if lzo original author want to support it or not. It'd be better to get some opinion before keeping it on-disk. Thanks, Gao Xiang > - Eric > > > __

Re: [f2fs-dev] [PATCH v4 2/3] fscrypt: Have filesystems handle their d_ops

2020-11-23 Thread Gao Xiang
On Mon, Nov 23, 2020 at 02:51:44PM -0800, Eric Biggers wrote: > On Sun, Nov 22, 2020 at 01:12:18PM +0800, Gao Xiang wrote: > > Hi all, > > > > On Thu, Nov 19, 2020 at 06:09:03AM +, Daniel Rosenberg wrote: > > > This shifts the responsibility of setting up dentry

Re: [f2fs-dev] [PATCH v4 2/3] fscrypt: Have filesystems handle their d_ops

2020-11-21 Thread Gao Xiang
ird() For more details, see: https://android-review.googlesource.com/c/device/linaro/hikey/+/1483316/2#message-2e1f6ab0010a3e35e7d8effea73f60341f84ee4d Just found it by chance (and not sure if it's vital for now), and a kind reminder about this. Thanks, Gao Xiang ___

Re: [f2fs-dev] [PATCH v3] f2fs: change virtual mapping way for compression pages

2020-08-12 Thread Gao Xiang
/s > 1048576000 bytes (0.9 G) copied, 1.986380 s, 503 M/s > The reason why I raised up this was that I once observed vmap() vs vm_map_ram() on arm64 kirin platform as well. it indeed had some impact (~10%) but not as huge as this. Anyway, such description with test environment looks ok. Thanks, Gao Xi

Re: [f2fs-dev] [PATCH] f2fs: change virtual mapping way for compression pages

2020-08-11 Thread Gao Xiang
message is a bit of unclear. At least, if you think that is really the case, I'm ok with that. Thanks, Gao Xiang > > Thanks, > > 2020년 8월 11� (화) 오후 7:18, Gao Xiang 님� > 작성: > > > > On Tue, Aug 11, 2020 at 06:33:26PM +0900, Daeho Jeong wrote: >

Re: [f2fs-dev] [PATCH] f2fs: change virtual mapping way for compression pages

2020-08-11 Thread Gao Xiang
not arguing this commit, just a note about this commit message. > > > >> 1048576000 bytes (0.9 G) copied, 9.146217 s, 109 M/s > > > >> 1048576000 bytes (0.9 G) copied, 9.997542 s, 100 M/s > > > >> 1048576000 bytes (0.9 G) copied, 10.109727 s, 99 M

Re: [f2fs-dev] [PATCH] f2fs: change virtual mapping way for compression pages

2020-08-11 Thread Gao Xiang
On Tue, Aug 11, 2020 at 12:37:53PM +0900, Daeho Jeong wrote: > From: Daeho Jeong > > By profiling f2fs compression works, I've found vmap() callings are > bottlenecks of f2fs decompression path. Changing these with > vm_map_ram(), we can enhance f2fs decompression speed pretty much. > >

Re: [f2fs-dev] IO hang due to f2fs checkpoint and writeback stuck

2020-07-09 Thread Gao Xiang via Linux-f2fs-devel
for in-core use, as well as blk_flush_plug(). Thanks, Gao Xiang ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] Fwd: Any tools of f2fs to inspect infos?

2020-07-01 Thread Gao Xiang
(cc linux-f2fs-devel, Jaegeuk, Chao. It'd be better to ask related people and cc the corresponding list.) On Wed, Jul 01, 2020 at 03:29:41PM +0800, lampahome wrote: > As title > > Any tools of f2fs to inspect like allocated segments, hot/warm/cold > ratio, or gc is running? > > thx

Re: [f2fs-dev] [PATCH] f2fs: compress: allow lz4 to compress data partially

2020-05-08 Thread Gao Xiang via Linux-f2fs-devel
lock. so IMO "partial compression" for an invalid lz4 block may be confusing. I think some words like "because lz4 implementation can handle output buffer budget properly" or similar stuff could be better. The same to the patch title and the commit message. Thanks, Gao Xiang

Re: [f2fs-dev] [PATCH] f2fs: get parent inode when recovering pino

2020-05-07 Thread Gao Xiang
Hi Chao, On Thu, May 07, 2020 at 02:38:39PM +0800, Chao Yu wrote: > On 2020/5/7 6:36, Gao Xiang wrote: > > On Wed, May 06, 2020 at 12:16:13PM -0700, Eric Biggers wrote: > >> On Wed, May 06, 2020 at 02:47:19PM +0800, Gao Xiang wrote: > >>> On Wed, May 06, 2020 at 09:

Re: [f2fs-dev] [PATCH] f2fs: get parent inode when recovering pino

2020-05-06 Thread Gao Xiang
On Wed, May 06, 2020 at 12:16:13PM -0700, Eric Biggers wrote: > On Wed, May 06, 2020 at 02:47:19PM +0800, Gao Xiang wrote: > > On Wed, May 06, 2020 at 09:58:22AM +0800, Gao Xiang wrote: > > > On Tue, May 05, 2020 at 06:24:28PM -0700, Eric Biggers wrote: > > > > On W

Re: [f2fs-dev] [PATCH] f2fs: get parent inode when recovering pino

2020-05-06 Thread Gao Xiang
On Wed, May 06, 2020 at 09:58:22AM +0800, Gao Xiang wrote: > On Tue, May 05, 2020 at 06:24:28PM -0700, Eric Biggers wrote: > > On Wed, May 06, 2020 at 08:14:07AM +0800, Gao Xiang wrote: > > > > > > > > Actually, I think this is wrong because the fsync can be d

Re: [f2fs-dev] [PATCH] f2fs: get parent inode when recovering pino

2020-05-05 Thread Gao Xiang via Linux-f2fs-devel
On Tue, May 05, 2020 at 06:24:28PM -0700, Eric Biggers wrote: > On Wed, May 06, 2020 at 08:14:07AM +0800, Gao Xiang wrote: > > > > > > Actually, I think this is wrong because the fsync can be done via a file > > > descriptor that was opened to a now-deleted link t

Re: [f2fs-dev] [PATCH] f2fs: get parent inode when recovering pino

2020-05-05 Thread Gao Xiang
nk == 1'. directory counting towards 'inode->i_nlink == 1', what's happening? > > I think d_find_alias() is what we're looking for. It may be simply dentry->d_parent (stable/positive as you said before, and it's not empty). why need to d_find_alias()? And what is the original proble

Re: [f2fs-dev] [PATCH v11 19/25] erofs: Convert compressed files from readpages to readahead

2020-04-21 Thread Gao Xiang via Linux-f2fs-devel
... } ... 230 err = bio_add_page(bio, page, PAGE_SIZE, 0); 231 /* out of the extent or bio is full */ 232 if (err < PAGE_SIZE) 233 goto submit_bio_retry; 234 235 *last_block = current_block; so bio != NULL, and last_block will be assigned then as well.

Re: [f2fs-dev] [PATCH v6 12/19] erofs: Convert uncompressed files from readpages to readahead

2020-02-18 Thread Gao Xiang
On Mon, Feb 17, 2020 at 10:46:01AM -0800, Matthew Wilcox wrote: > From: "Matthew Wilcox (Oracle)" > > Use the new readahead operation in erofs > > Signed-off-by: Matthew Wilcox (Oracle) > --- It looks good to me, and will test it later as well.. Acked-by: Gao Xian

Re: [f2fs-dev] [PATCH v6 11/16] erofs: Convert compressed files from readpages to readahead

2020-02-18 Thread Gao Xiang
d make a straight-forward transform first, and I haven't tested the whole series for now... Will test it later. Acked-by: Gao Xiang Thanks, Gao Xiang > --- > fs/erofs/zdata.c | 29 + > 1 file changed, 9 insertions(+), 20 deletions(-) > > diff --git a/fs/

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
On Thu, Jan 23, 2020 at 09:42:56PM -0800, Eric Biggers wrote: > On Fri, Jan 24, 2020 at 01:34:23PM +0800, Gao Xiang wrote: > > On Thu, Jan 23, 2020 at 09:16:01PM -0800, Eric Biggers wrote: > > > > [] > > > > > So we need READ_ONCE() to ensure that a consisten

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
her uses (such as, avoid accessing a variable twice due to compiler optimization and it will break some logic potentially or need some data dependency barrier...) Thanks, Gao Xiang ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
On Thu, Jan 23, 2020 at 09:16:01PM -0800, Eric Biggers wrote: > On Fri, Jan 24, 2020 at 01:04:25PM +0800, Gao Xiang wrote: > > > diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c > > > index 8964778aabefb..0129d14629881 100644 > > > --- a/fs/ext4/dir.c > > > +++

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
dentry->d_parent->d_inode; > + const struct dentry *parent = READ_ONCE(dentry->d_parent); I'm not sure if we really need READ_ONCE d_parent here (p.s. d_parent won't be NULL anyway), and d_seq will guard all its validity. If I'm wrong, correct me kindly... Otherwise, it looks

Re: [f2fs-dev] [PATCH v5] fs: introduce is_dot_or_dotdot helper for cleanup

2019-12-11 Thread Gao Xiang via Linux-f2fs-devel
Hi Matthew, On Wed, Dec 11, 2019 at 05:40:14AM -0800, Matthew Wilcox wrote: > On Wed, Dec 11, 2019 at 03:17:11PM +0800, Gao Xiang wrote: > > > static inline bool is_dot_or_dotdot(const unsigned char *name, size_t len) > > > { > > > i

Re: [f2fs-dev] [PATCH v5] fs: introduce is_dot_or_dotdot helper for cleanup

2019-12-10 Thread Gao Xiang
n for other callers: > > static inline bool is_dot_or_dotdot(const unsigned char *name, size_t len) > { > if (len >= 1 && unlikely(name[0] == '.')) { And I suggest drop "unlikely" here since files start with prefix '.' (plus specical ".", "..&

Re: [f2fs-dev] Potential data corruption?

2019-12-08 Thread Gao Xiang via Linux-f2fs-devel
rsisted. So if fsync() is not successful, the old data should be readed but for this case, unexpected data (not A or A', could be random data C) will be considered validly since its node is ok. It seems it should FLUSH data before the related node chain written or introduce some data checksum though. If I am wrong, kindly correct me... Thanks, Gao Xiang ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH 4/8] vfs: Fold casefolding into vfs

2019-12-02 Thread Gao Xiang
orage files are not casefolded (even in Android). "those dentry ops interfere with the dentry_ops required for fscrypt", I don't think it's a real diffculty and it could be done with some better approach instead. Thanks, Gao Xiang ___

Re: [f2fs-dev] [PATCH 10/10] errno.h: Provide EFSCORRUPTED for everybody

2019-11-03 Thread Gao Xiang
t; Acked-by: Theodore Ts'o For EROFS part, Acked-by: Gao Xiang > --- > drivers/staging/exfat/exfat.h| 2 -- > fs/erofs/internal.h | 2 -- > fs/ext4/ext4.h | 1 - > fs/f2fs/f2fs.h | 1 - > fs/xfs/xfs_linux.h

Re: [f2fs-dev] [PATCH 2/2] f2fs: support data compression

2019-10-30 Thread Gao Xiang via Linux-f2fs-devel
ce my English is not very well, I delay it util now... [1] https://elixir.bootlin.com/linux/v3.18.140/source/mm/zsmalloc.c#L379 https://lore.kernel.org/linux-mm/1459321935-3655-7-git-send-email-minc...@kernel.org and some not very related topic: https://lwn.net/Articles/752564/ Thanks, Ga

Re: [f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-30 Thread Gao Xiang via Linux-f2fs-devel
On Wed, Oct 30, 2019 at 09:33:13AM -0700, Jaegeuk Kim wrote: > On 10/30, Theodore Y. Ts'o wrote: > > On Wed, Oct 30, 2019 at 11:50:37PM +0800, Gao Xiang wrote: > > > > > > So I'm curious about the original issue in commit 740432f83560 > > > ("f2fs: h

Re: [f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-30 Thread Gao Xiang via Linux-f2fs-devel
Hi Ted, On Wed, Oct 30, 2019 at 11:14:45AM -0400, Theodore Y. Ts'o wrote: > On Wed, Oct 30, 2019 at 06:43:45PM +0800, Gao Xiang wrote: > > > You're right, in low memory scenario, allocation with bioset will be > > > faster, as > > > you mentioned offline, maybe we

Re: [f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-30 Thread Gao Xiang
On Wed, Oct 30, 2019 at 05:27:54PM +0800, Chao Yu wrote: > Hi Xiang, > > On 2019/10/30 17:15, Gao Xiang wrote: > > Hi Chao, > > > > On Wed, Oct 30, 2019 at 04:56:17PM +0800, Chao Yu wrote: > >> On 2019/10/30 11:55, Gao Xiang wrote: > >>> remo

Re: [f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-30 Thread Gao Xiang
Hi Chao, On Wed, Oct 30, 2019 at 04:56:17PM +0800, Chao Yu wrote: > On 2019/10/30 11:55, Gao Xiang wrote: > > remove such useless code and related fault injection. > > Hi Xiang, > > Although, there is so many 'nofail' allocation in f2fs, I think we'd better > avoid

[f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-29 Thread Gao Xiang
remove such useless code and related fault injection. Signed-off-by: Gao Xiang --- Documentation/filesystems/f2fs.txt | 1 - fs/f2fs/data.c | 6 ++ fs/f2fs/f2fs.h | 21 - fs/f2fs/segment.c | 5 + fs/f2fs

Re: [f2fs-dev] [PATCH] f2fs: fix comment of f2fs_evict_inode

2019-09-28 Thread Gao Xiang
On Sun, Sep 29, 2019 at 08:53:05AM +0800, Chao Yu wrote: > Hi Jaegeuk, > > On 2019/9/28 2:31, Jaegeuk Kim wrote: > > Hi Chao, > > > > On 09/25, Chao Yu wrote: > >> evict() should be called once i_count is zero, rather than i_nlinke > >> is zero. >

Re: [PATCH] f2fs: fix comment of f2fs_evict_inode

2019-09-27 Thread Gao Xiang
Hi Jaegeuk, On Fri, Sep 27, 2019 at 11:31:50AM -0700, Jaegeuk Kim wrote: > Hi Chao, > > On 09/25, Chao Yu wrote: > > evict() should be called once i_count is zero, rather than i_nlinke > > is zero. > > > > Reported-by: Gao Xiang > > Signed-off-by: Ch

Re: [f2fs-dev] [PATCH] f2fs: fix comment of f2fs_evict_inode

2019-09-25 Thread Gao Xiang
Hi Chao, On Wed, Sep 25, 2019 at 05:30:50PM +0800, Chao Yu wrote: > evict() should be called once i_count is zero, rather than i_nlinke > is zero. > > Reported-by: Gao Xiang > Signed-off-by: Chao Yu > --- > fs/f2fs/inode.c | 2 +- > 1 file changed, 1 insertion(+),

Re: [f2fs-dev] [PATCH V4 5/8] f2fs: Use read_callbacks for decrypting file data

2019-08-20 Thread Gao Xiang via Linux-f2fs-devel
Hi Ted, On Tue, Aug 20, 2019 at 12:25:10PM -0400, Theodore Y. Ts'o wrote: > On Tue, Aug 20, 2019 at 01:12:36PM +0800, Gao Xiang wrote: > > Add a word, I have some little concern about post read procession order > > a bit as I mentioned before, because I'd like to mo

Re: [f2fs-dev] [PATCH V4 5/8] f2fs: Use read_callbacks for decrypting file data

2019-08-19 Thread Gao Xiang
On Tue, Aug 20, 2019 at 01:12:36PM +0800, Gao Xiang wrote: > Hi Chandan, > > On Tue, Aug 20, 2019 at 10:35:29AM +0530, Chandan Rajendra wrote: > > On Friday, August 16, 2019 11:48 AM Chandan Rajendra wrote: > > > F2FS has a copy of "post read processing" code usi

Re: [f2fs-dev] [PATCH V4 5/8] f2fs: Use read_callbacks for decrypting file data

2019-08-19 Thread Gao Xiang
ther fses to use after we think it's mature enough. It seems the current code mainly addresses eliminating duplicated code, therefore I have no idea about that... Thanks, Gao Xiang > > -- > chandan > > > ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH v2] f2fs: ratelimit recovery messages

2019-05-27 Thread Gao Xiang
; for knowing the status of recovered files I thought. Do you think we need >> individually each file status as well? > > Yes, I think so, we need them for the detailed info. :) I personally agree with Chao's suggestion as well. Sometimes huawei got stuck into rare potential f2fs stability

Re: [f2fs-dev] [PATCH v2] f2fs: fix to avoid deadlock of atomic file operations

2019-02-25 Thread Gao Xiang
On 2019/2/25 17:34, Chao Yu wrote: > Hi Xiang, > > On 2019/2/25 17:25, Gao Xiang wrote: >> Hi Chao, >> >> On 2019/2/25 17:11, Chao Yu wrote: >>> Thread AThread B >>> - __fput >>> - f2fs_release_file &g

Re: [f2fs-dev] [PATCH v2] f2fs: fix to avoid deadlock of atomic file operations

2019-02-25 Thread Gao Xiang
Sorry, please ignore my reply... (If there no truncate and commit race...) On 2019/2/25 17:25, Gao Xiang wrote: > Hi Chao, > > On 2019/2/25 17:11, Chao Yu wrote: >> Thread A Thread B >> - __fput >> - f2fs_release_file >> - dr

Re: [f2fs-dev] [PATCH v2] f2fs: fix to avoid deadlock of atomic file operations

2019-02-25 Thread Gao Xiang
b_info *sbi = F2FS_I_SB(inode); > struct inmem_pages *cur, *tmp; > @@ -227,7 +228,16 @@ static int __revoke_inmem_pages(struct inode *inode, > if (drop) > trace_f2fs_commit_inmem_page(page, INMEM_DROP); > > - lock_page(page); > + if (trylock) { &

[f2fs-dev] [PATCH] f2fs: no need to take page lock in readdir

2019-02-20 Thread Gao Xiang
VFS will take inode_lock for readdir, therefore no need to take page lock in readdir at all just as the majority of other generic filesystems. This patch improves concurrency since .iterate_shared was introduced to VFS years ago. Signed-off-by: Gao Xiang --- personally tend to use

[f2fs-dev] [PATCH] f2fs: silence VM_WARN_ON_ONCE in mempool_alloc

2019-02-18 Thread Gao Xiang
Note that __GFP_ZERO is not supported for mempool_alloc, which also documented in the mempool_alloc comments. Signed-off-by: Gao Xiang --- fs/f2fs/data.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index f91d8630c9a2..83c14b31aaba 100644

[f2fs-dev] [PATCH] f2fs: use xattr_prefix to wrap up

2019-01-25 Thread Gao Xiang
Let's use xattr_prefix instead of open code. No logic changes. Signed-off-by: Gao Xiang --- fs/f2fs/xattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/xattr.c b/fs/f2fs/xattr.c index 18d5ffbc5e8c..fa620d31ea5f 100644 --- a/fs/f2fs/xattr.c +++ b/fs/f2fs/xattr.c

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-26 Thread Gao Xiang via Linux-f2fs-devel
..) 2) How about adding a gfp_t input argument since I don't know whether GFP_KERNEL is suitable for all use cases... It seems there could be more fsverity_verify_page users as well as fsverity_verify_bio ;) Sorry for interruption... Thanks, Gao Xiang >>> + if (unlikely(!re

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-26 Thread Gao Xiang via Linux-f2fs-devel
Hi Ted, Sorry for the late reply... On 2018/8/26 1:06, Theodore Y. Ts'o wrote: > On Sat, Aug 25, 2018 at 03:43:43PM +0800, Gao Xiang wrote: >>> I don't know of any plan to use fs-verity on Android's system partition or >>> to >>> replace dm-verity on the system pa

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-25 Thread Gao Xiang
Hi Ted, Please ignore the following email, Eric has replied to me. :) I need to dig into these fs-verity patches later and best wishes to fs-verity. Thanks, Gao Xiang On 2018/8/25 15:33, Gao Xiang wrote: > Hi Ted, > > Thanks for your detailed reply. Sorry about my english, the wo

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-25 Thread Gao Xiang
t; On Sat, Aug 25, 2018 at 10:29:26AM +0800, Gao Xiang wrote: >> Hi, >> >> On 2018/8/25 0:16, Eric Biggers wrote: >>> +/** >>> + * fsverity_verify_page - verify a data page >>> + * >>> + * Verify a page that has just been read from a file aga

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-24 Thread Gao Xiang
Hi Ted, On 2018/8/25 11:45, Theodore Y. Ts'o wrote: > On Sat, Aug 25, 2018 at 10:29:26AM +0800, Gao Xiang wrote: >> My first question is that 'Is there any way to skip to verify pages in a >> bio?' >> I am thinking about >> If metadata and data page are mixed i

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-24 Thread Gao Xiang
em in per-file approach? at least --- support for the interface? At last, I hope filesystems could select the on-disk position of hash tree and 'struct fsverity_descriptor' rather than fixed in the end of verity files...I think if fs-verity preparing such support and interfaces could be b

Re: [f2fs-dev] [PATCH] f2fs: avoid the global name 'fault_name'

2018-06-30 Thread Gao Xiang
Oh, I just found the same patch written by Chao at: https://git.kernel.org/pub/scm/linux/kernel/git/chao/linux.git/commit/?h=f2fs-dev I just fixed the erofs name conflicts, so fixed f2fs as well. Please ignore this one. Thanks, Gao Xiang On 2018/6/30 23:57, Gao Xiang wrote: > Non-pre

[f2fs-dev] [PATCH] f2fs: avoid the global name 'fault_name'

2018-06-30 Thread Gao Xiang
-off-by: Gao Xiang --- fs/f2fs/f2fs.h | 4 ++-- fs/f2fs/super.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 4d8b1de..11a2e09 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -65,7 +65,7 @@ struct f2fs_fault_info { unsigned

Re: [f2fs-dev] [RESEND PATCH] f2fs: no need to take the address of the array of sb->s_uuid

2018-04-08 Thread Gao Xiang
Hi Chao and Jaegeuk, On 2018/4/8 20:16, Chao Yu wrote: > On 2018/4/5 11:58, Gao Xiang wrote: >> Keep in line with the common case since it is some weird >> to take the address of an array again. > > I encounter compile error after applying this patch: > > super.c: I

Re: [f2fs-dev] [PATCH] f2fs: no need to take the address of the array of sb->s_uuid

2018-04-04 Thread Gao Xiang
Hi Jaegeuk, On 2018/4/5 11:54, Jaegeuk Kim wrote: > Hi Gao, > > Could you please check your email settings? > It's broken. > > Thanks,Sorry to bother for the pervious email.. My business email client was just in a mess. :( Thanks, > > On 04/05, Gao Xian

[f2fs-dev] [RESEND PATCH] f2fs: no need to take the address of the array of sb->s_uuid

2018-04-04 Thread Gao Xiang
Keep in line with the common case since it is some weird to take the address of an array again. Signed-off-by: Gao Xiang <gaoxian...@huawei.com> --- fix auto-wrapping of email client fs/f2fs/super.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/super.c b/f

[f2fs-dev] [PATCH] f2fs: no need to take the address of the array of sb->s_uuid

2018-04-04 Thread Gao Xiang
Keep in line with the common case since it is some weird to take the address of an array again. Signed-off-by: Gao Xiang <gaoxian...@huawei.com> --- fs/f2fs/super.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index 42d564

[f2fs-dev] [PATCH RFC v5] f2fs: flush cp pack except cp pack 2 page at first

2018-02-09 Thread Gao Xiang via Linux-f2fs-devel
write in the UFS specification) This patch submits the whole cp pack except cp pack 2 page at first, and then writes the cp pack 2 page with an extra independent bio with pre-io barrier. Signed-off-by: Gao Xiang <gaoxian...@huawei.com> Reviewed-by: Chao Yu <yuch...@huawei.com> --- Change

Re: [f2fs-dev] [PATCH] f2fs: fix to handle looped node chain during recovery

2018-02-03 Thread Gao Xiang via Linux-f2fs-devel
Sorry, I saw the related code entirely, please ignore these replies. On 2018/2/3 18:45, Gao Xiang wrote: On 2018/2/3 18:35, Gao Xiang wrote: Hi Chao and YunLei, On 2018/2/3 17:44, Chao Yu wrote: There is no checksum in node block now, so bit-transition from hardware can make

Re: [f2fs-dev] [PATCH] f2fs: fix to handle looped node chain during recovery

2018-02-03 Thread Gao Xiang via Linux-f2fs-devel
On 2018/2/3 18:35, Gao Xiang wrote: Hi Chao and YunLei, On 2018/2/3 17:44, Chao Yu wrote: There is no checksum in node block now, so bit-transition from hardware can make node_footer.next_blkaddr being corrupted w/o any detection, result in node chain becoming looped one

Re: [f2fs-dev] [PATCH] f2fs: fix to handle looped node chain during recovery

2018-02-03 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao and YunLei, On 2018/2/3 17:44, Chao Yu wrote: There is no checksum in node block now, so bit-transition from hardware can make node_footer.next_blkaddr being corrupted w/o any detection, result in node chain becoming looped one. For this condition, during recovery, in order to avoid

Re: [f2fs-dev] [PATCH] f2fs: use crc ^ cp_ver instead of crc | cp_ver for recovery

2018-02-01 Thread Gao Xiang
Hi Chao, On 2018/2/1 22:30, Chao Yu wrote: We can use this calculation since cp_ver is complete 64-bits random number now. Sorry, I meant we can't. Alright, I think it is an unimportant proposal for now. We could fix now, sometime later or never :) Just saw by chance and a little

Re: [f2fs-dev] [PATCH] f2fs: use crc ^ cp_ver instead of crc | cp_ver for recovery

2018-02-01 Thread Gao Xiang
On 2018/2/1 21:57, Gao Xiang wrote: Hi Chao, On 2018/2/1 21:27, Chao Yu wrote: Hi Xiang, On 2018/2/1 0:16, Gaoxiang (OS) wrote: This patch add a flag CP_CRC_RECOVERY_XOR_FLAG to use XORed crc ^ cp_ver since crc | cp_ver is more likely to get a collision or become 11.. | cp_ver. FYI

Re: [f2fs-dev] [PATCH] f2fs: use crc ^ cp_ver instead of crc | cp_ver for recovery

2018-02-01 Thread Gao Xiang
because of another work and saw this part of code by chance. I think XOR-ed calculation is much better in mathematics, and typical method for mixing two values (eg. XOR encryption) is also XOR-ed calculation rather than OR-ed... Thanks, Thanks, Signed-off-by: Gao Xiang <gaox

Re: [f2fs-dev] [PATCH RFC v4] f2fs: flush cp pack except cp pack 2 page at first

2018-01-31 Thread Gao Xiang
with pre-io barrier. Signed-off-by: Gao Xiang <gaoxian...@huawei.com> Reviewed-by: Chao Yu <yuch...@huawei.com> --- Change log from v3: - further review comments are applied from Jaegeuk and Chao - Tested on this patch (without multiple-device): mount, boot Android with f2fs userd

Re: [f2fs-dev] [PATCH RFC] f2fs: flush cp pack except cp page2 at first

2018-01-24 Thread Gao Xiang via Linux-f2fs-devel
, but payload or current summaries are still outdated. (see reliable write in UFS spec) This patch write the whole cp pack except cp page2 with FLUSH at first, and then write the cp page2 with an extra independent bio with FLUSH. Signed-off-by: Gao Xiang <gaoxian...@huawei.com> --- fs/f2fs/checkp

Re: [f2fs-dev] [PATCH RFC] f2fs: add PRE2 to mark segments free to one checkpoint but obsolete to the other

2018-01-20 Thread Gao Xiang via Linux-f2fs-devel
or not. In addition, if some data segments change between checkpoint A and C, some weird data that it didn't have (new data or data from other files) would be gotten when switching back to checkpoint A. Thanks, Thanks, *From:*Gao Xiang *To:*guoweichao, *Cc:*linux-f2fs-devel@lists.sourceforge.net,heyunlei

Re: [f2fs-dev] [PATCH RFC] f2fs: add PRE2 to mark segments free to one checkpoint but obsolete to the other

2018-01-20 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao and Weichao, On 2018/1/21 10:34, Chao Yu wrote: Hi Weichao, On 2018/1/20 23:50, guoweichao wrote: Hi Chao, Yes, it is exactly what I mean. It seems that F2FS has no responsibility to cover hardware problems. However, file systems usually choose redundancy for super block fault

Re: [f2fs-dev] [PATCH RFC] f2fs: add PRE2 to mark segments free to one checkpoint but obsolete to the other

2018-01-19 Thread Gao Xiang via Linux-f2fs-devel
Hi Weichao, On 2018/1/19 23:47, guoweichao wrote: A critical case is using the free segments as data segments which are previously node segments to the old checkpoint. With fault injecting of the newer CP pack, fsck can find errors when checking the sanity of nid. Sorry to interrupt because

Re: [f2fs-dev] [PATCH v2] mkfs.f2fs: expand scalability of nat bitmap

2018-01-16 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao Yu, On 2018/1/17 11:15, Chao Yu wrote: Hi Jaegeuk, On 2018/1/17 8:47, Jaegeuk Kim wrote: Hi Chao, On 01/15, Chao Yu wrote: Previously, our total node number (nat_bitmap) and total nat segment count will not monotonously increase along with image size, and max nat_bitmap size is

Re: [f2fs-dev] [PATCH] mkfs.f2fs: expand scalability of nat bitmap

2018-01-13 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao, On 2018/01/12 18:24, Chao Yu wrote: Previously, our total node number (nat_bitmap) and total nat segment count will not monotonously increase along with image size, and max nat_bitmap size is limited by "CHECKSUM_OFFSET - sizeof(struct f2fs_checkpoint) + 1", it is with bad

[f2fs-dev] [PATCH RFC] f2fs: refactor get_new_segment

2017-12-16 Thread Gao Xiang via Linux-f2fs-devel
init = false; goto find_other_zone; } Signed-off-by: Gao Xiang <hsiang...@aol.com> --- fs/f2fs/segment.c | 137 -- 1 file changed, 81 insertions(+), 56 deletions(-) diff --git a/fs/f2fs/segment.c b/fs/f2fs/se

  1   2   >