[f2fs-dev] [PATCH RFC] f2fs: refactor get_new_segment

2017-12-16 Thread Gao Xiang via Linux-f2fs-devel
get_new_segment is too unclear to understand how it works. This patch refactor it in a straight-forward way and I think it is equal to the original one. This patch also fixes two issues in the original get_new_segment: 1) left_start could be overflowed when hint == 0 at first: ...

Re: [f2fs-dev] [PATCH RFC] f2fs: add PRE2 to mark segments free to one checkpoint but obsolete to the other

2018-01-20 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao and Weichao, On 2018/1/21 10:34, Chao Yu wrote: Hi Weichao, On 2018/1/20 23:50, guoweichao wrote: Hi Chao, Yes, it is exactly what I mean. It seems that F2FS has no responsibility to cover hardware problems. However, file systems usually choose redundancy for super block fault

Re: [f2fs-dev] [PATCH RFC] f2fs: add PRE2 to mark segments free to one checkpoint but obsolete to the other

2018-01-20 Thread Gao Xiang via Linux-f2fs-devel
Hi Weichao, On 2018/1/21 0:02, guoweichao wrote: Hi Xiang, it's not related to SPOR. Just consider the case given by Chao Yu. (It seems this email was not sent successfully, I resend it just for reference only) Oh I see, I have considered the scenario what Chao said before. 1. write

Re: [f2fs-dev] [PATCH v2] mkfs.f2fs: expand scalability of nat bitmap

2018-01-16 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao Yu, On 2018/1/17 11:15, Chao Yu wrote: Hi Jaegeuk, On 2018/1/17 8:47, Jaegeuk Kim wrote: Hi Chao, On 01/15, Chao Yu wrote: Previously, our total node number (nat_bitmap) and total nat segment count will not monotonously increase along with image size, and max nat_bitmap size is

Re: [f2fs-dev] [PATCH] mkfs.f2fs: expand scalability of nat bitmap

2018-01-13 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao, On 2018/01/12 18:24, Chao Yu wrote: Previously, our total node number (nat_bitmap) and total nat segment count will not monotonously increase along with image size, and max nat_bitmap size is limited by "CHECKSUM_OFFSET - sizeof(struct f2fs_checkpoint) + 1", it is with bad

Re: [f2fs-dev] [PATCH] f2fs: fix to handle looped node chain during recovery

2018-02-03 Thread Gao Xiang via Linux-f2fs-devel
Sorry, I saw the related code entirely, please ignore these replies. On 2018/2/3 18:45, Gao Xiang wrote: On 2018/2/3 18:35, Gao Xiang wrote: Hi Chao and YunLei, On 2018/2/3 17:44, Chao Yu wrote: There is no checksum in node block now, so bit-transition from hardware can make

Re: [f2fs-dev] [PATCH] f2fs: fix to handle looped node chain during recovery

2018-02-03 Thread Gao Xiang via Linux-f2fs-devel
On 2018/2/3 18:35, Gao Xiang wrote: Hi Chao and YunLei, On 2018/2/3 17:44, Chao Yu wrote: There is no checksum in node block now, so bit-transition from hardware can make node_footer.next_blkaddr being corrupted w/o any detection, result in node chain becoming looped one. For this

Re: [f2fs-dev] [PATCH] f2fs: fix to handle looped node chain during recovery

2018-02-03 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao and YunLei, On 2018/2/3 17:44, Chao Yu wrote: There is no checksum in node block now, so bit-transition from hardware can make node_footer.next_blkaddr being corrupted w/o any detection, result in node chain becoming looped one. For this condition, during recovery, in order to avoid

[f2fs-dev] [PATCH RFC v5] f2fs: flush cp pack except cp pack 2 page at first

2018-02-09 Thread Gao Xiang via Linux-f2fs-devel
Previously, we attempt to flush the whole cp pack in a single bio, however, when suddenly powering off at this time, we could get into an extreme scenario that cp pack 1 page and cp pack 2 page are updated and latest, but payload or current summaries are still partially outdated. (see reliable

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-26 Thread Gao Xiang via Linux-f2fs-devel
Hi Ted, Sorry for the late reply... On 2018/8/26 1:06, Theodore Y. Ts'o wrote: > On Sat, Aug 25, 2018 at 03:43:43PM +0800, Gao Xiang wrote: >>> I don't know of any plan to use fs-verity on Android's system partition or >>> to >>> replace dm-verity on the system partition. The use cases so far

Re: [f2fs-dev] [RFC PATCH 02/10] fs-verity: add data verification hooks for ->readpages()

2018-08-26 Thread Gao Xiang via Linux-f2fs-devel
Hi Eric, On 2018/8/27 1:04, Eric Biggers wrote: > Hi Chuck, > > On Sun, Aug 26, 2018 at 11:55:57AM -0400, Chuck Lever wrote: >>> + >>> +/** >>> + * fsverity_verify_page - verify a data page >>> + * >>> + * Verify a page that has just been read from a file against that file's >>> Merkle >>> + *

Re: [f2fs-dev] [PATCH RFC] f2fs: add PRE2 to mark segments free to one checkpoint but obsolete to the other

2018-01-19 Thread Gao Xiang via Linux-f2fs-devel
Hi Weichao, On 2018/1/19 23:47, guoweichao wrote: A critical case is using the free segments as data segments which are previously node segments to the old checkpoint. With fault injecting of the newer CP pack, fsck can find errors when checking the sanity of nid. Sorry to interrupt because

Re: [f2fs-dev] [PATCH RFC] f2fs: flush cp pack except cp page2 at first

2018-01-24 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao, On 2018/1/24 23:57, Chao Yu wrote: On 2018/1/24 14:53, Gaoxiang (OS) wrote: Previously, we attempt to flush the whole cp pack in a single bio, however, when suddenly power off at this time, we could meet an extreme scenario that cp page1 and cp page2 are updated and latest, but

Re: [f2fs-dev] [PATCH V4 5/8] f2fs: Use read_callbacks for decrypting file data

2019-08-20 Thread Gao Xiang via Linux-f2fs-devel
Hi Ted, On Tue, Aug 20, 2019 at 12:25:10PM -0400, Theodore Y. Ts'o wrote: > On Tue, Aug 20, 2019 at 01:12:36PM +0800, Gao Xiang wrote: > > Add a word, I have some little concern about post read procession order > > a bit as I mentioned before, because I'd like to move common EROFS > >

Re: [f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-30 Thread Gao Xiang via Linux-f2fs-devel
Hi Ted, On Wed, Oct 30, 2019 at 11:14:45AM -0400, Theodore Y. Ts'o wrote: > On Wed, Oct 30, 2019 at 06:43:45PM +0800, Gao Xiang wrote: > > > You're right, in low memory scenario, allocation with bioset will be > > > faster, as > > > you mentioned offline, maybe we can add/use a priviate bioset

Re: [f2fs-dev] [PATCH] f2fs: bio_alloc should never fail

2019-10-30 Thread Gao Xiang via Linux-f2fs-devel
On Wed, Oct 30, 2019 at 09:33:13AM -0700, Jaegeuk Kim wrote: > On 10/30, Theodore Y. Ts'o wrote: > > On Wed, Oct 30, 2019 at 11:50:37PM +0800, Gao Xiang wrote: > > > > > > So I'm curious about the original issue in commit 740432f83560 > > > ("f2fs: handle failed bio allocation"). Since f2fs

Re: [f2fs-dev] [PATCH 2/2] f2fs: support data compression

2019-10-30 Thread Gao Xiang via Linux-f2fs-devel
Hi Eric, (add some mm folks...) On Wed, Oct 30, 2019 at 09:50:56AM -0700, Eric Biggers wrote: > > >>> > > >>> It isn't really appropriate to create fake pagecache pages like this. > > >>> Did you > > >>> consider changing f2fs to use fscrypt_decrypt_block_inplace() instead? > > >> > > >> We

Re: [f2fs-dev] [PATCH v5] fs: introduce is_dot_or_dotdot helper for cleanup

2019-12-11 Thread Gao Xiang via Linux-f2fs-devel
Hi Matthew, On Wed, Dec 11, 2019 at 05:40:14AM -0800, Matthew Wilcox wrote: > On Wed, Dec 11, 2019 at 03:17:11PM +0800, Gao Xiang wrote: > > > static inline bool is_dot_or_dotdot(const unsigned char *name, size_t len) > > > { > > > if (len >= 1 && unlikely(name[0] == '.')) { > > > > > >

Re: [f2fs-dev] Potential data corruption?

2019-12-08 Thread Gao Xiang via Linux-f2fs-devel
Hi, On Sun, Dec 08, 2019 at 09:15:55PM +0800, Hongwei Qin wrote: > Hi, > > On Sun, Dec 8, 2019 at 12:01 PM Chao Yu wrote: > > > > Hello, > > > > On 2019-12-7 18:10, 锟斤拷锟秸碉拷锟斤拷锟斤拷锟斤拷 wrote: > > > Hi F2FS experts, > > > The following confuses me: > > > > > > A typical fsync() goes like this: > >

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
Hi Eric, On Thu, Jan 23, 2020 at 08:12:34PM -0800, Eric Biggers wrote: > From: Eric Biggers > > Since ->d_compare() and ->d_hash() can be called in RCU-walk mode, > ->d_parent and ->d_inode can be concurrently modified, and in > particular, ->d_inode may be changed to NULL. For ext4_d_hash()

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
On Thu, Jan 23, 2020 at 09:16:01PM -0800, Eric Biggers wrote: [] > So we need READ_ONCE() to ensure that a consistent value is used. By the way, my understanding is all pointer could be accessed atomicly guaranteed by compiler. In my opinion, we generally use READ_ONCE() on pointers for other

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
On Thu, Jan 23, 2020 at 09:16:01PM -0800, Eric Biggers wrote: > On Fri, Jan 24, 2020 at 01:04:25PM +0800, Gao Xiang wrote: > > > diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c > > > index 8964778aabefb..0129d14629881 100644 > > > --- a/fs/ext4/dir.c > > > +++ b/fs/ext4/dir.c > > > @@ -671,9 +671,11 @@

Re: [f2fs-dev] [PATCH] ext4: fix race conditions in ->d_compare() and ->d_hash()

2020-01-23 Thread Gao Xiang via Linux-f2fs-devel
On Thu, Jan 23, 2020 at 09:42:56PM -0800, Eric Biggers wrote: > On Fri, Jan 24, 2020 at 01:34:23PM +0800, Gao Xiang wrote: > > On Thu, Jan 23, 2020 at 09:16:01PM -0800, Eric Biggers wrote: > > > > [] > > > > > So we need READ_ONCE() to ensure that a consistent value is used. > > > > By the way,

Re: [f2fs-dev] [PATCH v11 19/25] erofs: Convert compressed files from readpages to readahead

2020-04-21 Thread Gao Xiang via Linux-f2fs-devel
Hi Andrew, On Mon, Apr 20, 2020 at 10:42:10PM -0700, Andrew Morton wrote: > On Tue, 14 Apr 2020 08:02:27 -0700 Matthew Wilcox wrote: > > > > > Use the new readahead operation in erofs. > > > > Well this is exciting. > > fs/erofs/data.c: In function erofs_raw_access_readahead: >

Re: [f2fs-dev] [PATCH] f2fs: get parent inode when recovering pino

2020-05-05 Thread Gao Xiang via Linux-f2fs-devel
On Tue, May 05, 2020 at 06:24:28PM -0700, Eric Biggers wrote: > On Wed, May 06, 2020 at 08:14:07AM +0800, Gao Xiang wrote: > > > > > > Actually, I think this is wrong because the fsync can be done via a file > > > descriptor that was opened to a now-deleted link to the file. > > > > I'm still

Re: [f2fs-dev] [PATCH] f2fs: compress: allow lz4 to compress data partially

2020-05-08 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao, On Fri, May 08, 2020 at 05:47:09PM +0800, Chao Yu wrote: > For lz4 worst compress case, caller should allocate buffer with size > of LZ4_compressBound(inputsize) for target compressed data storing. > > However lz4 supports partial data compression, so we can get rid of > output buffer

Re: [f2fs-dev] IO hang due to f2fs checkpoint and writeback stuck

2020-07-09 Thread Gao Xiang via Linux-f2fs-devel
On Fri, Jul 10, 2020 at 10:54:13AM +0800, Chao Yu wrote: > Hi Sahitya, > > It looks block plug has already been removed by Jaegeuk with > below commit: > > commit 1f5f11a3c41e2b23288b2769435a00f74e02496b > Author: Jaegeuk Kim > Date: Fri May 8 12:25:45 2020 -0700 > > f2fs: remove

Re: [f2fs-dev] 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-06 Thread Gao Xiang via Linux-f2fs-devel
Hi Chao, On Thu, May 06, 2021 at 05:15:04PM +0800, Chao Yu wrote: > On 2021/4/26 17:00, Gao Xiang wrote: > > On Mon, Apr 26, 2021 at 04:42:20PM +0800, changfeng...@vivo.com wrote: > > > Thank you for the reminder, I hadn't thought about fallocate before. > > > I have done some tests and the

Re: [f2fs-dev] 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-06 Thread Gao Xiang via Linux-f2fs-devel
On Thu, May 06, 2021 at 06:37:45PM +0800, Chao Yu wrote: > Hi Xiang, > > On 2021/5/6 17:58, Gao Xiang wrote: > > Hi Chao, > > > > On Thu, May 06, 2021 at 05:15:04PM +0800, Chao Yu wrote: > > > On 2021/4/26 17:00, Gao Xiang wrote: > > > > On Mon, Apr 26, 2021 at 04:42:20PM +0800,

Re: [f2fs-dev] 答复: 答复: [PATCH] f2fs: compress: avoid unnecessary check in f2fs_prepare_compress_overwrite

2021-05-06 Thread Gao Xiang via Linux-f2fs-devel
On Thu, May 06, 2021 at 08:15:40PM +0800, changfeng...@vivo.com wrote: > This patch will not bring significant performance improvements, I > test this on mobile phone, use androbench, the sequential write test > case was open file with O_TRUNC, set write size to 4KB, performance > improved about