Re: [f2fs-dev] [PATCH] f2fs: avoid frequent costly fsck triggers

2018-11-30 Thread Jaegeuk Kim
On 11/30, Chao Yu wrote: > On 2018/11/30 10:35, Sheng Yong wrote: > > Hi, Jaegeuk and Chao, > > > > On 2018/11/29 1:48, Jaegeuk Kim wrote: > >> On 11/28, Chao Yu wrote: > >>> On 2018/11/28 16:10, Jaegeuk Kim wrote: > >>>> On 11/28, Chao Yu

Re: [f2fs-dev] [PATCH] f2fs: check memory boundary by insane namelen

2018-11-26 Thread Jaegeuk Kim
On 11/23, Sheng Yong wrote: > Hi, Jaegeuk and Chao, > > On 2018/11/15 15:50, Jaegeuk Kim wrote: > > If namelen is corrupted to have very long value, fill_dentries can copy > > wrong memory area. > > > Is there any scenario that could hit this corruption? Or

Re: linux-next: build warning after merge of the f2fs tree

2018-11-26 Thread Jaegeuk Kim
On 11/27, Stephen Rothwell wrote: > Hi Jaegeuk, > > On Mon, 26 Nov 2018 13:59:24 -0800 Jaegeuk Kim wrote: > > > > On 11/26, Stephen Rothwell wrote: > > > > > > After merging the f2fs tree, today's linux-next build > > > (x86_64_allmodconfig) prod

Re: linux-next: build warning after merge of the f2fs tree

2018-11-26 Thread Jaegeuk Kim
Hi Stephen, On 11/26, Stephen Rothwell wrote: > Hi Jaegeuk, > > After merging the f2fs tree, today's linux-next build > (x86_64_allmodconfig) produced this warning: > > In file included from fs/f2fs/dir.c:11: > fs/f2fs/f2fs.h: In function '__mark_inode_dirty_flag': > fs/f2fs/f2fs.h:2388:6:

Re: linux-next: build failure after merge of the vfs tree

2018-10-16 Thread Jaegeuk Kim
On 10/16, Stephen Rothwell wrote: > Hi all, > > On Wed, 3 Oct 2018 10:32:22 +1000 Stephen Rothwell > wrote: > > > > After merging the vfs tree, today's linux-next build (x86_64 allmodconfig) > > failed like this: > > > > /home/sfr/next/next/fs/f2fs/super.c: In function 'f2fs_remount': > >

Re: [f2fs-dev] [PATCH] f2fs: fix quota info to adjust recovered data

2018-09-19 Thread Jaegeuk Kim
On 09/19, Chao Yu wrote: > On 2018/9/19 0:45, Jaegeuk Kim wrote: > > On 09/18, Chao Yu wrote: > >> On 2018/9/18 10:05, Jaegeuk Kim wrote: > >>> On 09/18, Chao Yu wrote: > >>>> On 2018/9/18 9:19, Jaegeuk Kim wrote: > >>>>> On 09/1

Re: [PATCH v6] f2fs: guarantee journalled quota data by checkpoint

2018-09-07 Thread Jaegeuk Kim
Chao, I was testing the previous patch and removed in the queue due to quota-related hang during fault injection + shutdown test. Let me try this later. Thanks, On 09/07, Chao Yu wrote: > For journalled quota mode, let checkpoint to flush dquot dirty data > and quota file data to guarntee

[PATCH] f2fs: submit bio after shutdown

2018-09-07 Thread Jaegeuk Kim
Sometimes, some merged IOs could get a chance to be submitted, resulting in system hang in shutdown test. This issues IOs all the time after shutdown. Signed-off-by: Jaegeuk Kim --- fs/f2fs/data.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index

Re: [f2fs-dev] [PATCH v3] f2fs: fix performance issue observed with multi-thread sequential read

2018-08-14 Thread Jaegeuk Kim
On 08/14, Chao Yu wrote: > On 2018/8/14 12:04, Jaegeuk Kim wrote: > > On 08/14, Chao Yu wrote: > >> On 2018/8/14 4:11, Jaegeuk Kim wrote: > >>> On 08/13, Chao Yu wrote: > >>>> Hi Jaegeuk, > >>>> > >>>> On 2018/8/11 2:56, J

Re: [PATCH 2/2] f2fs: tune discard speed with storage usage rate

2018-08-13 Thread Jaegeuk Kim
On 08/10, Chao Yu wrote: > Previously, discard speed was fixed mostly, and in high usage rate > device, we will speed up issuing discard, but it doesn't make sense > that in a non-full filesystem, we still issue discard with slow speed. Could you please elaborate the problem in more detail? The

Re: [PATCH v6 1/2] f2fs: fix to avoid broken of dnode block list

2018-08-01 Thread Jaegeuk Kim
On 07/29, Chao Yu wrote: > From: Chao Yu > > f2fs recovery flow is relying on dnode block link list, it means fsynced > file recovery depends on previous dnode's persistence in the list, so > during fsync() we should wait on all regular inode's dnode writebacked > before issuing flush. > > By

Re: [PATCH] f2fs: fix invalid memory access

2018-08-01 Thread Jaegeuk Kim
On 08/01, Chao Yu wrote: > From: Chao Yu > > syzbot found the following crash on: > > HEAD commit:d9bd94c0bcaa Add linux-next specific files for 20180801 > git tree: linux-next > console output: https://syzkaller.appspot.com/x/log.txt?x=1001189c40 > kernel config:

[PATCH] f2fs: avoid f2fs_bug_on() in cp_error case

2018-08-01 Thread Jaegeuk Kim
There is a subtle race condition to invoke f2fs_bug_on() in shutdown tests. I've confirmed that the last checkpoint is preserved in consistent state, so it'd be fine to just return error at this moment. Signed-off-by: Jaegeuk Kim --- fs/f2fs/node.c | 4 1 file changed, 4 insertions

[PATCH] f2fs: avoid fi->i_gc_rwsem[WRITE] lock in f2fs_gc

2018-07-24 Thread Jaegeuk Kim
The f2fs_gc() called by f2fs_balance_fs() requires to be called outside of fi->i_gc_rwsem[WRITE], since f2fs_gc() can try to grab it in a loop. Signed-off-by: Jaegeuk Kim --- fs/f2fs/file.c| 2 ++ fs/f2fs/segment.c | 1 - 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/f

Re: [PATCH v2] f2fs: split discard command in prior to block layer

2018-07-23 Thread Jaegeuk Kim
On 07/15, Chao Yu wrote: > On 2018/7/15 11:13, Jaegeuk Kim wrote: > > On 07/15, Chao Yu wrote: > >> Hi Jaegeuk, > >> > >> On 2018/7/15 9:27, Jaegeuk Kim wrote: > >>> On 07/08, Chao Yu wrote: > >>>> From: Chao Yu &g

Re: [PATCH 1/1] f2fs: checkpoint disabling

2018-07-14 Thread Jaegeuk Kim
On 07/11, Daniel Rosenberg wrote: > This adds a lightweight non-persistent snapshotting scheme to f2fs. > > To use, mount with the option checkpoint=disable, and to return to > normal operation, remount with checkpoint=enable. If the filesystem > is shut down before remounting with

Re: [PATCH] f2fs: split discard command in prior to block layer

2018-07-06 Thread Jaegeuk Kim
On 07/07, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/7/7 6:45, Jaegeuk Kim wrote: > > On 07/04, Chao Yu wrote: > >> From: Chao Yu > >> > >> Some devices has small max_{hw,}discard_sectors, so that in > >> __blkdev_issue_discard(), one big size dis

[PATCH 2/4] f2fs: allow wrong configure dio to buffered write

2018-07-06 Thread Jaegeuk Kim
This fixes to support unaligned dio as buffered writes. Signed-off-by: Jaegeuk Kim --- fs/f2fs/data.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index e66379961804..6e8e78bb64a7 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -2425,7

[GIT PULL] f2fs update for 4.18-rc1

2018-06-11 Thread Jaegeuk Kim
unlock_new_inode() before d_instantiate() f2fs: fix features filename in sysfs documentation Jaegeuk Kim (16): Revert "f2fs: introduce f2fs_set_page_dirty_nobuffer" f2fs: check cap_resource only for data blocks f2fs: clear PageError on writepage f2fs: av

Re: [PATCH 3/3] f2fs: clean up symbol namespace

2018-05-29 Thread Jaegeuk Kim
Hi Chao, Thank you for the work. I resolved some conflicts and modified some function names. Please take a look at this. https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/commit/?h=dev-test --- fs/f2fs/checkpoint.c | 8 fs/f2fs/dir.c| 6 +++--- fs/f2fs/f2fs.h

[RFC PATCH] f2fs: add fsync_mode=nobarrier for non-atomic files

2018-05-25 Thread Jaegeuk Kim
For non-atomic files, this patch adds an option to give nobarrier which doesn't issue flush commands to the device. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- Documentation/filesystems/f2fs.txt | 16 +--- fs/f2fs/f2fs.h | 1 + fs/f2fs/

Re: [PATCH v2] f2fs: let fstrim issue discard commands in lower priority

2018-05-25 Thread Jaegeuk Kim
size to 2MB per candidate, and check IO congestion when issuing them to disk. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- Change log from v1: - wait all discard bios in put_super & congested case in trimfs fs/f2fs/f2fs.h| 4 +- fs/f2fs/segme

Re: [PATCH] f2fs: let fstrim issue discard commands in lower priority

2018-05-25 Thread Jaegeuk Kim
On 05/25, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/5/25 13:10, Jaegeuk Kim wrote: > > The fstrim gathers huge number of large discard commands, and tries to issue > > without IO awareness, which results in long user-perceive IO latencies on > > READ, WRITE, and FLUSH

[PATCH] f2fs: let fstrim issue discard commands in lower priority

2018-05-24 Thread Jaegeuk Kim
size to 2MB per candidate, and check IO congestion when issuing them to disk. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/f2fs.h| 4 +- fs/f2fs/segment.c | 123 +++--- 2 files changed, 64 insertions(+), 63 deletions(-) diff --gi

Re: [PATCH v2] f2fs: Fix deadlock in shutdown ioctl

2018-05-17 Thread Jaegeuk Kim
On 05/17, Chao Yu wrote: > On 2018/5/17 16:03, Sahitya Tummala wrote: > > f2fs_ioc_shutdown() ioctl gets stuck in the below path > > when issued with F2FS_GOING_DOWN_FULLSYNC option. > > > > __switch_to+0x90/0xc4 > > percpu_down_write+0x8c/0xc0 > > freeze_super+0xec/0x1e4 > >

Re: [PATCH] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Jaegeuk Kim
On 05/08, Chao Yu wrote: > Thread A Thread B > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store >sbi->gc_thread = NULL; > access sbi->gc_thread->gc_* > > Previously, we allocate memory for sbi->gc_thread

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-07 Thread Jaegeuk Kim
On 05/08, Chao Yu wrote: > On 2018/5/8 4:46, Jaegeuk Kim wrote: > > On 04/27, Chao Yu wrote: > >> On 2018/4/27 0:36, Jaegeuk Kim wrote: > >>> On 04/26, Chao Yu wrote: > >>>> On 2018/4/26 23:48, Jaegeuk Kim wrote: >

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Jaegeuk Kim
On 05/08, Chao Yu wrote: > On 2018/5/8 5:36, Jaegeuk Kim wrote: > > On 05/07, Chao Yu wrote: > >> Thread A Thread BThread C > >> - f2fs_remount > >> - stop_gc_thread > >>

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Jaegeuk Kim
Signed-off-by: Chao Yu <yuch...@huawei.com> > --- > v4: > - use introduced sbi.gc_rwsem lock instead of sb.s_umount. We can use this first. >From e62e8d3ece6ee8a4aeac8ffd6161d25851f8b3f0 Mon Sep 17 00:00:00 2001 From: Jaegeuk Kim <jaeg...@kernel.org> Date: M

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-07 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/27 0:36, Jaegeuk Kim wrote: > > On 04/26, Chao Yu wrote: > >> On 2018/4/26 23:48, Jaegeuk Kim wrote: > >>> On 04/26, Chao Yu wrote: > >>>> Thread A Thread B > >>>>

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-05-04 Thread Jaegeuk Kim
On 05/02, Chao Yu wrote: > On 2018/4/28 10:34, Jaegeuk Kim wrote: > > On 04/27, Chao Yu wrote: > >> On 2018/4/26 23:54, Jaegeuk Kim wrote: > >>> On 04/24, Chao Yu wrote: > >>>> f2fs doesn't allow abuse on atomic write class interface, so except >

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-05-04 Thread Jaegeuk Kim
On 05/02, Chao Yu wrote: > On 2018/4/28 10:36, Jaegeuk Kim wrote: > > On 04/27, Chao Yu wrote: > >> On 2018/4/27 0:03, Jaegeuk Kim wrote: > >>> On 04/24, Chao Yu wrote: > >>>> Thread A Thread BThread C &

[PATCH] f2fs: avoid fsync() failure caused by EAGAIN in writepage()

2018-05-04 Thread Jaegeuk Kim
pageout() in MM traslates EAGAIN, so calls handle_write_error() -> mapping_set_error() -> set_bit(AS_EIO, ...). file_write_and_wait_range() will see EIO error, which is critical to return value of fsync() followed by atomic_write failure to user. Signed-off-by: Jaegeuk Kim <jaeg...@k

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-27 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/27 10:04, Chao Yu wrote: > > On 2018/4/27 0:03, Jaegeuk Kim wrote: > >> On 04/24, Chao Yu wrote: > >>> Thread A Thread BThread C > >>> - f2fs_remount > >>> - stop_gc_t

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-27 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/27 0:03, Jaegeuk Kim wrote: > > On 04/24, Chao Yu wrote: > >> Thread A Thread BThread C > >> - f2fs_remount > >> - stop_gc_thread > >>

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-04-27 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/26 23:54, Jaegeuk Kim wrote: > > On 04/24, Chao Yu wrote: > >> f2fs doesn't allow abuse on atomic write class interface, so except > >> limiting in-mem pages' total memory usage capacity, we need to limit > >> atomic

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Jaegeuk Kim
On 04/26, Chao Yu wrote: > On 2018/4/26 23:48, Jaegeuk Kim wrote: > > On 04/26, Chao Yu wrote: > >> Thread A Thread B > >> - f2fs_ioc_commit_atomic_write > >> - commit_inmem_pages > >> - f2fs

Re: [PATCH 3/3] f2fs: maintain discard interface separately

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > This patch adds a new file discard.c to maintain discard related > function separately. I don't think we need this at all. Thanks, > > BTW, fix below checkpatch errors: > > ERROR: space required before the open brace '{' > + } else if (issued == -1){ > >

Re: [PATCH 2/3] f2fs: introduce {create,destroy}_discard_caches for cleanup

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > Split discard slab cache related initial/release codes into separated > function {create,destroy}_discard_caches, later we can maintain those > independent functions in separated discard.c No need this as well. > > Signed-off-by: Chao Yu > --- >

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-26 Thread Jaegeuk Kim
On 04/24, Chao Yu wrote: > Thread A Thread BThread C > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store > - issue_discard_thread >sbi->gc_thread = NULL; >

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-04-26 Thread Jaegeuk Kim
On 04/24, Chao Yu wrote: > f2fs doesn't allow abuse on atomic write class interface, so except > limiting in-mem pages' total memory usage capacity, we need to limit > atomic-write usage as well when filesystem is seriously fragmented, > otherwise we may run into infinite loop during foreground GC

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Jaegeuk Kim
On 04/26, Chao Yu wrote: > Thread A Thread B > - f2fs_ioc_commit_atomic_write > - commit_inmem_pages > - f2fs_submit_merged_write_cond > : write data > - write_checkpoint >-

Re: [f2fs-dev] [PATCH 5/5] f2fs: enforce fsync_mode=strict for renamed directory

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > On 2018/4/25 13:46, Jaegeuk Kim wrote: > > This is to give a option for user to be able to recover B/foo in the below > > case. > > > > mkdir A > > sync() > > rename(A, B) > > creat (B/foo) > > fsync (B/foo) > >

Re: [f2fs-dev] [PATCH 4/5 v2] f2fs: sanity check for total valid blocks

2018-04-26 Thread Jaegeuk Kim
-by: syzbot+0a725420475916460...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- Change log from v1: - check valid node count fs/f2fs/segment.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 20250b

Re: [f2fs-dev] [PATCH 4/5] f2fs: sanity check for total valid blocks

2018-04-26 Thread Jaegeuk Kim
nt, which works for syzbot case as well. > > Thanks, > > On 2018/4/25 13:46, Jaegeuk Kim wrote: > > This patch enhances sanity check for SIT entries. > > > > syzbot hit the following crash on upstream commit > > 83beed7b2b26f232d782127792dd0cd4362fdc41 (Fri

Re: [f2fs-dev] [PATCH 1/5] f2fs: give message and set need_fsck given broken node id

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > On 2018/4/25 13:46, Jaegeuk Kim wrote: > > syzbot hit the following crash on upstream commit > > 83beed7b2b26f232d782127792dd0cd4362fdc41 (Fri Apr 20 17:56:32 2018 +) > > Merge branch 'fixes' of > > git://git.kernel.org/pub/scm/linux/ke

Re: [PATCH] f2fs: clear page_error for all the writebacking pages

2018-04-26 Thread Jaegeuk Kim
On 04/24, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/4/24 6:49, Jaegeuk Kim wrote: > > This patch clear page_error bit, if the page is going to be writebacked. > > This patch is similar to previous patch ("f2fs: clear PageError on > writepage"), > only covera

[PATCH 2/5] f2fs: avoid bug_on on corrupted inode

2018-04-24 Thread Jaegeuk Kim
[inline] RSP: 8801c44a70e8 RIP: f2fs_iget+0x3307/0x3ca0 fs/f2fs/inode.c:313 RSP: 8801c44a70e8 invalid opcode: [#2] SMP KASAN ---[ end trace 1cbcbec2156680bc ]--- Reported-and-tested-by: syzbot+41a1b341571f0952b...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.

[PATCH 4/5] f2fs: sanity check for total valid blocks

2018-04-24 Thread Jaegeuk Kim
-by: syzbot+0a725420475916460...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/segment.c | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 20250b88bf51..a55647f61232 100644 --- a/fs/f2fs/segment.c ++

[PATCH 5/5] f2fs: enforce fsync_mode=strict for renamed directory

2018-04-24 Thread Jaegeuk Kim
This is to give a option for user to be able to recover B/foo in the below case. mkdir A sync() rename(A, B) creat (B/foo) fsync (B/foo) ---crash--- Sugessted-by: Velayudhan Pillai <vi...@cs.utexas.edu> Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/namei.c | 5 +

[PATCH 3/5] f2fs: sanity check on sit entry

2018-04-24 Thread Jaegeuk Kim
-and-tested-by: syzbot+83699adeb2d13579c...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/segment.c | 9 + 1 file changed, 9 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index e4e8bdd645ee..20250b88bf51 100644 --- a/fs/f2fs/segment.c

[PATCH 1/5] f2fs: give message and set need_fsck given broken node id

2018-04-24 Thread Jaegeuk Kim
: __get_node_page+0xb68/0x16e0 fs/f2fs/node.c:1185 RSP: 8801d960e820 ---[ end trace 4edbeb71f002bb76 ]--- Reported-and-tested-by: syzbot+d154ec99402c6f628...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/f2fs.h | 13 + fs/f2fs/inode.

[PATCH] f2fs: avoid bug_on on corrupted inode

2018-04-24 Thread Jaegeuk Kim
[inline] RSP: 8801c44a70e8 RIP: f2fs_iget+0x3307/0x3ca0 fs/f2fs/inode.c:313 RSP: 8801c44a70e8 invalid opcode: [#2] SMP KASAN ---[ end trace 1cbcbec2156680bc ]--- Reported-and-tested-by: syzbot+41a1b341571f0952b...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.

Re: kernel BUG at fs/f2fs/node.c:LINE!

2018-04-24 Thread Jaegeuk Kim
On 04/24, Dmitry Vyukov wrote: > On Tue, Apr 24, 2018 at 8:42 AM, syzbot > wrote: > > Hello, > > > > syzbot tried to test the proposed patch but build/boot failed: > > > > failed to create VM pool: failed to create GCE image: failed to get

Re: kernel BUG at fs/f2fs/node.c:LINE!

2018-04-24 Thread Jaegeuk Kim
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git dev >From 65b966f58e8264477da6c8b70703ce362cb1f8f0 Mon Sep 17 00:00:00 2001 From: Jaegeuk Kim <jaeg...@kernel.org> Date: Mon, 23 Apr 2018 23:02:31 -0600 Subject: [PATCH] f2fs: give message and set need_fsck given br

Re: kernel BUG at fs/f2fs/node.c:LINE!

2018-04-24 Thread Jaegeuk Kim
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git dev-test >From 65b966f58e8264477da6c8b70703ce362cb1f8f0 Mon Sep 17 00:00:00 2001 From: Jaegeuk Kim <jaeg...@kernel.org> Date: Mon, 23 Apr 2018 23:02:31 -0600 Subject: [PATCH] f2fs: give message and set need_f

[PATCH] f2fs: give message and set need_fsck given broken node id

2018-04-23 Thread Jaegeuk Kim
: __get_node_page+0xb68/0x16e0 fs/f2fs/node.c:1185 RSP: 8801d960e820 ---[ end trace 4edbeb71f002bb76 ]--- Reported-by: syzbot+d154ec99402c6f628...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/f2fs.h | 13 + fs/f2fs/inode.c | 6 +- f

[PATCH] f2fs: clear page_error for all the writebacking pages

2018-04-23 Thread Jaegeuk Kim
This patch clear page_error bit, if the page is going to be writebacked. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/gc.c | 1 + fs/f2fs/inline.c | 1 + fs/f2fs/node.c| 1 + fs/f2fs/segment.c | 1 + 4 files changed, 4 insertions(+) diff --git a/fs/f2fs/gc.c b/f

Re: [PATCH 3/3] f2fs: wait IO writeback in commit_inmem_pages()

2018-04-23 Thread Jaegeuk Kim
On 04/23, Chao Yu wrote: > Thread A Thread B > - f2fs_ioc_commit_atomic_write > - commit_inmem_pages > - f2fs_submit_merged_write_cond > : write data > - write_checkpoint >-

Re: [PATCH] f2fs: sepearte hot/cold in free nid

2018-04-23 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > On 2018/4/20 12:04, Chao Yu wrote: > > On 2018/4/20 11:37, Jaegeuk Kim wrote: > >> On 04/20, Chao Yu wrote: > >>> As most indirect node, dindirect node, and xattr node won't be updated > >>> after they are created, but in

Re: [PATCH v3] ext4: handle errors on ext4_commit_super

2018-04-23 Thread Jaegeuk Kim
block 0, lost sync page write EXT4-fs (dm-1): re-mounted. Opts: (null) Buffer I/O error on dev dm-1, logical block 80, lost async page write Cc: "Theodore Ts'o" <ty...@mit.edu> Cc: Andreas Dilger <adilger.ker...@dilger.ca> Cc: linux-e...@vger.kernel.org Cc: Jaegeuk Kim <j

[PATCH] f2fs: check cap_resource only for data blocks

2018-04-21 Thread Jaegeuk Kim
This patch changes the rule to check cap_resource for data blocks, not inode or node blocks in order to avoid selinux denial. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/f2fs.h | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/f2fs/f2fs.h b/f

[PATCH] Revert "f2fs: introduce f2fs_set_page_dirty_nobuffer"

2018-04-20 Thread Jaegeuk Kim
This patch reverts copied f2fs_set_page_dirty_nobuffer to use generic function for stability. This reverts commit fe76b796fc5194cc3d57265002e3a748566d073f. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/checkpoint.c | 2 +- fs/f2fs/data.c

Re: [PATCH 5/5] f2fs: fix to avoid race during access gc_thread pointer

2018-04-19 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > On 2018/4/20 11:19, Jaegeuk Kim wrote: > > On 04/18, Chao Yu wrote: > >> Thread A Thread BThread C > >> - f2fs_remount > >> - stop_gc_thread > >>

Re: [PATCH] f2fs: sepearte hot/cold in free nid

2018-04-19 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > As most indirect node, dindirect node, and xattr node won't be updated > after they are created, but inode node and other direct node will change > more frequently, so store their nat entries mixedly in whole nat table > will suffer: > - fragment nat table soon due to

Re: [PATCH 3/5] f2fs: avoid stucking GC due to atomic write

2018-04-19 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > On 2018/4/20 11:12, Jaegeuk Kim wrote: > > On 04/18, Chao Yu wrote: > >> f2fs doesn't allow abuse on atomic write class interface, so except > >> limiting in-mem pages' total memory usage capacity, we need to limit > >> atomic

Re: [PATCH 5/5] f2fs: fix to avoid race during access gc_thread pointer

2018-04-19 Thread Jaegeuk Kim
On 04/18, Chao Yu wrote: > Thread A Thread BThread C > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store > - issue_discard_thread >sbi->gc_thread = NULL; >

Re: [PATCH 4/5] f2fs: show GC failure info in debugfs

2018-04-19 Thread Jaegeuk Kim
On 04/18, Chao Yu wrote: > This patch adds to show GC failure information in debugfs, now it just > shows count of failure caused by atomic write. > > Signed-off-by: Chao Yu > --- > fs/f2fs/debug.c | 5 + > fs/f2fs/f2fs.h | 1 + > fs/f2fs/gc.c| 13 +++-- >

Re: [PATCH 3/5] f2fs: avoid stucking GC due to atomic write

2018-04-19 Thread Jaegeuk Kim
On 04/18, Chao Yu wrote: > f2fs doesn't allow abuse on atomic write class interface, so except > limiting in-mem pages' total memory usage capacity, we need to limit > atomic-write usage as well when filesystem is seriously fragmented, > otherwise we may run into infinite loop during foreground GC

Re: [PATCH v2] ext4: handle errors on ext4_commit_super

2018-04-19 Thread Jaegeuk Kim
block 0, lost sync page write EXT4-fs (dm-1): re-mounted. Opts: (null) Buffer I/O error on dev dm-1, logical block 80, lost async page write Cc: "Theodore Ts'o" <ty...@mit.edu> Cc: Andreas Dilger <adilger.ker...@dilger.ca> Cc: linux-e...@vger.kernel.org Cc: Jaegeuk Kim <j

Re: [PATCH] ext4: handle errors on ext4_commit_super

2018-04-19 Thread Jaegeuk Kim
Hi Jan, Thank you for the review. On 04/19, Jan Kara wrote: > On Thu 29-03-18 18:44:29, Jaegeuk Kim wrote: > > From: Jaegeuk Kim <jaeg...@google.com> > > > > When remounting ext4 from ro to rw, currently it allows its transition, > > even if ext4_commit_supe

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-17 Thread Jaegeuk Kim
On 04/17, Chao Yu wrote: > On 2018/4/17 14:44, Chao Yu wrote: > > On 2018/4/17 4:16, Jaegeuk Kim wrote: > >> On 04/13, Chao Yu wrote: > >>> On 2018/4/13 12:05, Jaegeuk Kim wrote: > >>>> On 04/13, Chao Yu wrote: > >>>>> On 2018

Re: [PATCH 2/2] f2fs: support {d,id,did,x}node checksum

2018-04-16 Thread Jaegeuk Kim
> Hi Jaegeuk, > >> > >> On 2018/2/10 10:52, Chao Yu wrote: > >>> On 2018/2/10 9:41, Jaegeuk Kim wrote: > >>>> On 02/01, Chao Yu wrote: > >>>>> > >>>>> > >>>>> On 2018/2/1 6:15, Jaegeuk Kim wrote:

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-16 Thread Jaegeuk Kim
On 04/13, Chao Yu wrote: > On 2018/4/13 12:05, Jaegeuk Kim wrote: > > On 04/13, Chao Yu wrote: > >> On 2018/4/13 9:04, Jaegeuk Kim wrote: > >>> On 04/10, Chao Yu wrote: > >>>> Hi Jaegeuk, > >>>> > >>>> On 2018/4/8 1

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-12 Thread Jaegeuk Kim
On 04/13, Chao Yu wrote: > On 2018/4/13 9:06, Jaegeuk Kim wrote: > > On 04/10, Chao Yu wrote: > >> On 2018/4/10 12:10, Jaegeuk Kim wrote: > >>> On 04/10, Chao Yu wrote: > >>>> On 2018/4/10 2:02, Jaegeuk Kim wrote: > >>>>> On 04/0

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-12 Thread Jaegeuk Kim
On 04/13, Chao Yu wrote: > On 2018/4/13 9:04, Jaegeuk Kim wrote: > > On 04/10, Chao Yu wrote: > >> Hi Jaegeuk, > >> > >> On 2018/4/8 16:13, Chao Yu wrote: > >>> f2fs doesn't allow abuse on atomic write class interface, so except > >>>

[PATCH] f2fs: clear PageError on writepage

2018-04-12 Thread Jaegeuk Kim
This patch clears PageError in some pages tagged by read path, but when we write the pages with valid contents, writepage should clear the bit likewise ext4. Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/data.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/da

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-12 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > On 2018/4/10 12:10, Jaegeuk Kim wrote: > > On 04/10, Chao Yu wrote: > >> On 2018/4/10 2:02, Jaegeuk Kim wrote: > >>> On 04/08, Chao Yu wrote: > >>>> On 2018/4/5 11:51, Jaegeuk Kim wrote: > >>>>> On 04/04,

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-12 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/4/8 16:13, Chao Yu wrote: > > f2fs doesn't allow abuse on atomic write class interface, so except > > limiting in-mem pages' total memory usage capacity, we need to limit > > start-commit time as well, otherwise we may run into infinite loop > >

Re: [PATCH 2/2] page cache: Mask off unwanted GFP flags

2018-04-10 Thread Jaegeuk Kim
On 04/10, Matthew Wilcox wrote: > On Tue, Apr 10, 2018 at 10:45:45PM +0900, Minchan Kim wrote: > > On Tue, Apr 10, 2018 at 05:53:51AM -0700, Matthew Wilcox wrote: > > > From: Matthew Wilcox > > > > > > The page cache has used the mapping's GFP flags for allocating > > >

Re: [f2fs-dev] [PATCH v3] f2fs: don't use GFP_ZERO for page caches

2018-04-10 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > On 2018/4/10 11:29, Jaegeuk Kim wrote: > > Change log from v2: > > - consider IO error case when dealing with metapage > > - memset by fill_node_footer > > > > Change log from v1: > > - don't memset for recovered page > >

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-09 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > On 2018/4/10 2:02, Jaegeuk Kim wrote: > > On 04/08, Chao Yu wrote: > >> On 2018/4/5 11:51, Jaegeuk Kim wrote: > >>> On 04/04, Chao Yu wrote: > >>>> This patch enlarges block plug coverage in __issue_discard_cmd, in > >&g

Re: [PATCH] f2fs: don't use GFP_ZERO for page caches

2018-04-09 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > On 2018/4/10 3:00, Jaegeuk Kim wrote: > > From: Chao Yu <yuch...@huawei.com> > > > > Related to https://lkml.org/lkml/2018/4/8/661 > > > > Sometimes, we need to write meta data to new allocated block address, > > then we

Re: [f2fs-dev] [PATCH v3] f2fs: don't use GFP_ZERO for page caches

2018-04-09 Thread Jaegeuk Kim
__GFP_ZERO, and do initialization by ourselves to avoid unneeded/redundant zeroing from mm. Cc: <sta...@vger.kernel.org> Signed-off-by: Chao Yu <yuch...@huawei.com> Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/checkpoint.c | 4 +++- fs/f2fs/inode.c | 4 ++--

Re: [f2fs-dev] [PATCH v2] f2fs: don't use GFP_ZERO for page caches

2018-04-09 Thread Jaegeuk Kim
On 04/09, Jaegeuk Kim wrote: > Change log from v1: > - don't memset for recovered page > > Related to https://lkml.org/lkml/2018/4/8/661 > > Sometimes, we need to write meta data to new allocated block address, > then we will allocate a zeroed page in inner inode's addr

Re: [PATCH v2] f2fs: don't use GFP_ZERO for page caches

2018-04-09 Thread Jaegeuk Kim
sta...@vger.kernel.org> Signed-off-by: Chao Yu <yuch...@huawei.com> Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/inode.c | 4 ++-- fs/f2fs/node.c | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c index 417c9dcd0269..8

Re: [PATCH] f2fs: don't use GFP_ZERO for page caches

2018-04-09 Thread Jaegeuk Kim
Chao, I have to test this for a while. Meanwhile, could you take a look at this? On 04/09, Jaegeuk Kim wrote: > From: Chao Yu <yuch...@huawei.com> > > Related to https://lkml.org/lkml/2018/4/8/661 > > Sometimes, we need to write meta data to new allocated block address, &g

[PATCH] f2fs: don't use GFP_ZERO for page caches

2018-04-09 Thread Jaegeuk Kim
ernel.org> Signed-off-by: Chao Yu <yuch...@huawei.com> Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/inode.c| 4 ++-- fs/f2fs/node.c | 6 -- fs/f2fs/node.h | 7 ++- fs/f2fs/recovery.c | 3 +-- 4 files changed, 9 insertions(+), 11 deletions(-) diff --gi

Re: [PATCH] mm: workingset: fix NULL ptr dereference

2018-04-09 Thread Jaegeuk Kim
On 04/09, Minchan Kim wrote: > On Mon, Apr 09, 2018 at 04:14:03AM -0700, Matthew Wilcox wrote: > > On Mon, Apr 09, 2018 at 12:09:30PM +0900, Minchan Kim wrote: > > > On Sun, Apr 08, 2018 at 07:49:25PM -0700, Matthew Wilcox wrote: > > > > On Mon, Apr 09, 2018 at 10:58:15AM +0900, Minchan Kim wrote:

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-09 Thread Jaegeuk Kim
On 04/08, Chao Yu wrote: > On 2018/4/5 11:51, Jaegeuk Kim wrote: > > On 04/04, Chao Yu wrote: > >> This patch enlarges block plug coverage in __issue_discard_cmd, in > >> order to collect more pending bios before issuing them, to avoid > >> being disturbed

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-04 Thread Jaegeuk Kim
On 04/04, Chao Yu wrote: > This patch enlarges block plug coverage in __issue_discard_cmd, in > order to collect more pending bios before issuing them, to avoid > being disturbed by previous discard I/O in IO aware discard mode. Hmm, then we need to wait for huge discard IO for over 10 secs,

Re: [PATCH] f2fs: fix to show encrypt flag in FS_IOC_GETFLAGS

2018-04-04 Thread Jaegeuk Kim
On 04/03, Chao Yu wrote: > On 2018/4/3 4:21, Jaegeuk Kim wrote: > > On 04/02, Chao Yu wrote: > >> This patch fixes to show encrypt flag in FS_IOC_GETFLAGS like ext4 does. > > > > Actually, we have to show internal flags owned by f2fs, not generic ones. > &g

[GIT PULL] f2fs update for 4.17-rc1

2018-04-04 Thread Jaegeuk Kim
ion f2fs: add nowait aio support Jaegeuk Kim (11): f2fs: handle quota for orphan inodes f2fs: don't stop GC if GC is contended f2fs: add mount option for segment allocation policy f2fs: add auto tuning for small devices f2fs: set readdir_ra by default f2fs: is

Re: [PATCH v2] f2fs: remain written times to update inode during fsync

2018-04-03 Thread Jaegeuk Kim
On 04/03, Chao Yu wrote: > On 2018/4/3 13:23, Jaegeuk Kim wrote: > > On 04/03, Chao Yu wrote: > >> On 2018/3/31 0:30, Jaegeuk Kim wrote: > >>> Change log from v1: > >>> - add more description > >>> > >>> This fixes xfstests/

Re: [PATCH v2] f2fs: remain written times to update inode during fsync

2018-04-02 Thread Jaegeuk Kim
On 04/03, Chao Yu wrote: > On 2018/3/31 0:30, Jaegeuk Kim wrote: > > Change log from v1: > > - add more description > > > > This fixes xfstests/generic/392. > > > > The failure was caused by different times between 1) one marked in the last > >

Re: [f2fs-dev] [PATCH v3] f2fs: remain written times to update inode during fsync

2018-04-02 Thread Jaegeuk Kim
was that we skipped updating inode block at 1), since its i_size was recoverable along with 4KB-aligned data writes, which was fixed by: "f2fs: fix a wrong condition in f2fs_skip_inode_update" Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/f2fs.h | 15 +++ fs/f

Re: [PATCH] f2fs: fix to show encrypt flag in FS_IOC_GETFLAGS

2018-04-02 Thread Jaegeuk Kim
On 04/02, Chao Yu wrote: > This patch fixes to show encrypt flag in FS_IOC_GETFLAGS like ext4 does. Actually, we have to show internal flags owned by f2fs, not generic ones. We may need to define all of them separately? > > Signed-off-by: Chao Yu > --- > fs/f2fs/file.c | 9

[PATCH] f2fs: truncate preallocated blocks in error case

2018-03-30 Thread Jaegeuk Kim
If write is failed, we must deallocate the blocks that we couldn't write. Cc: sta...@vger.kernel.org Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/file.c | 9 + 1 file changed, 9 insertions(+) diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index 8068b015ece5..f18f62

Re: [PATCH] f2fs: remain written times to update inode during fsync

2018-03-30 Thread Jaegeuk Kim
On 03/30, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/3/30 13:51, Jaegeuk Kim wrote: > > This fixes xfstests/generic/392. > > Hmm... Could you please give more details about this issue and solution in > commit message, since I can catch up the solution only with the co

Re: [PATCH v2] f2fs: remain written times to update inode during fsync

2018-03-30 Thread Jaegeuk Kim
its i_size was recoverable along with 4KB-aligned data writes, which was fixed by: "f2fs: fix a wrong condition in f2fs_skip_inode_update" Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org> --- fs/f2fs/f2fs.h | 15 +++ fs/f2fs/inode.c | 4 2 files changed, 19 ins

  1   2   3   4   5   6   7   8   9   10   >