[PATCH] f2fs: sanity check of xattr entry size

2018-12-26 Thread Jaegeuk Kim
There is a security report where f2fs_getxattr() has a hole to expose wrong memory region when the image is malformed like this. f2fs_getxattr: entry->e_name_len: 4, size: 12288, buffer_size: 16384, len: 4 Cc: Signed-off-by: Jaegeuk Kim --- fs/f2fs/xattr.c | 18 +- 1 f

Re: [PATCH v2] f2fs: rebuild nat_bits during umount

2018-12-26 Thread Jaegeuk Kim
On 12/24, Chao Yu wrote: > On 2018/12/13 16:50, Chao Yu wrote: > > If all free_nat_bitmap are available, we can rebuild nat_bits from > > free_nat_bitmap entirely during umount, let's make another chance > > to reenable nat_bits for image. > > Jaegeuk, > > Could you please have a try with this pa

Re: [PATCH v3] f2fs: add bio cache for IPU

2018-12-26 Thread Jaegeuk Kim
On 12/24, Chao Yu wrote: > Jaegeuk, > > Will kernel still hang with this v3? I'll consider this later, since it blocked local tests before. > > Thanks, > > On 2018/12/19 17:29, Chao Yu wrote: > > SQLite in Wal mode may trigger sequential IPU write in db-wal file, after > > commit d1b3e72d5490

Re: [PATCH v2] f2fs: fix sbi->extent_list corruption issue

2018-12-19 Thread Jaegeuk Kim
On 12/19, Chao Yu wrote: > On 2018/12/19 6:47, Jaegeuk Kim wrote: > > On 12/18, Chao Yu wrote: > >> On 2018/12/14 22:25, Jaegeuk Kim wrote: > >>> On 12/14, Sahitya Tummala wrote: > >>>> On Wed, Dec 12, 2018 at 11:36:08AM +0800, Chao Yu wrote: >

Re: [f2fs-dev] [PATCH v2] f2fs: add bio cache for IPU

2018-12-18 Thread Jaegeuk Kim
On 12/14, Jaegeuk Kim wrote: > On 12/14, Chao Yu wrote: > > Hi Jaegeuk, > > > > I guess ("f2fs: clear PG_writeback if IPU failed") and v2 patch can fix > > hang issue caused by v1 one, could you please have a try with v2? > > Sure. Hang again. I&

Re: [PATCH v2] f2fs: fix missing unlock(sbi->gc_mutex)

2018-12-18 Thread Jaegeuk Kim
This fixes missing unlock call. Cc: Signed-off-by: Jaegeuk Kim --- v1 -> v2: - clean up fs/f2fs/super.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index b79677639108..cec7ef27b389 100644 --- a/fs/f2fs/super.c +++ b/fs/f

Re: [PATCH v2] f2fs: fix sbi->extent_list corruption issue

2018-12-18 Thread Jaegeuk Kim
On 12/18, Chao Yu wrote: > On 2018/12/14 22:25, Jaegeuk Kim wrote: > > On 12/14, Sahitya Tummala wrote: > >> On Wed, Dec 12, 2018 at 11:36:08AM +0800, Chao Yu wrote: > >>> On 2018/12/12 11:17, Sahitya Tummala wrote: > >>>> On Fri, Dec 07, 2018 at 05:47:

Re: [f2fs-dev] [PATCH 3/3] f2fs: flush stale issued discard candidates

2018-12-18 Thread Jaegeuk Kim
On 12/18, Chao Yu wrote: > On 2018/12/14 13:01, Jaegeuk Kim wrote: > > Sometimes, I could observe # of issuing_discard to be 1 which blocks > > background > > jobs due to is_idle()=false. > > The only way to get out of it was to trigger gc_urgent. This patch avoids &

Re: [PATCH v3] loop: drop caches if offset or block_size are changed

2018-12-18 Thread Jaegeuk Kim
ffset on the loop file 3) mount is failed due to the cached pages having wrong superblock Cc: Jens Axboe Cc: linux-bl...@vger.kernel.org Reported-by: Martijn Coenen Signed-off-by: Jaegeuk Kim --- v2 -> v3: - avoid to submit IOs on frozen mq Jens, how about this? Thanks, drivers/block/loo

Re: dm: give req_size precisely given by user

2018-12-18 Thread Jaegeuk Kim
On 12/18, Mike Snitzer wrote: > On Tue, Dec 18 2018 at 12:45pm -0500, > Jens Axboe wrote: > > > On 12/18/18 10:25 AM, Jaegeuk Kim wrote: > > > This patch makes dm be aware of io_pages to assign sane req_size for > > > reads. > > > > Reviewed-by:

[PATCH] dm: give req_size precisely given by user

2018-12-18 Thread Jaegeuk Kim
This patch makes dm be aware of io_pages to assign sane req_size for reads. Cc: sta...@vger.kernel.org Cc: Jens Axboe Cc: Alasdair Kergon Cc: Mike Snitzer Cc: dm-de...@redhat.com Fixes: 9491ae4a ("mm: don't cap request size based on read-ahead setting") Signed-off-by: Jaegeuk Ki

[PATCH] f2fs: fix missing unlock(sbi->gc_mutex)

2018-12-17 Thread Jaegeuk Kim
This fixes missing unlock call. Cc: Signed-off-by: Jaegeuk Kim --- fs/f2fs/super.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index b79677639108..2689a2cb56cc 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -1462,6 +1462,9 @@ static int

Re: [PATCH v2] loop: drop caches if offset or block_size are changed

2018-12-17 Thread Jaegeuk Kim
ffset on the loop file 3) mount is failed due to the cached pages having wrong superblock Cc: Jens Axboe Cc: linux-bl...@vger.kernel.org Reported-by: Martijn Coenen Signed-off-by: Jaegeuk Kim --- v1 to v2: - cover block_size change drivers/block/loop.c | 15 +++ 1 file change

[PATCH] loop: drop caches if offset is changed

2018-12-14 Thread Jaegeuk Kim
ailed due to the cached pages having wrong superblock This patch drops caches when we change lo->offset. Cc: Jens Axboe Cc: linux-bl...@vger.kernel.org Reported-by: Martijn Coenen Signed-off-by: Jaegeuk Kim --- drivers/block/loop.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a

Re: [PATCH v2] f2fs: add bio cache for IPU

2018-12-14 Thread Jaegeuk Kim
On 12/14, Chao Yu wrote: > Hi Jaegeuk, > > I guess ("f2fs: clear PG_writeback if IPU failed") and v2 patch can fix > hang issue caused by v1 one, could you please have a try with v2? Sure. > > Thanks, > > On 2018/11/19 11:29, Chao Yu wrote: > > SQLite in Wal mode may trigger sequential IPU wri

Re: [PATCH v2] f2fs: fix sbi->extent_list corruption issue

2018-12-14 Thread Jaegeuk Kim
On 12/14, Sahitya Tummala wrote: > On Wed, Dec 12, 2018 at 11:36:08AM +0800, Chao Yu wrote: > > On 2018/12/12 11:17, Sahitya Tummala wrote: > > > On Fri, Dec 07, 2018 at 05:47:31PM +0800, Chao Yu wrote: > > >> On 2018/12/1 4:33, Jaegeuk Kim wrote: > >

[PATCH 1/3] f2fs: use kvmalloc, if kmalloc is failed

2018-12-13 Thread Jaegeuk Kim
) from [] (vfs_kern_mount+0x78/0x134) (vfs_kern_mount) from [] (do_mount+0x474/0xca4) (do_mount) from [] (SyS_mount+0x94/0xbc) (SyS_mount) from [] (ret_fast_syscall+0x0/0x48) Signed-off-by: Jaegeuk Kim --- fs/f2fs/acl.c| 6 ++-- fs/f2fs/checkpoint.c | 2 +- fs/f2fs/data.c | 2

[PATCH 3/3] f2fs: flush stale issued discard candidates

2018-12-13 Thread Jaegeuk Kim
Sometimes, I could observe # of issuing_discard to be 1 which blocks background jobs due to is_idle()=false. The only way to get out of it was to trigger gc_urgent. This patch avoids that by checking any candidates as done in the list. Signed-off-by: Jaegeuk Kim --- fs/f2fs/segment.c | 4

[PATCH 2/3] f2fs: correct wrong spelling, issing_*

2018-12-13 Thread Jaegeuk Kim
Let's use "queued" instead of "issuing". Signed-off-by: Jaegeuk Kim --- fs/f2fs/debug.c | 4 ++-- fs/f2fs/f2fs.h| 10 +- fs/f2fs/segment.c | 26 +- 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/fs/f2fs/debug.

Re: [f2fs-dev] [PATCH] f2fs: add an ioctl() to explicitly trigger fsck later

2018-12-12 Thread Jaegeuk Kim
On 12/07, Chao Yu wrote: > On 2018/12/1 4:36, Jaegeuk Kim wrote: > > On 11/30, Chao Yu wrote: > >> On 2018/11/29 9:52, Jaegeuk Kim wrote: > >>> This adds an option in ioctl(F2FS_IOC_SHUTDOWN) in order to trigger fsck > >>> by > >>> setting a

Re: [PATCH v2] f2fs: remove codes of unused wio_mutex

2018-12-12 Thread Jaegeuk Kim
On 12/12, Yunlong Song wrote: > v1->v2: delete comments in f2fs.h: "/* bio ordering for NODE/DATA */" It's quite late for me to rebase the repo. Could you please send another patch for this? > > Signed-off-by: Yunlong Song > Reviewed-by: Chao Yu > Signed-off-

Re: [f2fs-dev] [PATCH] f2fs: avoid frequent costly fsck triggers

2018-11-30 Thread Jaegeuk Kim
On 11/30, Chao Yu wrote: > On 2018/11/30 10:35, Sheng Yong wrote: > > Hi, Jaegeuk and Chao, > > > > On 2018/11/29 1:48, Jaegeuk Kim wrote: > >> On 11/28, Chao Yu wrote: > >>> On 2018/11/28 16:10, Jaegeuk Kim wrote: > >>>> On 11/28, Chao Yu

Re: [f2fs-dev] [PATCH] f2fs: check memory boundary by insane namelen

2018-11-26 Thread Jaegeuk Kim
On 11/23, Sheng Yong wrote: > Hi, Jaegeuk and Chao, > > On 2018/11/15 15:50, Jaegeuk Kim wrote: > > If namelen is corrupted to have very long value, fill_dentries can copy > > wrong memory area. > > > Is there any scenario that could hit this corruption? Or

Re: linux-next: build warning after merge of the f2fs tree

2018-11-26 Thread Jaegeuk Kim
On 11/27, Stephen Rothwell wrote: > Hi Jaegeuk, > > On Mon, 26 Nov 2018 13:59:24 -0800 Jaegeuk Kim wrote: > > > > On 11/26, Stephen Rothwell wrote: > > > > > > After merging the f2fs tree, today's linux-next build > > > (x86_64_allmodconfig)

Re: linux-next: build warning after merge of the f2fs tree

2018-11-26 Thread Jaegeuk Kim
Hi Stephen, On 11/26, Stephen Rothwell wrote: > Hi Jaegeuk, > > After merging the f2fs tree, today's linux-next build > (x86_64_allmodconfig) produced this warning: > > In file included from fs/f2fs/dir.c:11: > fs/f2fs/f2fs.h: In function '__mark_inode_dirty_flag': > fs/f2fs/f2fs.h:2388:6: warni

Re: linux-next: build failure after merge of the vfs tree

2018-10-16 Thread Jaegeuk Kim
On 10/16, Stephen Rothwell wrote: > Hi all, > > On Wed, 3 Oct 2018 10:32:22 +1000 Stephen Rothwell > wrote: > > > > After merging the vfs tree, today's linux-next build (x86_64 allmodconfig) > > failed like this: > > > > /home/sfr/next/next/fs/f2fs/super.c: In function 'f2fs_remount': > > /home

Re: [f2fs-dev] [PATCH] f2fs: fix quota info to adjust recovered data

2018-09-19 Thread Jaegeuk Kim
On 09/19, Chao Yu wrote: > On 2018/9/19 0:45, Jaegeuk Kim wrote: > > On 09/18, Chao Yu wrote: > >> On 2018/9/18 10:05, Jaegeuk Kim wrote: > >>> On 09/18, Chao Yu wrote: > >>>> On 2018/9/18 9:19, Jaegeuk Kim wrote: > >>>>> On 09/1

Re: [PATCH v6] f2fs: guarantee journalled quota data by checkpoint

2018-09-07 Thread Jaegeuk Kim
Chao, I was testing the previous patch and removed in the queue due to quota-related hang during fault injection + shutdown test. Let me try this later. Thanks, On 09/07, Chao Yu wrote: > For journalled quota mode, let checkpoint to flush dquot dirty data > and quota file data to guarntee persis

[PATCH] f2fs: submit bio after shutdown

2018-09-07 Thread Jaegeuk Kim
Sometimes, some merged IOs could get a chance to be submitted, resulting in system hang in shutdown test. This issues IOs all the time after shutdown. Signed-off-by: Jaegeuk Kim --- fs/f2fs/data.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index

Re: [f2fs-dev] [PATCH v3] f2fs: fix performance issue observed with multi-thread sequential read

2018-08-14 Thread Jaegeuk Kim
On 08/14, Chao Yu wrote: > On 2018/8/14 12:04, Jaegeuk Kim wrote: > > On 08/14, Chao Yu wrote: > >> On 2018/8/14 4:11, Jaegeuk Kim wrote: > >>> On 08/13, Chao Yu wrote: > >>>> Hi Jaegeuk, > >>>> > >>>> On 2018/8/11 2:56, J

Re: [PATCH 2/2] f2fs: tune discard speed with storage usage rate

2018-08-13 Thread Jaegeuk Kim
On 08/10, Chao Yu wrote: > Previously, discard speed was fixed mostly, and in high usage rate > device, we will speed up issuing discard, but it doesn't make sense > that in a non-full filesystem, we still issue discard with slow speed. Could you please elaborate the problem in more detail? The sp

Re: [PATCH v6 1/2] f2fs: fix to avoid broken of dnode block list

2018-08-01 Thread Jaegeuk Kim
On 07/29, Chao Yu wrote: > From: Chao Yu > > f2fs recovery flow is relying on dnode block link list, it means fsynced > file recovery depends on previous dnode's persistence in the list, so > during fsync() we should wait on all regular inode's dnode writebacked > before issuing flush. > > By th

Re: [PATCH] f2fs: fix invalid memory access

2018-08-01 Thread Jaegeuk Kim
On 08/01, Chao Yu wrote: > From: Chao Yu > > syzbot found the following crash on: > > HEAD commit:d9bd94c0bcaa Add linux-next specific files for 20180801 > git tree: linux-next > console output: https://syzkaller.appspot.com/x/log.txt?x=1001189c40 > kernel config: https://syzkalle

[PATCH] f2fs: avoid f2fs_bug_on() in cp_error case

2018-08-01 Thread Jaegeuk Kim
There is a subtle race condition to invoke f2fs_bug_on() in shutdown tests. I've confirmed that the last checkpoint is preserved in consistent state, so it'd be fine to just return error at this moment. Signed-off-by: Jaegeuk Kim --- fs/f2fs/node.c | 4 1 file changed, 4

[PATCH] f2fs: avoid fi->i_gc_rwsem[WRITE] lock in f2fs_gc

2018-07-24 Thread Jaegeuk Kim
The f2fs_gc() called by f2fs_balance_fs() requires to be called outside of fi->i_gc_rwsem[WRITE], since f2fs_gc() can try to grab it in a loop. Signed-off-by: Jaegeuk Kim --- fs/f2fs/file.c| 2 ++ fs/f2fs/segment.c | 1 - 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/f

Re: [PATCH v2] f2fs: split discard command in prior to block layer

2018-07-23 Thread Jaegeuk Kim
On 07/15, Chao Yu wrote: > On 2018/7/15 11:13, Jaegeuk Kim wrote: > > On 07/15, Chao Yu wrote: > >> Hi Jaegeuk, > >> > >> On 2018/7/15 9:27, Jaegeuk Kim wrote: > >>> On 07/08, Chao Yu wrote: > >>>> From: Chao Yu > >&

Re: [PATCH 1/1] f2fs: checkpoint disabling

2018-07-14 Thread Jaegeuk Kim
On 07/11, Daniel Rosenberg wrote: > This adds a lightweight non-persistent snapshotting scheme to f2fs. > > To use, mount with the option checkpoint=disable, and to return to > normal operation, remount with checkpoint=enable. If the filesystem > is shut down before remounting with checkpoint=enab

Re: [PATCH] f2fs: split discard command in prior to block layer

2018-07-06 Thread Jaegeuk Kim
On 07/07, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/7/7 6:45, Jaegeuk Kim wrote: > > On 07/04, Chao Yu wrote: > >> From: Chao Yu > >> > >> Some devices has small max_{hw,}discard_sectors, so that in > >> __blkdev_issue_discard(), one big size dis

[PATCH 2/4] f2fs: allow wrong configure dio to buffered write

2018-07-06 Thread Jaegeuk Kim
This fixes to support unaligned dio as buffered writes. Signed-off-by: Jaegeuk Kim --- fs/f2fs/data.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index e66379961804..6e8e78bb64a7 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -2425,7

[GIT PULL] f2fs update for 4.18-rc1

2018-06-11 Thread Jaegeuk Kim
ead path to allow multiple postprocessing steps f2fs: call unlock_new_inode() before d_instantiate() f2fs: fix features filename in sysfs documentation Jaegeuk Kim (16): Revert "f2fs: introduce f2fs_set_page_dirty_nobuffer" f2fs: check cap_resource only for da

Re: [PATCH 3/3] f2fs: clean up symbol namespace

2018-05-29 Thread Jaegeuk Kim
Hi Chao, Thank you for the work. I resolved some conflicts and modified some function names. Please take a look at this. https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/commit/?h=dev-test --- fs/f2fs/checkpoint.c | 8 fs/f2fs/dir.c| 6 +++--- fs/f2fs/f2fs.h

[RFC PATCH] f2fs: add fsync_mode=nobarrier for non-atomic files

2018-05-25 Thread Jaegeuk Kim
For non-atomic files, this patch adds an option to give nobarrier which doesn't issue flush commands to the device. Signed-off-by: Jaegeuk Kim --- Documentation/filesystems/f2fs.txt | 16 +--- fs/f2fs/f2fs.h | 1 + fs/f2fs/file.c | 2 +

Re: [PATCH v2] f2fs: let fstrim issue discard commands in lower priority

2018-05-25 Thread Jaegeuk Kim
ximum size to 2MB per candidate, and check IO congestion when issuing them to disk. Signed-off-by: Jaegeuk Kim --- Change log from v1: - wait all discard bios in put_super & congested case in trimfs fs/f2fs/f2fs.h| 4 +- fs/f2fs/segment.c | 139 +---

Re: [PATCH] f2fs: let fstrim issue discard commands in lower priority

2018-05-24 Thread Jaegeuk Kim
On 05/25, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/5/25 13:10, Jaegeuk Kim wrote: > > The fstrim gathers huge number of large discard commands, and tries to issue > > without IO awareness, which results in long user-perceive IO latencies on > > READ, WRITE, and FLUSH

[PATCH] f2fs: let fstrim issue discard commands in lower priority

2018-05-24 Thread Jaegeuk Kim
ximum size to 2MB per candidate, and check IO congestion when issuing them to disk. Signed-off-by: Jaegeuk Kim --- fs/f2fs/f2fs.h| 4 +- fs/f2fs/segment.c | 123 +++--- 2 files changed, 64 insertions(+), 63 deletions(-) diff --git a/fs/f2fs/f2fs.h b/fs

Re: [PATCH v2] f2fs: Fix deadlock in shutdown ioctl

2018-05-17 Thread Jaegeuk Kim
On 05/17, Chao Yu wrote: > On 2018/5/17 16:03, Sahitya Tummala wrote: > > f2fs_ioc_shutdown() ioctl gets stuck in the below path > > when issued with F2FS_GOING_DOWN_FULLSYNC option. > > > > __switch_to+0x90/0xc4 > > percpu_down_write+0x8c/0xc0 > > freeze_super+0xec/0x1e4 > > freeze_bdev+0xc4/0xcc

Re: [PATCH] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Jaegeuk Kim
On 05/08, Chao Yu wrote: > Thread A Thread B > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store >sbi->gc_thread = NULL; > access sbi->gc_thread->gc_* > > Previously, we allocate memory for sbi->gc_thread ba

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-07 Thread Jaegeuk Kim
On 05/08, Chao Yu wrote: > On 2018/5/8 4:46, Jaegeuk Kim wrote: > > On 04/27, Chao Yu wrote: > >> On 2018/4/27 0:36, Jaegeuk Kim wrote: > >>> On 04/26, Chao Yu wrote: > >>>> On 2018/4/26 23:48, Jaegeuk Kim wrote: >

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Jaegeuk Kim
On 05/08, Chao Yu wrote: > On 2018/5/8 5:36, Jaegeuk Kim wrote: > > On 05/07, Chao Yu wrote: > >> Thread A Thread BThread C > >> - f2fs_remount > >> - stop_gc_thread > >>

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Jaegeuk Kim
t; Signed-off-by: Chao Yu > --- > v4: > - use introduced sbi.gc_rwsem lock instead of sb.s_umount. We can use this first. >From e62e8d3ece6ee8a4aeac8ffd6161d25851f8b3f0 Mon Sep 17 00:00:00 2001 From: Jaegeuk Kim Date: Mon, 7 May 2018 14:22:40 -0700 Subject: [PATCH] f2fs: introdu

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-07 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/27 0:36, Jaegeuk Kim wrote: > > On 04/26, Chao Yu wrote: > >> On 2018/4/26 23:48, Jaegeuk Kim wrote: > >>> On 04/26, Chao Yu wrote: > >>>> Thread A Thread B > >>>>

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-05-04 Thread Jaegeuk Kim
On 05/02, Chao Yu wrote: > On 2018/4/28 10:34, Jaegeuk Kim wrote: > > On 04/27, Chao Yu wrote: > >> On 2018/4/26 23:54, Jaegeuk Kim wrote: > >>> On 04/24, Chao Yu wrote: > >>>> f2fs doesn't allow abuse on atomic write class interface, so excep

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-05-04 Thread Jaegeuk Kim
On 05/02, Chao Yu wrote: > On 2018/4/28 10:36, Jaegeuk Kim wrote: > > On 04/27, Chao Yu wrote: > >> On 2018/4/27 0:03, Jaegeuk Kim wrote: > >>> On 04/24, Chao Yu wrote: > >>>> Thread A Thread BThread C &

[PATCH] f2fs: avoid fsync() failure caused by EAGAIN in writepage()

2018-05-04 Thread Jaegeuk Kim
pageout() in MM traslates EAGAIN, so calls handle_write_error() -> mapping_set_error() -> set_bit(AS_EIO, ...). file_write_and_wait_range() will see EIO error, which is critical to return value of fsync() followed by atomic_write failure to user. Signed-off-by: Jaegeuk Kim --- fs/f2fs/

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-27 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/27 10:04, Chao Yu wrote: > > On 2018/4/27 0:03, Jaegeuk Kim wrote: > >> On 04/24, Chao Yu wrote: > >>> Thread A Thread BThread C > >>> - f2fs_remount > >>> - stop_gc_t

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-27 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/27 0:03, Jaegeuk Kim wrote: > > On 04/24, Chao Yu wrote: > >> Thread A Thread BThread C > >> - f2fs_remount > >> - stop_gc_thread > >>

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-04-27 Thread Jaegeuk Kim
On 04/27, Chao Yu wrote: > On 2018/4/26 23:54, Jaegeuk Kim wrote: > > On 04/24, Chao Yu wrote: > >> f2fs doesn't allow abuse on atomic write class interface, so except > >> limiting in-mem pages' total memory usage capacity, we need to limit > >>

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Jaegeuk Kim
On 04/26, Chao Yu wrote: > On 2018/4/26 23:48, Jaegeuk Kim wrote: > > On 04/26, Chao Yu wrote: > >> Thread A Thread B > >> - f2fs_ioc_commit_atomic_write > >> - commit_inmem_pages > >> - f2fs

Re: [PATCH 3/3] f2fs: maintain discard interface separately

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > This patch adds a new file discard.c to maintain discard related > function separately. I don't think we need this at all. Thanks, > > BTW, fix below checkpatch errors: > > ERROR: space required before the open brace '{' > + } else if (issued == -1){ > >

Re: [PATCH 2/3] f2fs: introduce {create,destroy}_discard_caches for cleanup

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > Split discard slab cache related initial/release codes into separated > function {create,destroy}_discard_caches, later we can maintain those > independent functions in separated discard.c No need this as well. > > Signed-off-by: Chao Yu > --- > fs/f2fs/f2fs.h|

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-26 Thread Jaegeuk Kim
On 04/24, Chao Yu wrote: > Thread A Thread BThread C > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store > - issue_discard_thread >sbi->gc_thread = NULL; >

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-04-26 Thread Jaegeuk Kim
On 04/24, Chao Yu wrote: > f2fs doesn't allow abuse on atomic write class interface, so except > limiting in-mem pages' total memory usage capacity, we need to limit > atomic-write usage as well when filesystem is seriously fragmented, > otherwise we may run into infinite loop during foreground GC

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Jaegeuk Kim
On 04/26, Chao Yu wrote: > Thread A Thread B > - f2fs_ioc_commit_atomic_write > - commit_inmem_pages > - f2fs_submit_merged_write_cond > : write data > - write_checkpoint >- do_checkpoint

Re: [f2fs-dev] [PATCH 5/5] f2fs: enforce fsync_mode=strict for renamed directory

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > On 2018/4/25 13:46, Jaegeuk Kim wrote: > > This is to give a option for user to be able to recover B/foo in the below > > case. > > > > mkdir A > > sync() > > rename(A, B) > > creat (B/foo) > > fsync (B/foo) > >

Re: [f2fs-dev] [PATCH 4/5 v2] f2fs: sanity check for total valid blocks

2018-04-26 Thread Jaegeuk Kim
-tested-by: syzbot+0a725420475916460...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- Change log from v1: - check valid node count fs/f2fs/segment.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 20250b88bf51..c60f87822

Re: [f2fs-dev] [PATCH 4/5] f2fs: sanity check for total valid blocks

2018-04-26 Thread Jaegeuk Kim
e_count, which works for syzbot case as well. > > Thanks, > > On 2018/4/25 13:46, Jaegeuk Kim wrote: > > This patch enhances sanity check for SIT entries. > > > > syzbot hit the following crash on upstream commit > > 83beed7b2b26f232d782127792dd0cd4362fdc41 (Fri

Re: [f2fs-dev] [PATCH 1/5] f2fs: give message and set need_fsck given broken node id

2018-04-26 Thread Jaegeuk Kim
On 04/25, Chao Yu wrote: > On 2018/4/25 13:46, Jaegeuk Kim wrote: > > syzbot hit the following crash on upstream commit > > 83beed7b2b26f232d782127792dd0cd4362fdc41 (Fri Apr 20 17:56:32 2018 +) > > Merge branch 'fixes' of > > git://git.kernel.org/pub/sc

Re: [PATCH] f2fs: clear page_error for all the writebacking pages

2018-04-26 Thread Jaegeuk Kim
On 04/24, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/4/24 6:49, Jaegeuk Kim wrote: > > This patch clear page_error bit, if the page is going to be writebacked. > > This patch is similar to previous patch ("f2fs: clear PageError on > writepage"), > only covera

[PATCH 2/5] f2fs: avoid bug_on on corrupted inode

2018-04-24 Thread Jaegeuk Kim
c:238 [inline] RSP: 8801c44a70e8 RIP: f2fs_iget+0x3307/0x3ca0 fs/f2fs/inode.c:313 RSP: 8801c44a70e8 invalid opcode: [#2] SMP KASAN ---[ end trace 1cbcbec2156680bc ]--- Reported-and-tested-by: syzbot+41a1b341571f0952b...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- fs/f2fs/in

[PATCH 4/5] f2fs: sanity check for total valid blocks

2018-04-24 Thread Jaegeuk Kim
-tested-by: syzbot+0a725420475916460...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- fs/f2fs/segment.c | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 20250b88bf51..a55647f61232 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs

[PATCH 5/5] f2fs: enforce fsync_mode=strict for renamed directory

2018-04-24 Thread Jaegeuk Kim
This is to give a option for user to be able to recover B/foo in the below case. mkdir A sync() rename(A, B) creat (B/foo) fsync (B/foo) ---crash--- Sugessted-by: Velayudhan Pillai Signed-off-by: Jaegeuk Kim --- fs/f2fs/namei.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff

[PATCH 3/5] f2fs: sanity check on sit entry

2018-04-24 Thread Jaegeuk Kim
- Reported-and-tested-by: syzbot+83699adeb2d13579c...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- fs/f2fs/segment.c | 9 + 1 file changed, 9 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index e4e8bdd645ee..20250b88bf51 100644 --- a/fs/f2fs/segment.c +++ b/fs

[PATCH 1/5] f2fs: give message and set need_fsck given broken node id

2018-04-24 Thread Jaegeuk Kim
000 RIP: __get_node_page+0xb68/0x16e0 fs/f2fs/node.c:1185 RSP: 8801d960e820 ---[ end trace 4edbeb71f002bb76 ]--- Reported-and-tested-by: syzbot+d154ec99402c6f628...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- fs/f2fs/f2fs.h | 13 + fs/f2fs/inode.c | 13 ++-

[PATCH] f2fs: avoid bug_on on corrupted inode

2018-04-24 Thread Jaegeuk Kim
c:238 [inline] RSP: 8801c44a70e8 RIP: f2fs_iget+0x3307/0x3ca0 fs/f2fs/inode.c:313 RSP: 8801c44a70e8 invalid opcode: [#2] SMP KASAN ---[ end trace 1cbcbec2156680bc ]--- Reported-and-tested-by: syzbot+41a1b341571f0952b...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- fs/f2fs/in

Re: kernel BUG at fs/f2fs/node.c:LINE!

2018-04-24 Thread Jaegeuk Kim
On 04/24, Dmitry Vyukov wrote: > On Tue, Apr 24, 2018 at 8:42 AM, syzbot > wrote: > > Hello, > > > > syzbot tried to test the proposed patch but build/boot failed: > > > > failed to create VM pool: failed to create GCE image: failed to get create > > image operation operation-1524552040215-56a926e

Re: kernel BUG at fs/f2fs/node.c:LINE!

2018-04-23 Thread Jaegeuk Kim
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git dev >From 65b966f58e8264477da6c8b70703ce362cb1f8f0 Mon Sep 17 00:00:00 2001 From: Jaegeuk Kim Date: Mon, 23 Apr 2018 23:02:31 -0600 Subject: [PATCH] f2fs: give message and set need_fsck given broken node id syzbot hit

Re: kernel BUG at fs/f2fs/node.c:LINE!

2018-04-23 Thread Jaegeuk Kim
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git dev-test >From 65b966f58e8264477da6c8b70703ce362cb1f8f0 Mon Sep 17 00:00:00 2001 From: Jaegeuk Kim Date: Mon, 23 Apr 2018 23:02:31 -0600 Subject: [PATCH] f2fs: give message and set need_fsck given broken node id syz

[PATCH] f2fs: give message and set need_fsck given broken node id

2018-04-23 Thread Jaegeuk Kim
000 RIP: __get_node_page+0xb68/0x16e0 fs/f2fs/node.c:1185 RSP: 8801d960e820 ---[ end trace 4edbeb71f002bb76 ]--- Reported-by: syzbot+d154ec99402c6f628...@syzkaller.appspotmail.com Signed-off-by: Jaegeuk Kim --- fs/f2fs/f2fs.h | 13 + fs/f2fs/inode.c | 6 +- fs/f2

[PATCH] f2fs: clear page_error for all the writebacking pages

2018-04-23 Thread Jaegeuk Kim
This patch clear page_error bit, if the page is going to be writebacked. Signed-off-by: Jaegeuk Kim --- fs/f2fs/gc.c | 1 + fs/f2fs/inline.c | 1 + fs/f2fs/node.c| 1 + fs/f2fs/segment.c | 1 + 4 files changed, 4 insertions(+) diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index

Re: [PATCH 3/3] f2fs: wait IO writeback in commit_inmem_pages()

2018-04-23 Thread Jaegeuk Kim
On 04/23, Chao Yu wrote: > Thread A Thread B > - f2fs_ioc_commit_atomic_write > - commit_inmem_pages > - f2fs_submit_merged_write_cond > : write data > - write_checkpoint >- do_checkpoint

Re: [PATCH] f2fs: sepearte hot/cold in free nid

2018-04-23 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > On 2018/4/20 12:04, Chao Yu wrote: > > On 2018/4/20 11:37, Jaegeuk Kim wrote: > >> On 04/20, Chao Yu wrote: > >>> As most indirect node, dindirect node, and xattr node won't be updated > >>> after they are created, b

Re: [PATCH v3] ext4: handle errors on ext4_commit_super

2018-04-23 Thread Jaegeuk Kim
block 0, lost sync page write EXT4-fs (dm-1): re-mounted. Opts: (null) Buffer I/O error on dev dm-1, logical block 80, lost async page write Cc: "Theodore Ts'o" Cc: Andreas Dilger Cc: linux-e...@vger.kernel.org Cc: Jaegeuk Kim Signed-off-by: Jaegeuk Kim --- Change log from v2:

[PATCH] f2fs: check cap_resource only for data blocks

2018-04-20 Thread Jaegeuk Kim
This patch changes the rule to check cap_resource for data blocks, not inode or node blocks in order to avoid selinux denial. Signed-off-by: Jaegeuk Kim --- fs/f2fs/f2fs.h | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index

[PATCH] Revert "f2fs: introduce f2fs_set_page_dirty_nobuffer"

2018-04-20 Thread Jaegeuk Kim
This patch reverts copied f2fs_set_page_dirty_nobuffer to use generic function for stability. This reverts commit fe76b796fc5194cc3d57265002e3a748566d073f. Signed-off-by: Jaegeuk Kim --- fs/f2fs/checkpoint.c | 2 +- fs/f2fs/data.c | 33 + fs/f2fs/f2fs.h

Re: [PATCH 5/5] f2fs: fix to avoid race during access gc_thread pointer

2018-04-19 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > On 2018/4/20 11:19, Jaegeuk Kim wrote: > > On 04/18, Chao Yu wrote: > >> Thread A Thread BThread C > >> - f2fs_remount > >> - stop_gc_thread > >>

Re: [PATCH] f2fs: sepearte hot/cold in free nid

2018-04-19 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > As most indirect node, dindirect node, and xattr node won't be updated > after they are created, but inode node and other direct node will change > more frequently, so store their nat entries mixedly in whole nat table > will suffer: > - fragment nat table soon due to dif

Re: [PATCH 3/5] f2fs: avoid stucking GC due to atomic write

2018-04-19 Thread Jaegeuk Kim
On 04/20, Chao Yu wrote: > On 2018/4/20 11:12, Jaegeuk Kim wrote: > > On 04/18, Chao Yu wrote: > >> f2fs doesn't allow abuse on atomic write class interface, so except > >> limiting in-mem pages' total memory usage capacity, we need to limit > >>

Re: [PATCH 5/5] f2fs: fix to avoid race during access gc_thread pointer

2018-04-19 Thread Jaegeuk Kim
On 04/18, Chao Yu wrote: > Thread A Thread BThread C > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store > - issue_discard_thread >sbi->gc_thread = NULL; >

Re: [PATCH 4/5] f2fs: show GC failure info in debugfs

2018-04-19 Thread Jaegeuk Kim
On 04/18, Chao Yu wrote: > This patch adds to show GC failure information in debugfs, now it just > shows count of failure caused by atomic write. > > Signed-off-by: Chao Yu > --- > fs/f2fs/debug.c | 5 + > fs/f2fs/f2fs.h | 1 + > fs/f2fs/gc.c| 13 +++-- > fs/f2fs/gc.h| 2

Re: [PATCH 3/5] f2fs: avoid stucking GC due to atomic write

2018-04-19 Thread Jaegeuk Kim
On 04/18, Chao Yu wrote: > f2fs doesn't allow abuse on atomic write class interface, so except > limiting in-mem pages' total memory usage capacity, we need to limit > atomic-write usage as well when filesystem is seriously fragmented, > otherwise we may run into infinite loop during foreground GC

Re: [PATCH v2] ext4: handle errors on ext4_commit_super

2018-04-19 Thread Jaegeuk Kim
block 0, lost sync page write EXT4-fs (dm-1): re-mounted. Opts: (null) Buffer I/O error on dev dm-1, logical block 80, lost async page write Cc: "Theodore Ts'o" Cc: Andreas Dilger Cc: linux-e...@vger.kernel.org Cc: Jaegeuk Kim Signed-off-by: Jaegeuk Kim --- Change log fro

Re: [PATCH] ext4: handle errors on ext4_commit_super

2018-04-19 Thread Jaegeuk Kim
Hi Jan, Thank you for the review. On 04/19, Jan Kara wrote: > On Thu 29-03-18 18:44:29, Jaegeuk Kim wrote: > > From: Jaegeuk Kim > > > > When remounting ext4 from ro to rw, currently it allows its transition, > > even if ext4_commit_super() returns EIO. Even worse t

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-17 Thread Jaegeuk Kim
On 04/17, Chao Yu wrote: > On 2018/4/17 14:44, Chao Yu wrote: > > On 2018/4/17 4:16, Jaegeuk Kim wrote: > >> On 04/13, Chao Yu wrote: > >>> On 2018/4/13 12:05, Jaegeuk Kim wrote: > >>>> On 04/13, Chao Yu wrote: > >>>>> On 2018

Re: [PATCH 2/2] f2fs: support {d,id,did,x}node checksum

2018-04-16 Thread Jaegeuk Kim
> Hi Jaegeuk, > >> > >> On 2018/2/10 10:52, Chao Yu wrote: > >>> On 2018/2/10 9:41, Jaegeuk Kim wrote: > >>>> On 02/01, Chao Yu wrote: > >>>>> > >>>>> > >>>>> On 2018/2/1 6:15, Jaegeuk Kim wrote:

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-16 Thread Jaegeuk Kim
On 04/13, Chao Yu wrote: > On 2018/4/13 12:05, Jaegeuk Kim wrote: > > On 04/13, Chao Yu wrote: > >> On 2018/4/13 9:04, Jaegeuk Kim wrote: > >>> On 04/10, Chao Yu wrote: > >>>> Hi Jaegeuk, > >>>> > >>>> On 2018/4/8 1

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-12 Thread Jaegeuk Kim
On 04/13, Chao Yu wrote: > On 2018/4/13 9:06, Jaegeuk Kim wrote: > > On 04/10, Chao Yu wrote: > >> On 2018/4/10 12:10, Jaegeuk Kim wrote: > >>> On 04/10, Chao Yu wrote: > >>>> On 2018/4/10 2:02, Jaegeuk Kim wrote: > >>>>> On 04/0

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-12 Thread Jaegeuk Kim
On 04/13, Chao Yu wrote: > On 2018/4/13 9:04, Jaegeuk Kim wrote: > > On 04/10, Chao Yu wrote: > >> Hi Jaegeuk, > >> > >> On 2018/4/8 16:13, Chao Yu wrote: > >>> f2fs doesn't allow abuse on atomic write class interface, so except > >>&g

[PATCH] f2fs: clear PageError on writepage

2018-04-12 Thread Jaegeuk Kim
This patch clears PageError in some pages tagged by read path, but when we write the pages with valid contents, writepage should clear the bit likewise ext4. Signed-off-by: Jaegeuk Kim --- fs/f2fs/data.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index

Re: [PATCH] f2fs: enlarge block plug coverage

2018-04-12 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > On 2018/4/10 12:10, Jaegeuk Kim wrote: > > On 04/10, Chao Yu wrote: > >> On 2018/4/10 2:02, Jaegeuk Kim wrote: > >>> On 04/08, Chao Yu wrote: > >>>> On 2018/4/5 11:51, Jaegeuk Kim wrote: > >>>>> On 04/04, Cha

Re: [PATCH] f2fs: set deadline to drop expired inmem pages

2018-04-12 Thread Jaegeuk Kim
On 04/10, Chao Yu wrote: > Hi Jaegeuk, > > On 2018/4/8 16:13, Chao Yu wrote: > > f2fs doesn't allow abuse on atomic write class interface, so except > > limiting in-mem pages' total memory usage capacity, we need to limit > > start-commit time as well, otherwise we may run into infinite loop > > d

<    1   2   3   4   5   6   7   8   9   10   >