[PATCH 1/4] f2fs: don't keep meta pages used for block migration

2018-07-27 Thread Chao Yu
For migration of encrypted inode's block, we load data of encrypted block into meta inode's page cache, after checkpoint, those all intermediate pages should be clean, and no one will read them again, so let's just release them for more memory. Signed-off-by: Chao Yu --- fs/f2

[PATCH 4/4] f2fs: fix to spread clear_cold_data()

2018-07-27 Thread Chao Yu
We need to drop PG_checked flag on page as well when we clear PG_uptodate flag, in order to avoid treating the page as GCing one later. Signed-off-by: Weichao Guo Signed-off-by: Chao Yu --- fs/f2fs/data.c| 8 +++- fs/f2fs/dir.c | 1 + fs/f2fs/segment.c | 4 +++- 3 files changed, 11

[PATCH 3/4] f2fs: fix avoid race between truncate and background GC

2018-07-27 Thread Chao Yu
- set_cold_data - f2fs_truncate - f2fs_setattr isize to 4k - read 4k <--- hit data in cached page #0 Above race condition can cause read out invalid data in a truncated page, fix it by i_gc_rwsem[WRITE] lock. Signed-off-by: Chao Yu --- fs/f2fs/data.c |

Re: [f2fs-dev] [PATCH] f2fs: avoid fi->i_gc_rwsem[WRITE] lock in f2fs_gc

2018-07-25 Thread Chao Yu
On 2018/7/25 11:22, Jaegeuk Kim wrote: > The f2fs_gc() called by f2fs_balance_fs() requires to be called outside of > fi->i_gc_rwsem[WRITE], since f2fs_gc() can try to grab it in a loop. It seems there are other paths having the same issue, how about fixing all of them in this patch? Thanks, >

Re: [PATCH 2/5] f2fs: add cur_victim_sec for BG_GC to avoid skipping BG_GC victim

2018-07-25 Thread Chao Yu
On 2018/7/24 23:19, Yunlong Song wrote: > > > On 2018/7/24 22:17, Chao Yu wrote: >> On 2018/7/24 21:39, Yunlong Song wrote: >>> >>> On 2018/7/24 21:11, Chao Yu wrote: >>>> On 2018/7/23 22:10, Yunlong Song wrote: >>>>> If f2fs abo

[PATCH] f2fs: fix to restrict mount condition when without CONFIG_QUOTA

2018-07-25 Thread Chao Yu
From: Chao Yu Like quota_ino feature, we need to reject mounting RDWR with image which enables project_quota feature when there is no CONFIG_QUOTA be set in kernel. Signed-off-by: Chao Yu --- fs/f2fs/super.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/f2fs/super.c b/fs/f2fs

Re: [f2fs-dev] [PATCH] f2fs: fix 32-bit format string warning

2018-07-25 Thread Chao Yu
On 2018/7/24 17:34, Arnd Bergmann wrote: > On 32-bit targets, size_t is often 'unsigned int', so printing it as %lu > causes a warning: > > fs/f2fs/inode.c: In function 'sanity_check_inode': > fs/f2fs/inode.c:247:4: error: format '%lu' expects argument of type 'long > unsigned int', but argument

[PATCH v4 1/2] f2fs: fix to avoid broken of dnode block list

2018-07-25 Thread Chao Yu
uk: "Practically, I don't see db corruption becase of this. We can excuse to lose the last transaction." Finally, we decide to keep original implementation of atomic write interface sematics that we don't wait all dnode writeback before preflush+fua submission. Signed-off-by: C

[PATCH v4 2/2] f2fs: let checkpoint flush dnode page of regular

2018-07-25 Thread Chao Yu
IO of dnode page can be elimitnated, making fsyncer only need to wait for sync IO. Signed-off-by: Chao Yu --- v4: - rebase to last dev-test. fs/f2fs/node.c| 8 +++- fs/f2fs/node.h| 5 + fs/f2fs/segment.c | 4 +++- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/fs

Re: [PATCH 2/5] f2fs: add cur_victim_sec for BG_GC to avoid skipping BG_GC victim

2018-07-24 Thread Chao Yu
On 2018/7/24 21:39, Yunlong Song wrote: > > > On 2018/7/24 21:11, Chao Yu wrote: >> On 2018/7/23 22:10, Yunlong Song wrote: >>> If f2fs aborts BG_GC, then the section bit of victim_secmap will be set, >>> which will cause the section skipped in the future get_vic

Re: [PATCH 2/5] f2fs: add cur_victim_sec for BG_GC to avoid skipping BG_GC victim

2018-07-24 Thread Chao Yu
On 2018/7/23 22:10, Yunlong Song wrote: > If f2fs aborts BG_GC, then the section bit of victim_secmap will be set, > which will cause the section skipped in the future get_victim of BG_GC. > In a worst case that each section in the victim_secmap is set and there > are enough free sections (so FG_GC

Re: [PATCH v2] f2fs: clear the remaining prefree_map of the section

2018-07-17 Thread Chao Yu
On 2018/7/16 18:03, Yunlong Song wrote: > For the case when sbi->segs_per_sec > 1 with lfs mode, take > section:segment = 5 for example, if the section prefree_map is > ...previous section | current section (1 1 0 1 1) | next section..., > then the start = x, end = x + 1, after start = start_segno

Re: [PATCH 5/5] f2fs: do not __punch_discard_cmd in lfs mode

2018-07-12 Thread Chao Yu
On 2018/7/12 23:09, Yunlong Song wrote: > In lfs mode, it is better to submit and wait for discard of the > new_blkaddr's overall section, rather than punch it which makes > more small discards and is not friendly with flash alignment. And > f2fs does not have to wait discard of each new_blkaddr ex

Re: [f2fs-dev] [PATCH 2/4] f2fs: allow wrong configure dio to buffered write

2018-07-09 Thread Chao Yu
On 2018/7/7 5:09, Jaegeuk Kim wrote: > This fixes to support unaligned dio as buffered writes. Should we return -EINVAL as manual of write said: EINVAL fd is attached to an object which is unsuitable for writing; or the file was opened with the O_DIRECT flag, and either the address specified

[PATCH] f2fs: enable real-time discard by default

2018-07-04 Thread Chao Yu
f2fs is focused on flash based storage, so let's enable real-time discard by default, if user don't want to enable it, 'nodiscard' mount option should be used on mount. Signed-off-by: Chao Yu --- fs/f2fs/super.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-)

[PATCH v2 1/2] f2fs: fix to avoid broken of dnode block list

2018-07-04 Thread Chao Yu
broken by out-of-order IO submission due to IO scheduler or driver. Signed-off-by: Chao Yu --- v2: add missing definition modification in f2fs.h. fs/f2fs/f2fs.h | 2 +- fs/f2fs/file.c | 17 - fs/f2fs/node.c | 4 ++-- 3 files changed, 7 insertions(+), 16 deletions(-) diff --git

Re: [PATCH] f2fs: Replace strncpy with memcpy

2018-07-01 Thread Chao Yu
On 2018/7/2 10:16, Guenter Roeck wrote: > On 07/01/2018 06:53 PM, Chao Yu wrote: >> On 2018/7/2 4:57, Guenter Roeck wrote: >>> gcc 8.1.0 complains: >>> >>> fs/f2fs/namei.c: In function 'f2fs_update_extension_list': >>> fs/f2fs/namei.c:25

[PATCH v2 2/2] f2fs: fix to do sanity check with extra_attr feature

2018-06-25 Thread Chao Yu
From: Chao Yu If FI_EXTRA_ATTR is set in inode by fuzzing, inode.i_addr[0] will be parsed as inode.i_extra_isize, then in __recover_inline_status, inline data address will beyond boundary of page, result in accessing invalid memory. So in this condition, during reading inode page, let'

[PATCH v2] f2fs: disable f2fs_check_rb_tree_consistence

2018-06-22 Thread Chao Yu
do not choose to kill it directly, instead, adding an flag to disable the check now, if there is related code change, we can reuse it to detect bugs. Signed-off-by: Yunlei He Signed-off-by: Chao Yu --- v2: use unlikely suggested by Jaegeuk. fs/f2fs/f2fs.h| 1 + fs/f2fs/segment.c | 10

[PATCH 1/2] f2fs: relocate readdir_ra configure initialization

2018-06-11 Thread Chao Yu
readdir_ra is sysfs configuration instead of mount option, so it should not be initialized in default_options(), otherwise after remount, it can be reset to be enabled which may not as user wish, so let's move it to f2fs_tuning_parameters(). Signed-off-by: Chao Yu --- fs/f2fs/super.c | 3 +

[PATCH 2/2] f2fs: fix error path of fill_super

2018-06-11 Thread Chao Yu
In fill_super, if root inode's attribute is incorrect, we need to call f2fs_destroy_stats to release stats memory. Signed-off-by: Chao Yu --- fs/f2fs/super.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index 19ef966a99e9..a790dae

[PATCH v2 2/2] f2fs: fix to update mtime correctly

2018-06-04 Thread Chao Yu
From: Chao Yu If we change system time to the past, get_mtime() will return a overflowed time, and SIT_I(sbi)->max_mtime will be udpated incorrectly, this patch fixes the two issues. Signed-off-by: Chao Yu --- v2: - fix to correct return value of get_mtime(). fs/f2fs/checkpoint.c | 2 +-

[PATCH 1/2] f2fs: fix to let caller retry allocating block address

2018-05-28 Thread Chao Yu
From: Chao Yu Configure io_bits with 2 and enable LFS mode, generic/013 reports below dmesg: BUG: unable to handle kernel NULL pointer dereference at 0104 *pdpt = 29b7b001 *pde = Oops: 0002 [#1] PREEMPT SMP Modules linked in: crc32_generic zram f2fs(O) rfcomm bnep

[PATCH 2/2] f2fs: fix to avoid accessing cross the boundary

2018-05-28 Thread Chao Yu
From: Chao Yu Configure io_bits with 2 and enable LFS mode, generic/017 reports below dmesg: BUG: unable to handle kernel NULL pointer dereference at 0039 *pdpt = 2fcb2001 *pde = Oops: [#1] PREEMPT SMP Modules linked in: crc32_generic zram f2fs(O) bnep rfcomm

[PATCH 2/2] f2fs: fix error path of move_data_page

2018-05-28 Thread Chao Yu
This patch fixes error path of move_data_page: - clear cold data flag if it fails to write page. - redirty page for non-ENOMEM case. Signed-off-by: Chao Yu --- fs/f2fs/gc.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index

[PATCH 1/2] f2fs: don't drop dentry pages after fs shutdown

2018-05-28 Thread Chao Yu
As description in commit "f2fs: don't drop any page on f2fs_cp_error() case": "We still provide readdir() after shtudown, so we should keep pages to avoid additional IOs." In order to provider lastest directory structure, let's keep dentry pages in cache after fs

[PATCH v3] f2fs: fix to avoid race during access gc_thread pointer

2018-05-28 Thread Chao Yu
thread mount option, the memory can be released if we turn off that mount option, but still there are several places access gc_thread pointer without considering race condition, result in NULL point dereference. In order to fix this issue, use sb->s_umount to exclude those operations. Signed-

Re: [RESEND PATCH V5 24/33] f2fs: conver to bio_for_each_page_all2

2018-05-28 Thread Chao Yu
On 2018/5/25 11:46, Ming Lei wrote: > bio_for_each_page_all() can't be used any more after multipage bvec is > enabled, so we have to convert to bio_for_each_page_all2(). > > Signed-off-by: Ming Lei Acked-by: Chao Yu Thanks,

Re: [PATCH] f2fs-tools: fix overflow bug of start_sector when computing zone_align_start_offset

2018-05-28 Thread Chao Yu
On 2018/5/26 16:09, Yunlong Song wrote: > zone_align_start_offset should be u64, but config.start_sector is u32, > so it may be overflow when computing zone_align_start_offset. Could you rebase this patch on top of "f2fs-tools: fix to match with the start_sector"? Thanks, > > Signed-off-by: Yun

Re: [PATCH v2] f2fs-tools: fix to match with the start_sector

2018-05-28 Thread Chao Yu
ce start_sector is got from ioctl HDIO_GETGEO and > is always in 512 size unit for a sector. To fix this problem, just > change the sector_size to the default value when computing with > start_sector. And fix sectors_per_blk as well. > > Signed-off-by: Yunlong Song Reviewed-by: Chao Yu Thanks,

[PATCH 2/2] f2fs: clean up with clear_radix_tree_dirty_tag

2018-05-26 Thread Chao Yu
Introduce clear_radix_tree_dirty_tag to include common codes for cleanup. Signed-off-by: Chao Yu --- fs/f2fs/data.c | 11 +++ fs/f2fs/dir.c| 8 +--- fs/f2fs/f2fs.h | 1 + fs/f2fs/inline.c | 7 +-- fs/f2fs/node.c | 12 ++-- 5 files changed, 16 insertions

[PATCH 1/2] f2fs: fix to don't trigger writeback during recovery

2018-05-26 Thread Chao Yu
order to avoid entering above writeback flow, let's detect recovery status and do skip in f2fs_balance_fs_bg. Signed-off-by: Chao Yu Signed-off-by: Yunlei He --- fs/f2fs/segment.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 7dc59ae

Re: [PATCH v2] f2fs: let fstrim issue discard commands in lower priority

2018-05-25 Thread Chao Yu
On 2018/5/26 3:49, Jaegeuk Kim wrote: > The fstrim gathers huge number of large discard commands, and tries to issue > without IO awareness, which results in long user-perceive IO latencies on > READ, WRITE, and FLUSH in UFS. We've observed some of commands take several > seconds due to long discar

Re: [RFC PATCH] f2fs: add fsync_mode=nobarrier for non-atomic files

2018-05-25 Thread Chao Yu
On 2018/5/26 9:04, Jaegeuk Kim wrote: > For non-atomic files, this patch adds an option to give nobarrier which > doesn't issue flush commands to the device. > > Signed-off-by: Jaegeuk Kim Reviewed-by: Chao Yu Thanks,

[PATCH v2] f2fs: keep migration IO order in LFS mode

2018-05-25 Thread Chao Yu
dd an additional lock to keep submitting order. Signed-off-by: Chao Yu Signed-off-by: Yunlong Song --- v2: - introduce variable lfs_mode to record historical option, it can avoid option being changed. fs/f2fs/f2fs.h| 2 ++ fs/f2fs/gc.c | 6 ++ fs/f2fs/segment.c | 5 + fs/f2fs/super.c

Re: [PATCH v2] f2fs: let fstrim issue discard commands in lower priority

2018-05-25 Thread Chao Yu
conds due to long discard latency. > > This patch limits the maximum size to 2MB per candidate, and check IO > congestion > when issuing them to disk. > > Signed-off-by: Jaegeuk Kim It looks good to me. :) Reviewed-by: Chao Yu Thanks,

Re: [PATCH v3] f2fs: Fix deadlock in shutdown ioctl

2018-05-25 Thread Chao Yu
f2fs_ioctl+0xc0c/0x1ce0 > f2fs_compat_ioctl+0x98/0x1f0 > > Signed-off-by: Sahitya Tummala Reviewed-by: Chao Yu Thanks,

Re: [PATCH] f2fs: let fstrim issue discard commands in lower priority

2018-05-24 Thread Chao Yu
On 2018/5/25 14:18, Jaegeuk Kim wrote: > On 05/25, Chao Yu wrote: >> Hi Jaegeuk, >> >> On 2018/5/25 13:10, Jaegeuk Kim wrote: >>> The fstrim gathers huge number of large discard commands, and tries to issue >>> without IO awareness, which results in long

Re: [PATCH] f2fs: let fstrim issue discard commands in lower priority

2018-05-24 Thread Chao Yu
Hi Jaegeuk, On 2018/5/25 13:10, Jaegeuk Kim wrote: > The fstrim gathers huge number of large discard commands, and tries to issue > without IO awareness, which results in long user-perceive IO latencies on > READ, WRITE, and FLUSH in UFS. We've observed some of commands take several > seconds due

[PATCH] f2fs: keep migration IO order in LFS mode

2018-05-24 Thread Chao Yu
dd an additional lock to keep submitting order. Signed-off-by: Chao Yu Signed-off-by: Yunlong Song --- fs/f2fs/f2fs.h| 2 ++ fs/f2fs/gc.c | 5 + fs/f2fs/segment.c | 5 + fs/f2fs/super.c | 1 + 4 files changed, 13 insertions(+) diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h

[PATCH 2/2] f2fs: detect synchronous writeback more earlier

2018-05-23 Thread Chao Yu
From: Chao Yu This patch changes to detect synchronous writeback more earlier before, in order to avoid unnecessary page writeback before exiting asynchronous writeback. Signed-off-by: Chao Yu --- fs/f2fs/data.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a

[PATCH 1/2] f2fs: clean up with is_valid_blkaddr()

2018-05-23 Thread Chao Yu
From: Chao Yu - rename is_valid_blkaddr() to is_valid_meta_blkaddr() for readability. - introduce is_valid_blkaddr() for cleanup. No logic change in this patch. Signed-off-by: Chao Yu --- fs/f2fs/checkpoint.c | 4 ++-- fs/f2fs/data.c | 18 +- fs/f2fs/f2fs.h | 9

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-17 Thread Chao Yu
On 2018/5/8 5:36, Jaegeuk Kim wrote: > On 05/07, Chao Yu wrote: >> Thread A Thread BThread C >> - f2fs_remount >> - stop_gc_thread >> - f2fs_sbi_store >>

Re: [PATCH 1/3] f2fs: fix to wait page writeback during revoking atomic write

2018-05-17 Thread Chao Yu
Hi Jaegeuk, Could you recheck this patch? On 2018/4/23 10:36, Chao Yu wrote: > After revoking atomic write, related LBA can be reused by others, so we > need to wait page writeback before reusing the LBA, in order to avoid > interference between old atomic written in-flight IO a

Re: [PATCH v2] f2fs: Fix deadlock in shutdown ioctl

2018-05-17 Thread Chao Yu
On 2018/5/17 16:03, Sahitya Tummala wrote: > f2fs_ioc_shutdown() ioctl gets stuck in the below path > when issued with F2FS_GOING_DOWN_FULLSYNC option. > > __switch_to+0x90/0xc4 > percpu_down_write+0x8c/0xc0 > freeze_super+0xec/0x1e4 > freeze_bdev+0xc4/0xcc > f2fs_ioctl+0xc0c/0x1ce0 > f2fs_compat_

Re: [PATCH] f2fs: Fix deadlock in shutdown ioctl

2018-05-15 Thread Chao Yu
On 2018/5/15 15:57, Sahitya Tummala wrote: > On Mon, May 14, 2018 at 11:39:42AM +0800, Chao Yu wrote: >> On 2018/5/10 21:20, Sahitya Tummala wrote: >>> f2fs_ioc_shutdown() ioctl gets stuck in the below path >>> when going down with full sync (F2FS_GOIN

Re: [PATCH v2 1/3] f2fs: fix to initialize i_current_depth according to inode type

2018-05-15 Thread Chao Yu
Sorry, I sent the wrong patch, please ignore this one. On 2018/5/15 18:50, Chao Yu wrote: > i_current_depth is used only for directory inode, but its space is > shared with i_gc_failures field used for regular inode, in order to > avoid affecting i_gc_failures' value, this

[PATCH] f2fs: fix to initialize min_mtime with ULLONG_MAX

2018-05-15 Thread Chao Yu
Since sit_i.min_mtime's type is unsigned long long, so we should initialize it with max value of the type ULLONG_MAX instead of LLONG_MAX. Signed-off-by: Chao Yu --- fs/f2fs/segment.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c

[PATCH v2 1/3] f2fs: fix to initialize i_current_depth according to inode type

2018-05-15 Thread Chao Yu
i_current_depth is used only for directory inode, but its space is shared with i_gc_failures field used for regular inode, in order to avoid affecting i_gc_failures' value, this patch fixes to initialize the union's fields according to inode type. Signed-off-by: Chao Yu --- v2: - r

Re: [PATCH] f2fs: Fix deadlock in shutdown ioctl

2018-05-13 Thread Chao Yu
On 2018/5/10 21:20, Sahitya Tummala wrote: > f2fs_ioc_shutdown() ioctl gets stuck in the below path > when going down with full sync (F2FS_GOING_DOWN_FULLSYNC) > option. > > __switch_to+0x90/0xc4 > percpu_down_write+0x8c/0xc0 > freeze_super+0xec/0x1e4 > freeze_bdev+0xc4/0xcc > f2fs_ioctl+0xc0c/0x1

Re: [PATCH 32/76] fs/f2fs: Use inode_sb() helper instead of inode->i_sb

2018-05-10 Thread Chao Yu
On 2018/5/9 2:03, Mark Fasheh wrote: > Signed-off-by: Mark Fasheh Reviewed-by: Chao Yu Thanks,

[PATCH] f2fs: fix to let checkpoint guarantee atomic page persistence

2018-05-07 Thread Chao Yu
patch fixes to treat atomic page as checkpoint guaranteed one, then in checkpoint, we can make sure all atomic page can be writebacked with metadata of atomic file. Signed-off-by: Chao Yu --- fs/f2fs/data.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-07 Thread Chao Yu
On 2018/5/8 11:31, Jaegeuk Kim wrote: > On 05/08, Chao Yu wrote: >> On 2018/5/8 4:46, Jaegeuk Kim wrote: >>> On 04/27, Chao Yu wrote: >>>> On 2018/4/27 0:36, Jaegeuk Kim wrote: >>>>> On 04/26, Chao Yu wrote: >>>>>> On 2018

[PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Chao Yu
thread mount option, the memory can be released if we turn off that mount option, but still there are several places access gc_thread pointer without considering race condition, result in NULL point dereference. In order to fix this issue, use sb->s_umount to exclude those operations. Signed-

[PATCH] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Chao Yu
thread mount option, the memory can be released if we turn off that mount option, but still there are several places access gc_thread pointer without considering race condition, result in NULL point dereference. In order to fix this issue, use sb->s_umount to exclude those operations. Signed-

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Chao Yu
On 2018/5/8 11:17, Jaegeuk Kim wrote: > On 05/08, Chao Yu wrote: >> On 2018/5/8 5:36, Jaegeuk Kim wrote: >>> On 05/07, Chao Yu wrote: >>>> Thread A Thread BThread C >>>> - f2fs_remount >>>> - stop_gc_t

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-07 Thread Chao Yu
On 2018/5/8 4:46, Jaegeuk Kim wrote: > On 04/27, Chao Yu wrote: >> On 2018/4/27 0:36, Jaegeuk Kim wrote: >>> On 04/26, Chao Yu wrote: >>>> On 2018/4/26 23:48, Jaegeuk Kim wrote: >>>>> On 04/26, Chao Yu wrote: >>>>>> Thread A

[PATCH] f2fs: fix to wait atomic pages writeback in block_operations()

2018-05-07 Thread Chao Yu
patch fixes to try to wait all atomic pages writeback in block_operations(). Signed-off-by: Chao Yu --- fs/f2fs/checkpoint.c | 4 +++- fs/f2fs/data.c | 2 ++ fs/f2fs/f2fs.h | 2 ++ fs/f2fs/segment.c| 17 + 4 files changed, 24 insertions(+), 1 deletion(-) diff --git

Re: [PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Chao Yu
On 2018/5/8 5:36, Jaegeuk Kim wrote: > On 05/07, Chao Yu wrote: >> Thread A Thread BThread C >> - f2fs_remount >> - stop_gc_thread >> - f2fs_sbi_store >>

[PATCH v2 3/3] f2fs: avoid stucking GC due to atomic write

2018-05-07 Thread Chao Yu
ition, his patch adds to show GC skip information in debugfs, now it just shows count of skipped caused by atomic write. Signed-off-by: Chao Yu --- v2: - rebase code. fs/f2fs/data.c| 2 +- fs/f2fs/debug.c | 6 ++ fs/f2fs/f2fs.h| 21 +++-- fs/f2fs/file

[PATCH v2 2/3] f2fs: introduce GC_I for cleanup

2018-05-07 Thread Chao Yu
Introduce GC_I to replace sbi->gc_thread for cleanup, no logic changes. Signed-off-by: Chao Yu --- v2: - rebase code. fs/f2fs/debug.c | 2 +- fs/f2fs/f2fs.h| 5 + fs/f2fs/gc.c | 14 +++--- fs/f2fs/segment.c | 4 ++-- fs/f2fs/super.c | 4 ++-- fs/f2fs/sysfs.c |

[PATCH v2 1/3] f2fs: fix to initialize i_current_depth according to inode type

2018-05-07 Thread Chao Yu
i_current_depth is used only for directory inode, but its space is shared with i_gc_failures field used for regular inode, in order to avoid affecting i_gc_failures' value, this patch fixes to initialize the union's fields according to inode type. Signed-off-by: Chao Yu --- v2: - r

[PATCH v4] f2fs: fix to avoid race during access gc_thread pointer

2018-05-07 Thread Chao Yu
are several places access gc_thread pointer without considering race condition, result in NULL point dereference. In order to fix this issue, introduce gc_rwsem to exclude those operations. Signed-off-by: Chao Yu --- v4: - use introduced sbi.gc_rwsem lock instead of sb.s_umount. fs/f2fs/f2fs.h

Re: [PATCH v3] f2fs: fix to avoid race during access gc_thread pointer

2018-05-05 Thread Chao Yu
On 2018/5/5 18:02, Chao Yu wrote: > Thread A Thread BThread C > - f2fs_remount > - stop_gc_thread > - f2fs_sbi_store > - issue_discard_thread >sbi

[PATCH 3/3] f2fs: avoid stucking GC due to atomic write

2018-05-05 Thread Chao Yu
ition, his patch adds to show GC skip information in debugfs, now it just shows count of skipped caused by atomic write. Signed-off-by: Chao Yu --- fs/f2fs/data.c| 2 +- fs/f2fs/debug.c | 8 fs/f2fs/f2fs.h| 19 +-- fs/f2fs/file.c| 20 ++-- fs

[PATCH 1/3] f2fs: fix to initialize i_current_depth according to inode type

2018-05-05 Thread Chao Yu
i_current_depth is used only for directory inode, but its space is shared with i_gc_failures field used for regular inode, in order to avoid affecting i_gc_failures' value, this patch fixes to initialize the union's fields according to inode type. Signed-off-by: Chao Yu --- fs/f2fs/in

[PATCH 2/3] f2fs: introduce GC_I for cleanup

2018-05-05 Thread Chao Yu
Introduce GC_I to replace sbi->gc_thread for cleanup, no logic changes. Signed-off-by: Chao Yu --- fs/f2fs/debug.c | 2 +- fs/f2fs/f2fs.h| 5 + fs/f2fs/gc.c | 14 +++--- fs/f2fs/segment.c | 4 ++-- fs/f2fs/super.c | 4 ++-- fs/f2fs/sysfs.c | 8 6 fi

[PATCH v3] f2fs: fix to avoid race during access gc_thread pointer

2018-05-05 Thread Chao Yu
are several places access gc_thread pointer without considering race condition, result in NULL point dereference. In order to fix this issue, use sb->s_umount to exclude those operations Signed-off-by: Chao Yu --- v3: - use sb->s_umount to make all race cases exclusive. fs/f2fs/gc.c

Re: [PATCH 2/2] Revert "f2fs: add ovp valid_blocks check for bg gc victim to fg_gc"

2018-05-04 Thread Chao Yu
Ping, On 2018/4/27 10:55, Chao Yu wrote: > Hi Jaegeuk, > > Missed this patch, or any problem in it? > > Thanks, > > On 2018/4/26 17:05, Chao Yu wrote: >> For extreme case: >> 10 section, op = 10%, no_fggc_threshold = 90% >> All section usage:

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-05-04 Thread Chao Yu
Ping, On 2018/4/27 10:37, Chao Yu wrote: > On 2018/4/27 0:36, Jaegeuk Kim wrote: >> On 04/26, Chao Yu wrote: >>> On 2018/4/26 23:48, Jaegeuk Kim wrote: >>>> On 04/26, Chao Yu wrote: >>>>> Thread A Thread B >>>>>

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-05-04 Thread Chao Yu
On 2018/5/5 2:59, Jaegeuk Kim wrote: > On 05/02, Chao Yu wrote: >> On 2018/4/28 10:36, Jaegeuk Kim wrote: >>> On 04/27, Chao Yu wrote: >>>> On 2018/4/27 0:03, Jaegeuk Kim wrote: >>>>> On 04/24, Chao Yu wrote: >>>>>> Thread A

Re: [PATCH] f2fs-tools: fix the sector_size to default value

2018-05-04 Thread Chao Yu
On 2018/5/4 17:20, Yunlong Song wrote: > f2fs-tools uses ioctl BLKSSZGET to get sector_size, however, this ioctl > will return a value which may be larger than 512 (according to the value > of q->limits.logical_block_size), then this will be inconsistent with > the start_sector, since start_sector

Re: [PATCH] f2fs: fix spelling mistake: "extenstion" -> "extension"

2018-05-04 Thread Chao Yu
On 2018/4/30 23:27, Colin King wrote: > From: Colin Ian King > > Trivial fix to spelling mistake in extension list text > > Signed-off-by: Colin Ian King Reviewed-by: Chao Yu Thanks,

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-05-01 Thread Chao Yu
On 2018/4/28 10:36, Jaegeuk Kim wrote: > On 04/27, Chao Yu wrote: >> On 2018/4/27 0:03, Jaegeuk Kim wrote: >>> On 04/24, Chao Yu wrote: >>>> Thread A Thread BThread C >>>> - f2fs_remount >>>> - stop_gc_t

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-05-01 Thread Chao Yu
On 2018/4/28 10:34, Jaegeuk Kim wrote: > On 04/27, Chao Yu wrote: >> On 2018/4/26 23:54, Jaegeuk Kim wrote: >>> On 04/24, Chao Yu wrote: >>>> f2fs doesn't allow abuse on atomic write class interface, so except >>>> limiting in-mem pages' total

Re: [PATCH 2/2] Revert "f2fs: add ovp valid_blocks check for bg gc victim to fg_gc"

2018-04-26 Thread Chao Yu
Hi Jaegeuk, Missed this patch, or any problem in it? Thanks, On 2018/4/26 17:05, Chao Yu wrote: > For extreme case: > 10 section, op = 10%, no_fggc_threshold = 90% > All section usage: 85% 85% 85% 85% 90% 90% 95% 95% 95% 95% > > During foreground GC, if we skip select dirty sect

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Chao Yu
On 2018/4/27 0:36, Jaegeuk Kim wrote: > On 04/26, Chao Yu wrote: >> On 2018/4/26 23:48, Jaegeuk Kim wrote: >>> On 04/26, Chao Yu wrote: >>>> Thread A Thread B >>>> - f2fs_ioc_commit_atomic_write >>>> - com

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-26 Thread Chao Yu
On 2018/4/27 10:04, Chao Yu wrote: > On 2018/4/27 0:03, Jaegeuk Kim wrote: >> On 04/24, Chao Yu wrote: >>> Thread AThread BThread C >>> - f2fs_remount >>> - stop_gc_thread >>>

Re: [PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-26 Thread Chao Yu
On 2018/4/27 0:03, Jaegeuk Kim wrote: > On 04/24, Chao Yu wrote: >> Thread A Thread BThread C >> - f2fs_remount >> - stop_gc_thread >> - f2fs_sbi_store >>

Re: [f2fs-dev] [PATCH 4/5 v2] f2fs: sanity check for total valid blocks

2018-04-26 Thread Chao Yu
d-and-tested-by: syzbot+bf9253040425feb15...@syzkaller.appspotmail.com > Reported-and-tested-by: syzbot+7d6d31d3bc702f566...@syzkaller.appspotmail.com > Reported-and-tested-by: syzbot+0a725420475916460...@syzkaller.appspotmail.com > Signed-off-by: Jaegeuk Kim Reviewed-by: Chao Yu Thanks,

Re: [PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-04-26 Thread Chao Yu
On 2018/4/26 23:54, Jaegeuk Kim wrote: > On 04/24, Chao Yu wrote: >> f2fs doesn't allow abuse on atomic write class interface, so except >> limiting in-mem pages' total memory usage capacity, we need to limit >> atomic-write usage as well when filesystem is seriou

Re: [f2fs-dev] [PATCH 5/5] f2fs: enforce fsync_mode=strict for renamed directory

2018-04-26 Thread Chao Yu
On 2018/4/26 23:32, Jaegeuk Kim wrote: > On 04/25, Chao Yu wrote: >> On 2018/4/25 13:46, Jaegeuk Kim wrote: >>> This is to give a option for user to be able to recover B/foo in the below >>> case. >>> >>> mkdir A >>> sync() >>>

Re: [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Chao Yu
On 2018/4/26 23:48, Jaegeuk Kim wrote: > On 04/26, Chao Yu wrote: >> Thread A Thread B >> - f2fs_ioc_commit_atomic_write >> - commit_inmem_pages >> - f2fs_submit_merged_write_cond >> : write data >>

[PATCH 1/2] f2fs: treat volatile file's data as hot one

2018-04-26 Thread Chao Yu
Volatile file's data will be updated oftenly, so it'd better to place its data into hot data segment. In addition, for atomic file, we change to check FI_ATOMIC_FILE instead of FI_HOT_DATA to make code readability better. Signed-off-by: Chao Yu --- fs/f2fs/file.c| 2 -- fs/f2fs

[PATCH 2/2] Revert "f2fs: add ovp valid_blocks check for bg gc victim to fg_gc"

2018-04-26 Thread Chao Yu
: Chao Yu --- fs/f2fs/f2fs.h| 3 --- fs/f2fs/gc.c | 16 fs/f2fs/segment.h | 9 - 3 files changed, 28 deletions(-) diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 64e3677998d8..9f8b327272df 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -1192,9 +1192,6

[PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

2018-04-26 Thread Chao Yu
it results in atomic written data being corrupted. This patch adds f2fs_wait_on_page_writeback in __revoke_inmem_pages() to keep data and node of atomic file being flushed orderly. Signed-off-by: Chao Yu --- fs/f2fs/file.c| 4 fs/f2fs/segment.c | 3 +++ 2 files changed, 7 inserti

Re: [PATCH v2] f2fs: wait IO writeback in commit_inmem_pages()

2018-04-25 Thread Chao Yu
y, let me merge these two patches. Thanks, On 2018/4/23 23:14, Chao Yu wrote: > From: Chao Yu > > Thread A Thread B > - f2fs_ioc_commit_atomic_write > - commit_inmem_pages > - f2fs_submit_m

Re: [f2fs-dev] [PATCH 5/5] f2fs: enforce fsync_mode=strict for renamed directory

2018-04-25 Thread Chao Yu
On 2018/4/25 13:46, Jaegeuk Kim wrote: > This is to give a option for user to be able to recover B/foo in the below > case. > > mkdir A > sync() > rename(A, B) > creat (B/foo) > fsync (B/foo) > ---crash--- That makes sense, IMO, it will be better to cover cross rename case as well? Thanks, > >

Re: [f2fs-dev] [PATCH 4/5] f2fs: sanity check for total valid blocks

2018-04-25 Thread Chao Yu
Hi Jaegeuk, This patch makes generic/008 failed, because for fallocate case, total valid block count can not be calculated by gathering valid_blocks of all sit entries. Thanks, On 2018/4/25 13:46, Jaegeuk Kim wrote: > This patch enhances sanity check for SIT entries. > > syzbot hit the followin

Re: [f2fs-dev] [PATCH 3/5] f2fs: sanity check on sit entry

2018-04-25 Thread Chao Yu
0a > R10: R11: 0297 R12: 0004 > R13: 00402c60 R14: R15: > RIP: build_sit_entries fs/f2fs/segment.c:3653 [inline] RSP: 8801b102e5b0 > RIP: build_segment_manager+0x7ef7/0xbf70 fs/f2fs/segment.c:3852 RSP

Re: [f2fs-dev] [PATCH 2/5] f2fs: avoid bug_on on corrupted inode

2018-04-25 Thread Chao Yu
/f2fs/inode.c:238 [inline] RSP: 8801c44a70e8 > RIP: f2fs_iget+0x3307/0x3ca0 fs/f2fs/inode.c:313 RSP: 8801c44a70e8 > invalid opcode: [#2] SMP KASAN > ---[ end trace 1cbcbec2156680bc ]--- > > Reported-and-tested-by: syzbot+41a1b341571f0952b...@syzkaller.appspotmail.com > Signed-off-by: Jaegeuk Kim Reviewed-by: Chao Yu Thanks,

Re: [f2fs-dev] [PATCH 1/5] f2fs: give message and set need_fsck given broken node id

2018-04-25 Thread Chao Yu
On 2018/4/25 13:46, Jaegeuk Kim wrote: > syzbot hit the following crash on upstream commit > 83beed7b2b26f232d782127792dd0cd4362fdc41 (Fri Apr 20 17:56:32 2018 +) > Merge branch 'fixes' of > git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal > syzbot dashboard link: > ht

[PATCH] f2fs: fix potential overflow

2018-04-25 Thread Chao Yu
From: Chao Yu In build_sit_entries(), if valid_blocks in SIT block is smaller than valid_blocks in journal, for below calculation: sbi->discard_blks += old_valid_blocks - se->valid_blocks; There will be two times potential overflow: - old_valid_blocks - se->valid_blocks will overflow,

[PATCH 2/3] f2fs: introduce {create,destroy}_discard_caches for cleanup

2018-04-25 Thread Chao Yu
Split discard slab cache related initial/release codes into separated function {create,destroy}_discard_caches, later we can maintain those independent functions in separated discard.c Signed-off-by: Chao Yu --- fs/f2fs/f2fs.h| 2 ++ fs/f2fs/segment.c | 24 fs/f2fs

[PATCH 3/3] f2fs: maintain discard interface separately

2018-04-25 Thread Chao Yu
devi, sbi->s_ndevs ? FDEV(devi).path: "", Signed-off-by: Chao Yu --- fs/f2fs/Makefile |2 +- fs/f2fs/discard.c | 1009 + fs/f2fs/f2fs.h| 34 +- fs/f2fs/segment.c | 991 +---

[PATCH 1/3] f2fs: introduce release_discard_addr() for cleanup

2018-04-25 Thread Chao Yu
Introduce release_discard_addr() to include common codes for cleanup. Signed-off-by: Chao Yu --- fs/f2fs/segment.c | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 9e0c6babacec..d5627195aa8e 100644 --- a/fs/f2fs

Re: [PATCH] f2fs: clear page_error for all the writebacking pages

2018-04-24 Thread Chao Yu
Hi Jaegeuk, On 2018/4/24 6:49, Jaegeuk Kim wrote: > This patch clear page_error bit, if the page is going to be writebacked. This patch is similar to previous patch ("f2fs: clear PageError on writepage"), only coverage is different, could you merge them? Thanks, > > Signed-off-by: Jaegeuk Kim

[PATCH] f2fs: rename dio_rwsem to i_gc_rwsem

2018-04-23 Thread Chao Yu
RW semphore dio_rwsem in struct f2fs_inode_info is introduced to avoid race between dio and data gc, but now, it is more wildly used to avoid foreground operation vs data gc. So rename it to i_gc_rwsem to improve its readability. Signed-off-by: Chao Yu --- fs/f2fs/data.c | 6 +++--- fs/f2fs

[PATCH v2] f2fs: avoid stucking GC due to atomic write

2018-04-23 Thread Chao Yu
ition, his patch adds to show GC skip information in debugfs, now it just shows count of skipped caused by atomic write. Signed-off-by: Chao Yu --- v2: - add to show skip info in debugfs. fs/f2fs/debug.c | 8 fs/f2fs/f2fs.h| 2 ++ fs/f2fs/file.c| 5 + fs/f2fs/gc.c

[PATCH v2] f2fs: fix to avoid race during access gc_thread pointer

2018-04-23 Thread Chao Yu
gc_thread. Signed-off-by: Chao Yu --- v2: - add a rw semaphore to exclude rw operation suggested by Jaegeuk. fs/f2fs/debug.c | 3 +-- fs/f2fs/f2fs.h| 9 ++- fs/f2fs/gc.c | 78 --- fs/f2fs/gc.h | 1 + fs/f2fs/segment.c | 1

<    5   6   7   8   9   10   11   12   13   14   >