Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance
Hi Jin, On 10/10/2013 04:09 PM, Jin Xu wrote: Hi Gu, I have a comment below. Date: Wed, 9 Oct 2013 12:04:09 +0800 From: guz.f...@cn.fujitsu.com To: yuan.mark.zh...@samsung.com CC: jaegeuk@samsung.com; linux-f2fs-devel@lists.sourceforge.net; linux-ker...@vger.kernel.org; linux-fsde...@vger.kernel.org; shu@samsung.com Subject: Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance Hi Yuan, On 10/08/2013 07:30 PM, Yuan Zhong wrote: ... Signed-off-by: Gu Zheng guz.f...@cn.fujitsu.com --- fs/f2fs/checkpoint.c | 11 +-- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 4 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index d808827..2a5999d 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -757,8 +757,15 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, bool is_umount) f2fs_put_page(cp_page, 1); /* wait for previous submitted node/meta pages writeback */ - while (get_pages(sbi, F2FS_WRITEBACK)) - congestion_wait(BLK_RW_ASYNC, HZ / 50); + sbi-cp_task = current; + while (get_pages(sbi, F2FS_WRITEBACK)) { + set_current_state(TASK_UNINTERRUPTIBLE); + if (!get_pages(sbi, F2FS_WRITEBACK)) + break; + io_schedule(); + } + __set_current_state(TASK_RUNNING); + sbi-cp_task = NULL; filemap_fdatawait_range(sbi-node_inode-i_mapping, 0, LONG_MAX); filemap_fdatawait_range(sbi-meta_inode-i_mapping, 0, LONG_MAX); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index a955a59..408ace7 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -365,6 +365,7 @@ struct f2fs_sb_info { struct mutex writepages; /* mutex for writepages() */ int por_doing; /* recovery is doing or not */ int on_build_free_nids; /* build_free_nids is doing */ + struct task_struct *cp_task; /* checkpoint task */ /* for orphan inode management */ struct list_head orphan_inode_list; /* orphan inode list */ diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index bd79bbe..3b20359 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -597,6 +597,10 @@ static void f2fs_end_io_write(struct bio *bio, int err) if (p-is_sync) complete(p-wait); + + if (!get_pages(p-sbi, F2FS_WRITEBACK) p-sbi-cp_task) + wake_up_process(p-sbi-cp_task); There is a risk of dereferencing a NULL pointer because here simply comparing the cp_task against NULL is not enough to avoid race in multi-thread environment. Another thread could have assigned it to NULL in the window between the comparison and waking up. Can not be that, checkpoint routine is always singleton and protected by cp_mutex and cp_rwsem. Thanks, Gu Regards, Jin + kfree(p); bio_put(bio); } -- 1.7.7 Regards, Gu This is a problem here, especially, when sync a large number of small files or dirs. In order to avoid this, a wait_list is introduced, the checkpoint thread will be dropped into the wait_list if the pages have not been written back, and will be waked up by contrast. Please pay some attention to the mail form, this mail is out of format in my mail client. Regards, Gu Regards, Yuan Signed-off-by: Yuan Zhong yuan.mark.zh...@samsung.com --- fs/f2fs/checkpoint.c | 3 +-- fs/f2fs/f2fs.h | 19 +++ fs/f2fs/segment.c | 1 + fs/f2fs/super.c | 1 + 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index ca39442..5d69ae0 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -758,8 +758,7 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, bool is_umount) f2fs_put_page(cp_page, 1); /* wait for previous submitted node/meta pages writeback */ - while (get_pages(sbi, F2FS_WRITEBACK)) - congestion_wait(BLK_RW_ASYNC, HZ / 50); + f2fs_writeback_wait(sbi); filemap_fdatawait_range(sbi-node_inode-i_mapping, 0, LONG_MAX); filemap_fdatawait_range(sbi-meta_inode-i_mapping, 0, LONG_MAX); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 7fd99d8..4b0d70e 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -18,6 +18,8 @@ #include linux/crc32.h #include linux/magic.h #include linux/kobject.h +#include linux/wait.h +#include linux/sched.h /* * For mount options @@ -368,6 +370,7 @@ struct f2fs_sb_info { struct mutex fs_lock[NR_GLOBAL_LOCKS]; /* blocking FS operations */ struct mutex node_write; /* locking node writes */ struct mutex writepages; /* mutex for writepages() */ + wait_queue_head_t writeback_wqh; /* wait_queue for writeback */ unsigned char next_lock_num; /* round-robin global locks */ int por_doing; /* recovery is doing or not */ int on_build_free_nids; /* build_free_nids is doing */ @@ -961,6 +964,22 @@ static inline int f2fs_readonly(struct super_block *sb) return sb-s_flags MS_RDONLY; } +static inline void f2fs_writeback_wait(struct f2fs_sb_info *sbi) +{ + DEFINE_WAIT(wait); +
[f2fs-dev] BUG_ON in recovery.c after powercut
Hi all, I've just triggered a BUG_ON, which looks similar to: http://www.mail-archive.com/linux-f2fs-devel@lists.sourceforge.net/msg00336.html I'm running 3.11.4 on a BeagleBone Black. Unfortunately, I need a couple of patches on top of 3.11.4, because I need MMC support: https://github.com/beagleboard/kernel/tree/3.11 If I put a 2012.12 Angstrom image for the beagle on the eMMC and use it as a rootfilesystem, I'm triggering the following issue, when doing a powercut: [...] [4.228477] F2FS-fs (mmcblk0p2): recover_inode and its dentry: ino = ca3d, name = system.journal, dir = ca24, err = 0 [4.242586] F2FS-fs (mmcblk0p2): recover_inode and its dentry: ino = ca3d, name = system.journal, dir = ca24, err = 0 [4.260272] F2FS-fs (mmcblk0p2): recover_inode and its dentry: ino = ca44, name = settings.UXHEXP, dir = a9df, err = 0 [4.276471] F2FS-fs (mmcblk0p2): recover_inode and its dentry: ino = ca5e, name = settings.8VMWXP, dir = a9e1, err = 0 [4.298343] F2FS-fs (mmcblk0p2): recover_data: ino = ca3d, recovered_data = 1 blocks, err = 0 [4.332907] [ cut here ] [4.337797] kernel BUG at /home/jan/work/beagle/linux-3.11/fs/f2fs/recovery.c:306! [4.345775] Internal error: Oops - BUG: 0 [#1] SMP ARM [4.351193] Modules linked in: [4.354435] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.11.4 #10 [4.360768] task: de05b440 ti: de05c000 task.ti: de05c000 [4.366483] PC is at recover_fsync_data+0xc0c/0xc48 [4.371629] LR is at recover_fsync_data+0x8fc/0xc48 [4.376773] pc : [c028e6a8]lr : [c028e398]psr: 8113 [4.376773] sp : de05dcf0 ip : fp : 0008101d [4.388867] r10: 2007fffb r9 : de2ba000 r8 : 200803f5 [4.394373] r7 : r6 : c134c4a0 r5 : de05dd6c r4 : dd9d3438 [4.401252] r3 : fffb r2 : c0f84000 r1 : 0470 r0 : de425000 [4.408132] Flags: Nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel [4.415834] Control: 10c5387d Table: 80004019 DAC: 0015 [4.421889] Process swapper/0 (pid: 1, stack limit = 0xde05c240) [4.428220] Stack: (0xde05dcf0 to 0xde05e000) [4.432817] dce0: 0001 c0092f34 [4.441440] dd00: 0001 0005d9bd c134c4c0 005e dd9c6508 00100100 [4.450063] dd20: 00200200 ca3d dd9c6508 c134c4c0 c134c660 ca3a 00d8 [4.458685] dd40: de05c000 000f de42505c dd9aac98 0002 ca3a [4.467308] dd60: ca3d dd9d3438 dd9d3418 00ca3dc8 [4.475930] dd80: dd9c6770 de2ba000 de2ba800 6e6790fc [4.484553] dda0: de05c000 c0281ab0 c0d81400 dd9c44c8 1000 dd803340 [4.493176] ddc0: 0081 dd8033e4 8001 de2ba800 8001 c0115da0 dd803340 de05de08 [4.501798] dde0: 62636d6d 70306b6c 0032 0008 0004 c010e398 79ec c051532c [4.510420] de00: 8001 de2bbcc0 c07ef0ac c027f188 [4.519042] de20: c0281530 c07973d8 8001 c0116a9c de05c000 c012dfd4 de01b200 c07ef0ac [4.527665] de40: de2bbcc0 8001 8001 c07dce38 c012e464 c07ef0ac [4.536286] de60: de2bbcc0 de2bbd00 0060 c01303b4 c0650a50 de2bbcc0 8001 [4.544909] de80: de01bb10 dd9ad270 000a 000a c00f02e0 000a de059000 [4.553531] dea0: c0650a50 8001 c0650a50 c134bd00 00a5 c0130ad0 [4.562153] dec0: 6180 de2bbd00 de2bbcc0 00b3 de3e8000 [4.570776] dee0: 8001 de3e8000 c0775f48 c0733f38 c0650a50 0b32 ff9c [4.579398] df00: c0066a80 dd9c19a8 c0121244 de01bb10 dd9aec48 c0775f48 00b3 [4.588020] df20: 00b3 c0775f59 c0775f20 c081ae40 c07334d0 c07887b4 00a5 c0734240 [4.596643] df40: de05df24 de05b440 c0775f48 c081ae60 c0775f59 c0775f20 c07343c0 [4.605264] df60: c081ae40 c07334d0 c07887b4 c0775f40 0007 c0733c54 0007 0007 [4.613886] df80: c07334d0 c1394fc0 c0500770 [4.622507] dfa0: c0500778 c0013788 [4.631128] dfc0: [4.639748] dfe0: 0013 ffdf67fc f77ffdd4 [4.648384] [c028e6a8] (recover_fsync_data+0xc0c/0xc48) from [c0281ab0] (f2fs_fill_super+0x580/0x670) [4.658487] [c0281ab0] (f2fs_fill_super+0x580/0x670) from [c0115da0] (mount_bdev+0x168/0x19c) [4.667848] [c0115da0] (mount_bdev+0x168/0x19c) from [c027f188] (f2fs_mount+0x18/0x20) [4.676565] [c027f188] (f2fs_mount+0x18/0x20) from [c0116a9c] (mount_fs+0x44/0x178) [4.685009] [c0116a9c] (mount_fs+0x44/0x178) from [c012e464] (vfs_kern_mount+0x4c/0xc0) [4.693819] [c012e464]
Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance
Date: Thu, 10 Oct 2013 16:11:53 +0800 From: guz.f...@cn.fujitsu.com To: jinuxst...@live.com CC: yuan.mark.zh...@samsung.com; jaegeuk@samsung.com; linux-f2fs-devel@lists.sourceforge.net; linux-ker...@vger.kernel.org; linux-fsde...@vger.kernel.org; shu@samsung.com Subject: Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance Hi Jin, On 10/10/2013 04:09 PM, Jin Xu wrote: Hi Gu, I have a comment below. Date: Wed, 9 Oct 2013 12:04:09 +0800 From: guz.f...@cn.fujitsu.com To: yuan.mark.zh...@samsung.com CC: jaegeuk@samsung.com; linux-f2fs-devel@lists.sourceforge.net; linux-ker...@vger.kernel.org; linux-fsde...@vger.kernel.org; shu@samsung.com Subject: Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance Hi Yuan, On 10/08/2013 07:30 PM, Yuan Zhong wrote: ... Signed-off-by: Gu Zheng guz.f...@cn.fujitsu.com --- fs/f2fs/checkpoint.c | 11 +-- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 4 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index d808827..2a5999d 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -757,8 +757,15 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, bool is_umount) f2fs_put_page(cp_page, 1); /* wait for previous submitted node/meta pages writeback */ - while (get_pages(sbi, F2FS_WRITEBACK)) - congestion_wait(BLK_RW_ASYNC, HZ / 50); + sbi-cp_task = current; + while (get_pages(sbi, F2FS_WRITEBACK)) { + set_current_state(TASK_UNINTERRUPTIBLE); + if (!get_pages(sbi, F2FS_WRITEBACK)) + break; + io_schedule(); + } + __set_current_state(TASK_RUNNING); + sbi-cp_task = NULL; filemap_fdatawait_range(sbi-node_inode-i_mapping, 0, LONG_MAX); filemap_fdatawait_range(sbi-meta_inode-i_mapping, 0, LONG_MAX); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index a955a59..408ace7 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -365,6 +365,7 @@ struct f2fs_sb_info { struct mutex writepages; /* mutex for writepages() */ int por_doing; /* recovery is doing or not */ int on_build_free_nids; /* build_free_nids is doing */ + struct task_struct *cp_task; /* checkpoint task */ /* for orphan inode management */ struct list_head orphan_inode_list; /* orphan inode list */ diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index bd79bbe..3b20359 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -597,6 +597,10 @@ static void f2fs_end_io_write(struct bio *bio, int err) if (p-is_sync) complete(p-wait); + + if (!get_pages(p-sbi, F2FS_WRITEBACK) p-sbi-cp_task) + wake_up_process(p-sbi-cp_task); There is a risk of dereferencing a NULL pointer because here simply comparing the cp_task against NULL is not enough to avoid race in multi-thread environment. Another thread could have assigned it to NULL in the window between the comparison and waking up. Can not be that, checkpoint routine is always singleton and protected by cp_mutex and cp_rwsem. The race could happen like this for example:On a SMP environment, thread 1 wakes up the checkpoint thread, thenthread 2 comes to the f2fs_end_io_write, compared the cp_task as not NULL,but at the same time, the checkpoint thread just assigned the cp_task to NULL.When thread 2 gets to the wake_up_process, dereferencing to NULL pointerhappens. Jin Thanks, Gu Regards, Jin + kfree(p); bio_put(bio); } -- 1.7.7 Regards, Gu This is a problem here, especially, when sync a large number of small files or dirs. In order to avoid this, a wait_list is introduced, the checkpoint thread will be dropped into the wait_list if the pages have not been written back, and will be waked up by contrast. Please pay some attention to the mail form, this mail is out of format in my mail client. Regards, Gu Regards, Yuan Signed-off-by: Yuan Zhong yuan.mark.zh...@samsung.com --- fs/f2fs/checkpoint.c | 3 +-- fs/f2fs/f2fs.h | 19 +++ fs/f2fs/segment.c | 1 + fs/f2fs/super.c | 1 + 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index ca39442..5d69ae0 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -758,8 +758,7 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, bool is_umount) f2fs_put_page(cp_page, 1); /* wait for previous submitted node/meta pages writeback */ - while (get_pages(sbi, F2FS_WRITEBACK)) - congestion_wait(BLK_RW_ASYNC, HZ / 50); + f2fs_writeback_wait(sbi); filemap_fdatawait_range(sbi-node_inode-i_mapping, 0, LONG_MAX); filemap_fdatawait_range(sbi-meta_inode-i_mapping, 0, LONG_MAX); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index 7fd99d8..4b0d70e 100644 ---
Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance
Hi Jin, On 10/11/2013 07:54 AM, Jin Xu wrote: Date: Thu, 10 Oct 2013 16:11:53 +0800 From: guz.f...@cn.fujitsu.com To: jinuxst...@live.com CC: yuan.mark.zh...@samsung.com; jaegeuk@samsung.com; linux-f2fs-devel@lists.sourceforge.net; linux-ker...@vger.kernel.org; linux-fsde...@vger.kernel.org; shu@samsung.com Subject: Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance Hi Jin, On 10/10/2013 04:09 PM, Jin Xu wrote: Hi Gu, I have a comment below. Date: Wed, 9 Oct 2013 12:04:09 +0800 From: guz.f...@cn.fujitsu.com To: yuan.mark.zh...@samsung.com CC: jaegeuk@samsung.com; linux-f2fs-devel@lists.sourceforge.net; linux-ker...@vger.kernel.org; linux-fsde...@vger.kernel.org; shu@samsung.com Subject: Re: [f2fs-dev] [PATCH v2] f2fs: avoid congestion_wait when do_checkpoint for better performance Hi Yuan, On 10/08/2013 07:30 PM, Yuan Zhong wrote: ... Signed-off-by: Gu Zheng guz.f...@cn.fujitsu.com --- fs/f2fs/checkpoint.c | 11 +-- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 4 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index d808827..2a5999d 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -757,8 +757,15 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, bool is_umount) f2fs_put_page(cp_page, 1); /* wait for previous submitted node/meta pages writeback */ - while (get_pages(sbi, F2FS_WRITEBACK)) - congestion_wait(BLK_RW_ASYNC, HZ / 50); + sbi-cp_task = current; + while (get_pages(sbi, F2FS_WRITEBACK)) { + set_current_state(TASK_UNINTERRUPTIBLE); + if (!get_pages(sbi, F2FS_WRITEBACK)) + break; + io_schedule(); + } + __set_current_state(TASK_RUNNING); + sbi-cp_task = NULL; filemap_fdatawait_range(sbi-node_inode-i_mapping, 0, LONG_MAX); filemap_fdatawait_range(sbi-meta_inode-i_mapping, 0, LONG_MAX); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index a955a59..408ace7 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -365,6 +365,7 @@ struct f2fs_sb_info { struct mutex writepages; /* mutex for writepages() */ int por_doing; /* recovery is doing or not */ int on_build_free_nids; /* build_free_nids is doing */ + struct task_struct *cp_task; /* checkpoint task */ /* for orphan inode management */ struct list_head orphan_inode_list; /* orphan inode list */ diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index bd79bbe..3b20359 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -597,6 +597,10 @@ static void f2fs_end_io_write(struct bio *bio, int err) if (p-is_sync) complete(p-wait); + + if (!get_pages(p-sbi, F2FS_WRITEBACK) p-sbi-cp_task) + wake_up_process(p-sbi-cp_task); There is a risk of dereferencing a NULL pointer because here simply comparing the cp_task against NULL is not enough to avoid race in multi-thread environment. Another thread could have assigned it to NULL in the window between the comparison and waking up. Can not be that, checkpoint routine is always singleton and protected by cp_mutex and cp_rwsem. The race could happen like this for example: On a SMP environment, thread 1 wakes up the checkpoint thread, then thread 2 comes to the f2fs_end_io_write, compared the cp_task as not NULL, but at the same time, the checkpoint thread just assigned the cp_task to NULL. When thread 2 gets to the wake_up_process, dereferencing to NULL pointer happens. The case means that two or more IO are going on. If thread 1 wake up checkpoint, it'll check get_pages(p-sbi, F2FS_WRITEBACK) before going down to assign the cp_task to NULL, so when thread 2 gets to the wake_up_process the cp_task is still valid. The while (get_pages(sbi, F2FS_WRITEBACK)) loop is used to avoid this issue. Thanks, Gu Jin Thanks, Gu Regards, Jin + kfree(p); bio_put(bio); } -- 1.7.7 Regards, Gu This is a problem here, especially, when sync a large number of small files or dirs. In order to avoid this, a wait_list is introduced, the checkpoint thread will be dropped into the wait_list if the pages have not been written back, and will be waked up by contrast. Please pay some attention to the mail form, this mail is out of format in my mail client. Regards, Gu Regards, Yuan Signed-off-by: Yuan Zhong yuan.mark.zh...@samsung.com --- fs/f2fs/checkpoint.c | 3 +-- fs/f2fs/f2fs.h | 19 +++ fs/f2fs/segment.c | 1 + fs/f2fs/super.c | 1 + 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index ca39442..5d69ae0 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -758,8 +758,7 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, bool is_umount) f2fs_put_page(cp_page, 1); /* wait for