> -----Original Message----- > From: changfeng...@vivo.com <changfeng...@vivo.com> On Behalf Of > Chao Yu > Sent: Tuesday, November 9, 2021 9:41 PM > To: 常凤楠 <changfeng...@vivo.com>; jaeg...@kernel.org > Cc: linux-f2fs-devel@lists.sourceforge.net > Subject: Re: Do we need serial io for compress file? > > On 2021/11/9 9:59, 常凤楠 wrote: > > > > > >> -----Original Message----- > >> From: changfeng...@vivo.com <changfeng...@vivo.com> On Behalf > Of Chao > >> Yu > >> Sent: Monday, November 8, 2021 10:21 PM > >> To: 常凤楠 <changfeng...@vivo.com>; jaeg...@kernel.org > >> Cc: linux-f2fs-devel@lists.sourceforge.net > >> Subject: Re: Do we need serial io for compress file? > >> > >> On 2021/11/8 11:54, Fengnan Chang wrote: > >>> In my test, serial io for compress file will make multithread small > >>> write performance drop a lot. > >>> > >>> I'm try to fingure out why we need __should_serialize_io, IMO, we > >>> use __should_serialize_io to avoid deadlock or try to improve > >>> sequential performance, but I don't understand why we should do this > >>> for > >> > >> It was introduced to avoid fragmentation of file blocks. > > > > So, for small write on compress file, is this still necessary? I think we > should treat compress file as regular file. > > Any real scenario there? let me know if I missed any cases, as I saw, most > compressible files are not small...
Maybe my description is incorrect, small write means write with small block size, for example 4K. When write multi compress file with bs=4k in multithread, the serialize io make performance drop a lot. > > >> > >>> compressed file. In my test, if we just remove this, write same file > >>> in multithread will have problem, but parallel write different files > >>> in multithread > >> > >> What do you mean by "write same file in multithread will have > problem"? > > > > If just remove compress file in __should_serialize_io() > > > > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index > > f4fd6c246c9a..7bd429b46429 100644 > > --- a/fs/f2fs/data.c > > +++ b/fs/f2fs/data.c > > @@ -3165,8 +3165,8 @@ static inline bool __should_serialize_io(struct > inode *inode, > > if (IS_NOQUOTA(inode)) > > return false; > > > > - if (f2fs_need_compress_data(inode)) > > - return true; > > + //if (f2fs_need_compress_data(inode)) > > + // return true; > > if (wbc->sync_mode != WB_SYNC_ALL) > > return true; > > if (get_dirty_pages(inode) >= > > SM_I(F2FS_I_SB(inode))->min_seq_blocks) > > > > and use fio to start multi thread to write same file, fio will hung. > > Any potential hangtask issue there? did you get any stack backtrace log? > Yes, it's quite easy to reproduce in my test. Backtrace: [ 493.915408] INFO: task fio:479 blocked for more than 122 seconds. [ 493.915729] Not tainted 5.15.0-rc1+ #516 [ 493.915845] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 493.916265] task:fio state:D stack:13008 pid: 479 ppid: 474 flags:0x00004000 [ 493.916686] Call Trace: [ 493.917310] __schedule+0x317/0x920 [ 493.917560] ? rcu_read_lock_sched_held+0x23/0x80 [ 493.917705] schedule+0x59/0xc0 [ 493.917798] io_schedule+0x12/0x40 [ 493.917895] __lock_page+0x141/0x230 [ 493.924703] ? filemap_invalidate_unlock_two+0x40/0x40 [ 493.926404] f2fs_write_multi_pages+0x1a7/0x9c0 [ 493.926579] f2fs_write_cache_pages+0x711/0x840 [ 493.926771] f2fs_write_data_pages+0x20e/0x3f0 [ 493.926906] ? rcu_read_lock_held_common+0xe/0x40 [ 493.928305] ? do_writepages+0xd1/0x190 [ 493.928439] do_writepages+0xd1/0x190 [ 493.928551] file_write_and_wait_range+0xa3/0xe0 [ 493.928690] f2fs_do_sync_file+0x10f/0x910 [ 493.928823] do_fsync+0x38/0x60 [ 493.928924] __x64_sys_fsync+0x10/0x20 [ 493.929208] do_syscall_64+0x3a/0x90 [ 493.929327] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 493.929486] RIP: 0033:0x7f64cb26ca97 [ 493.929636] RSP: 002b:00007ffe3863d180 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [ 493.929816] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f64cb26ca97 [ 493.929975] RDX: 0000000000000000 RSI: 000056378a8ad540 RDI: 0000000000000005 [ 493.931164] RBP: 00007f647e23f000 R08: 0000000000000000 R09: 00007ffe387e6170 [ 493.931347] R10: 00007ffe387e61b0 R11: 0000000000000293 R12: 0000000000000003 [ 493.931517] R13: 00007f647e2836d8 R14: 0000000000000000 R15: 000056378a8ad568 [ 493.931802] INFO: task fio:480 blocked for more than 122 seconds. [ 493.931946] Not tainted 5.15.0-rc1+ #516 [ 493.932202] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 493.932382] task:fio state:D stack:13216 pid: 480 ppid: 474 flags:0x00000000 [ 493.932596] Call Trace: [ 493.932681] __schedule+0x317/0x920 [ 493.932796] ? rcu_read_lock_sched_held+0x23/0x80 [ 493.932939] schedule+0x59/0xc0 [ 493.933180] io_schedule+0x12/0x40 [ 493.933285] __lock_page+0x141/0x230 [ 493.933391] ? filemap_invalidate_unlock_two+0x40/0x40 [ 493.933539] f2fs_write_cache_pages+0x38a/0x840 [ 493.933708] ? do_writepages+0xd1/0x190 [ 493.933821] ? do_writepages+0xd1/0x190 [ 493.933939] f2fs_write_data_pages+0x20e/0x3f0 [ 493.935516] ? rcu_read_lock_held_common+0xe/0x40 [ 493.935682] ? do_writepages+0xd1/0x190 [ 493.935790] do_writepages+0xd1/0x190 [ 493.935895] file_write_and_wait_range+0xa3/0xe0 [ 493.938074] f2fs_do_sync_file+0x10f/0x910 [ 493.938225] do_fsync+0x38/0x60 [ 493.938328] __x64_sys_fsync+0x10/0x20 [ 493.938430] do_syscall_64+0x3a/0x90 [ 493.938548] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 493.938683] RIP: 0033:0x7f64cb26ca97 [ 493.938783] RSP: 002b:00007ffe3863d180 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [ 493.938969] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f64cb26ca97 [ 493.940413] RDX: 0000000000000000 RSI: 000056378a8ad540 RDI: 0000000000000005 [ 493.940591] RBP: 00007f647e283a10 R08: 0000000000000000 R09: 00007ffe387e6170 [ 493.940759] R10: 00007ffe387e61b0 R11: 0000000000000293 R12: 0000000000000003 [ 493.940924] R13: 00007f647e2c80e8 R14: 0000000000000000 R15: 000056378a8ad568 [ 493.941341] INFO: lockdep is turned off. [ 616.796413] INFO: task fio:479 blocked for more than 245 seconds. [ 616.796706] Not tainted 5.15.0-rc1+ #516 [ 616.796824] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 616.827081] task:fio state:D stack:13008 pid: 479 ppid: 474 flags:0x00004000 [ 616.841373] Call Trace: [ 616.841492] __schedule+0x317/0x920 [ 616.841635] ? rcu_read_lock_sched_held+0x23/0x80 [ 616.841767] schedule+0x59/0xc0 [ 616.841858] io_schedule+0x12/0x40 [ 616.841952] __lock_page+0x141/0x230 [ 616.852135] ? filemap_invalidate_unlock_two+0x40/0x40 [ 616.852305] f2fs_write_multi_pages+0x1a7/0x9c0 [ 616.852455] f2fs_write_cache_pages+0x711/0x840 [ 616.852661] f2fs_write_data_pages+0x20e/0x3f0 [ 616.852785] ? rcu_read_lock_held_common+0xe/0x40 [ 616.852924] ? do_writepages+0xd1/0x190 [ 616.853281] do_writepages+0xd1/0x190 [ 616.853409] file_write_and_wait_range+0xa3/0xe0 [ 616.853537] f2fs_do_sync_file+0x10f/0x910 [ 616.853670] do_fsync+0x38/0x60 [ 616.853773] __x64_sys_fsync+0x10/0x20 [ 616.853869] do_syscall_64+0x3a/0x90 [ 616.853971] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 616.863145] RIP: 0033:0x7f64cb26ca97 [ 616.863273] RSP: 002b:00007ffe3863d180 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [ 616.863468] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f64cb26ca97 [ 616.863635] RDX: 0000000000000000 RSI: 000056378a8ad540 RDI: 0000000000000005 [ 616.863803] RBP: 00007f647e23f000 R08: 0000000000000000 R09: 00007ffe387e6170 [ 616.863975] R10: 00007ffe387e61b0 R11: 0000000000000293 R12: 0000000000000003 [ 616.870147] R13: 00007f647e2836d8 R14: 0000000000000000 R15: 000056378a8ad568 [ 616.870388] INFO: task fio:480 blocked for more than 245 seconds. [ 616.870537] Not tainted 5.15.0-rc1+ #516 [ 616.870650] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 616.870833] task:fio state:D stack:13216 pid: 480 ppid: 474 flags:0x00000000 [ 616.872161] Call Trace: [ 616.872257] __schedule+0x317/0x920 [ 616.872366] ? rcu_read_lock_sched_held+0x23/0x80 [ 616.872503] schedule+0x59/0xc0 [ 616.872596] io_schedule+0x12/0x40 [ 616.872694] __lock_page+0x141/0x230 [ 616.872803] ? filemap_invalidate_unlock_two+0x40/0x40 [ 616.872945] f2fs_write_cache_pages+0x38a/0x840 [ 616.873331] ? do_writepages+0xd1/0x190 [ 616.873448] ? do_writepages+0xd1/0x190 [ 616.873569] f2fs_write_data_pages+0x20e/0x3f0 [ 616.873694] ? rcu_read_lock_held_common+0xe/0x40 [ 616.873833] ? do_writepages+0xd1/0x190 [ 616.873936] do_writepages+0xd1/0x190 [ 616.874272] file_write_and_wait_range+0xa3/0xe0 [ 616.874424] f2fs_do_sync_file+0x10f/0x910 [ 616.874560] do_fsync+0x38/0x60 [ 616.874660] __x64_sys_fsync+0x10/0x20 [ 616.874759] do_syscall_64+0x3a/0x90 [ 616.874863] entry_SYSCALL_64_after_hwframe+0x44/0xae [ 616.875792] RIP: 0033:0x7f64cb26ca97 [ 616.875905] RSP: 002b:00007ffe3863d180 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [ 616.876345] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f64cb26ca97 [ 616.876515] RDX: 0000000000000000 RSI: 000056378a8ad540 RDI: 0000000000000005 [ 616.876692] RBP: 00007f647e283a10 R08: 0000000000000000 R09: 00007ffe387e6170 [ 616.876859] R10: 00007ffe387e61b0 R11: 0000000000000293 R12: 0000000000000003 [ 616.877231] R13: 00007f647e2c80e8 R14: 0000000000000000 R15: 000056378a8ad568 [ 616.877456] INFO: lockdep is turned off. > If there is, it needs to figure out the root cause. > > Thanks, > > > fio.conf: > > [global] > > direct=1 > > numjobs=8 > > time_based > > runtime=30 > > ioengine=sync > > iodepth=16 > > buffer_pattern="ZZZZ" > > fsync=1 > > > > [file0] > > name=fio-rand-RW > > filename=fio-rand-RW > > rw=rw > > rwmixread=60 > > rwmixwrite=40 > > bs=1M > > size=64M > > > > [file1] > > name=fio-rand-RW > > filename=fio-rand-RW > > rw=randrw > > rwmixread=60 > > rwmixwrite=40 > > bs=4K > > size=64M > > > >> > >> Thanks, > >> > >>> is ok. So I think maybe we should use another lock to allow write > >> different files in multithread. > >>> _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
