[f2fs-dev] [RFC PATCH] f2fs-tools: fix blocks allocation direction inside segment

[Junling Zheng]

This fixes the block allocation direction from left to right inside
one segment despite of the direction of segment allocation.

Signed-off-by: Junling Zheng 
---
 fsck/mount.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/fsck/mount.c b/fsck/mount.c
index 1396a2e..38d2aa2 100644
--- a/fsck/mount.c
+++ b/fsck/mount.c
@@ -1921,13 +1921,13 @@ int find_next_free_block(struct f2fs_sb_info *sbi, u64 
*to, int left, int type)
 
if (se->valid_blocks == sbi->blocks_per_seg ||
IS_CUR_SEGNO(sbi, segno, type)) {
-   *to = left ? START_BLOCK(sbi, segno) - 1:
+   *to = left ? START_BLOCK(sbi, segno - 1) :
START_BLOCK(sbi, segno + 1);
continue;
}
 
if (se->valid_blocks == 0 && not_enough) {
-   *to = left ? START_BLOCK(sbi, segno) - 1:
+   *to = left ? START_BLOCK(sbi, segno - 1) :
START_BLOCK(sbi, segno + 1);
continue;
}
@@ -1949,7 +1949,10 @@ int find_next_free_block(struct f2fs_sb_info *sbi, u64 
*to, int left, int type)
!f2fs_test_bit(offset, (const char *)se->cur_valid_map))
return 0;
 
-   *to = left ? *to - 1: *to + 1;
+   if (!(OFFSET_IN_SEG(sbi, *to + 1)) && left)
+   *to = START_BLOCK(sbi, segno - 1);
+   else
+   *to = *to + 1;
}
return -1;
 }
-- 
2.16.2


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH] f2fs: clear discard_wake earlier

[Sheng Yong]

If SBI_NEED_FSCK is set, discard_wake will never be cleared. As a
result, the condition of wait_event_interruptible_timeout() is always
true, which gets discard thread run too frequently.

Signed-off-by: Sheng Yong 
---
 fs/f2fs/segment.c | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 503a98abdb2f..e53e28fd397e 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -1404,6 +1404,10 @@ static int issue_discard_thread(void *data)
kthread_should_stop() || freezing(current) ||
dcc->discard_wake,
msecs_to_jiffies(wait_ms));
+
+   if (dcc->discard_wake)
+   dcc->discard_wake = 0;
+
if (try_to_freeze())
continue;
if (f2fs_readonly(sbi->sb))
@@ -1415,9 +1419,6 @@ static int issue_discard_thread(void *data)
continue;
}
 
-   if (dcc->discard_wake)
-   dcc->discard_wake = 0;
-
if (sbi->gc_thread && sbi->gc_thread->gc_urgent)
init_discard_policy(, DPOLICY_FORCE, 1);
 
-- 
2.17.0


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] Hung task when executing generic/256 test

[Chandan Rajendra]

Hi,

The following "Hung task" information is seen when executing generic/256 test
on next-20180503.

[ 1964.025145] F2FS-fs (loop1): Mounted with checkpoint version = 22d8b4fd
[ 2212.833962] INFO: task umount:16199 blocked for more than 120 seconds.
[ 2212.835736]   Tainted: GW 
4.17.0-rc3-next-20180503-04271-g6eb58cf86baa #1
[ 2212.837944] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
[ 2212.840002] umount  D14184 16199  16195 0x
[ 2212.841470] Call Trace:
[ 2212.842211]  ? __schedule+0x28a/0xad0
[ 2212.843234]  schedule+0x2f/0x90
[ 2212.844122]  io_schedule+0x12/0x40
[ 2212.845071]  wait_on_page_bit_common+0xe2/0x160
[ 2212.846159]  ? page_cache_tree_insert+0x130/0x130
[ 2212.846827]  __filemap_fdatawait_range+0xe5/0x150
[ 2212.847473]  ? sync_inodes_sb+0x1ff/0x450
[ 2212.848050]  filemap_fdatawait_keep_errors+0x1a/0x40
[ 2212.848751]  sync_inodes_sb+0x262/0x450
[ 2212.849282]  sync_filesystem+0x57/0x90
[ 2212.849821]  generic_shutdown_super+0x22/0x100
[ 2212.850435]  kill_block_super+0x21/0x50
[ 2212.850989]  deactivate_locked_super+0x2f/0x70
[ 2212.851596]  cleanup_mnt+0x3b/0x70
[ 2212.852092]  task_work_run+0xa1/0xd0
[ 2212.852580]  exit_to_usermode_loop+0x83/0x90
[ 2212.853185]  do_syscall_64+0x157/0x180
[ 2212.853725]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 2212.854420] RIP: 0033:0x7fadbca7d357
[ 2212.854935] Code: Bad RIP value.
[ 2212.855384] RSP: 002b:7ffe15b2bbb8 EFLAGS: 0246 ORIG_RAX: 
00a6
[ 2212.856418] RAX:  RBX: 561162acc2d0 RCX: 7fadbca7d357
[ 2212.857393] RDX: 0001 RSI:  RDI: 561162ad1390
[ 2212.858391] RBP:  R08: fef8 R09: feff
[ 2212.859367] R10:  R11: 0246 R12: 561162ad1390
[ 2212.860345] R13: 7fadbd8231a4 R14: 561162acc4b0 R15: 7ffe15b2be28

The following is the contents of the xfstests' local.config file,

# F2FS
export TEST_DEV=/dev/loop0
export TEST_DIR=/mnt/xfstest-test
export SCRATCH_DEV=/dev/loop1
export SCRATCH_MNT=/mnt/xfstest-scratch
MKFS_OPTIONS="-O encrypt"
MOUNT_OPTIONS="-o test_dummy_encryption"

-- 
chandan


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] f2fs: fix to wait IO writeback in __revoke_inmem_pages()

[Chao Yu]

On 2018/5/8 11:31, Jaegeuk Kim wrote:
> On 05/08, Chao Yu wrote:
>> On 2018/5/8 4:46, Jaegeuk Kim wrote:
>>> On 04/27, Chao Yu wrote:
 On 2018/4/27 0:36, Jaegeuk Kim wrote:
> On 04/26, Chao Yu wrote:
>> On 2018/4/26 23:48, Jaegeuk Kim wrote:
>>> On 04/26, Chao Yu wrote:
 Thread A   Thread B
 - f2fs_ioc_commit_atomic_write
  - commit_inmem_pages
   - f2fs_submit_merged_write_cond
   : write data
- write_checkpoint
 - do_checkpoint
 : commit all node within CP
 -> SPO
   - f2fs_do_sync_file
- file_write_and_wait_range
: wait data writeback

 In above race condition, data/node can be flushed in reversed order 
 when
 coming a checkpoint before f2fs_do_sync_file, after SPOR, it results in
 atomic written data being corrupted.
>>>
>>> Wait, what is the problem here? Thread B could succeed checkpoint, 
>>> there is
>>> no problem. If it fails, there is no fsync mark where we can recover 
>>> it, so
>>
>> Node is flushed by checkpoint before data, with reversed order, that's 
>> the problem.
>
> What do you mean? Data should be in disk, in order to proceed checkpoint.

 1. thread A: commit_inmem_pages submit data into block layer, but haven't 
 waited
 it writeback.
 2. thread A: commit_inmem_pages update related node.
 3. thread B: do checkpoint, flush all nodes to disk
>>>
>>> How about, in block_operations(),
>>>
>>> down_read_trylock(_I(inode)->i_gc_rwsem[WRITE]);
>>> if (fail)
>>> wait_on_all_pages_writeback(F2FS_WB_DATA);
>>> else
>>> up_read(_I(inode)->i_gc_rwsem[WRITE]);
>>
>> I sent one patch for that, could you check it?
>>
>> Adding wait_on_all_pages_writeback in block_operations() can make 
>> checkpoint()
>> wait pages writeback one more time, which break IO flow, so what's your 
>> concern
>> here?
> 
> Performance. And I can see wait_on_all_pages_writeback() waits only for
> F2FS_WB_CP_DATA in checkpoint()?

Oh, you mean wait all F2FS_WB_DATA pages writeback, what about just treating
atomic write page as F2FS_WB_CP_DATA, and we can wait atomic pages in
wait_on_all_pages_writeback() in do_checkpoitn().

Thanks,

> 
> 
>>
>> Thanks,
>>
>>>
>>>
 4. SPOR

 Then, atomic file becomes corrupted since nodes is flushed before data.

 Thanks,

>
>>
>> Thanks,
>>
>>> we can just ignore the last written data as nothing.
>>>

 This patch adds f2fs_wait_on_page_writeback in __revoke_inmem_pages() 
 to
 keep data and node of atomic file being flushed orderly.

 Signed-off-by: Chao Yu 
 ---
  fs/f2fs/file.c| 4 
  fs/f2fs/segment.c | 3 +++
  2 files changed, 7 insertions(+)

 diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
 index be7578774a47..a352804af244 100644
 --- a/fs/f2fs/file.c
 +++ b/fs/f2fs/file.c
 @@ -217,6 +217,9 @@ static int f2fs_do_sync_file(struct file *file, 
 loff_t start, loff_t end,
  
trace_f2fs_sync_file_enter(inode);
  
 +  if (atomic)
 +  goto write_done;
 +
/* if fdatasync is triggered, let's do in-place-update */
if (datasync || get_dirty_pages(inode) <= 
 SM_I(sbi)->min_fsync_blocks)
set_inode_flag(inode, FI_NEED_IPU);
 @@ -228,6 +231,7 @@ static int f2fs_do_sync_file(struct file *file, 
 loff_t start, loff_t end,
return ret;
}
  
 +write_done:
/* if the inode is dirty, let's recover all the time */
if (!f2fs_skip_inode_update(inode, datasync)) {
f2fs_write_inode(inode, NULL);
 diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
 index 584483426584..9ca3d0a43d93 100644
 --- a/fs/f2fs/segment.c
 +++ b/fs/f2fs/segment.c
 @@ -230,6 +230,8 @@ static int __revoke_inmem_pages(struct inode 
 *inode,
  
lock_page(page);
  
 +  f2fs_wait_on_page_writeback(page, DATA, true);
 +
if (recover) {
struct dnode_of_data dn;
struct node_info ni;
 @@ -415,6 +417,7 @@ static int __commit_inmem_pages(struct inode 
 *inode)
/* drop all uncommitted pages */
__revoke_inmem_pages(inode, >inmem_pages, true, 
 false);
} else {

[f2fs-dev] [PATCH] f2fs: fix to let checkpoint guarantee atomic page persistence

[Chao Yu]

1. thread A: commit_inmem_pages submit data into block layer, but
haven't waited it writeback.
2. thread A: commit_inmem_pages update related node.
3. thread B: do checkpoint, flush all nodes to disk.
4. SPOR

Then, atomic file becomes corrupted since nodes is flushed before data.

This patch fixes to treat atomic page as checkpoint guaranteed one,
then in checkpoint, we can make sure all atomic page can be writebacked
with metadata of atomic file.

Signed-off-by: Chao Yu 
---
 fs/f2fs/data.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 778b23fce4fa..734be00fab3a 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -45,6 +45,8 @@ static bool __is_cp_guaranteed(struct page *page)
if (inode->i_ino == F2FS_META_INO(sbi) ||
inode->i_ino ==  F2FS_NODE_INO(sbi) ||
S_ISDIR(inode->i_mode) ||
+   (S_ISREG(inode->i_mode) &&
+   is_inode_flag_set(inode, FI_ATOMIC_FILE)) ||
is_cold_data(page))
return true;
return false;
-- 
2.17.0.391.g1f1cddd558b5


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel