Re: [f2fs-dev] [PATCH] f2fs: sepearte hot/cold in free nid

2018-04-19 Thread Chao Yu
On 2018/4/20 10:30, heyunlei wrote:
> 
> 
>> -Original Message-
>> From: Chao Yu [mailto:yuch...@huawei.com]
>> Sent: Friday, April 20, 2018 9:53 AM
>> To: jaeg...@kernel.org
>> Cc: linux-kernel@vger.kernel.org; linux-f2fs-de...@lists.sourceforge.net
>> Subject: [f2fs-dev] [PATCH] f2fs: sepearte hot/cold in free nid
>>
>> As most indirect node, dindirect node, and xattr node won't be updated
>> after they are created, but inode node and other direct node will change
>> more frequently, so store their nat entries mixedly in whole nat table
>> will suffer:
>> - fragment nat table soon due to different update rate
>> - more nat block update due to fragmented nat table
>>
> 
> BTW, should we enable this patch:  f2fs: reuse nids more aggressively?
> 
> I think it will decrease nat area fragment and will decrease io of nat?

For a fragmented nat table, there will be no different in between reusing
obsolete nid or allocating nid from next nat block.

IMO, in order to decrease nat block write, it needs to add more allocation
algorithm like a filesystem does, but firstly, I'd like to separate hot entry
from cold one.

Thanks,



RE: [f2fs-dev] [PATCH] f2fs: sepearte hot/cold in free nid

2018-04-19 Thread heyunlei


>-Original Message-
>From: Chao Yu [mailto:yuch...@huawei.com]
>Sent: Friday, April 20, 2018 9:53 AM
>To: jaeg...@kernel.org
>Cc: linux-kernel@vger.kernel.org; linux-f2fs-de...@lists.sourceforge.net
>Subject: [f2fs-dev] [PATCH] f2fs: sepearte hot/cold in free nid
>
>As most indirect node, dindirect node, and xattr node won't be updated
>after they are created, but inode node and other direct node will change
>more frequently, so store their nat entries mixedly in whole nat table
>will suffer:
>- fragment nat table soon due to different update rate
>- more nat block update due to fragmented nat table
>

BTW, should we enable this patch:  f2fs: reuse nids more aggressively?

I think it will decrease nat area fragment and will decrease io of nat?

>In order to solve above issue, we're trying to separate whole nat table to
>two part:
>a. Hot free nid area:
> - range: [nid #0, nid #x)
> - store node block address for
>   * inode node
>   * other direct node
>b. Cold free nid area:
> - range: [nid #x, max nid)
> - store node block address for
>   * indirect node
>   * dindirect node
>   * xattr node
>
>Allocation strategy example:
>
>Free nid: '-'
>Used nid: '='
>
>1. Initial status:
>Free Nids: 
>|---|
>   ^   ^   ^   
> ^
>Alloc Range:   |---|   
>|---|
>   hot_start   hot_end 
> cold_start  cold_end
>
>2. Free nids have ran out:
>Free Nids: 
>|===-===|
>   ^   ^   ^   
> ^
>Alloc Range:   |===|   
>|===|
>   hot_start   hot_end 
> cold_start  cold_end
>
>3. Expand hot/cold area range:
>Free Nids: 
>|===-===|
>   ^   ^   ^   
> ^
>Alloc Range:   |===|   
>|===|
>   hot_start   hot_end cold_start  
> cold_end
>
>4. Hot free nids have ran out:
>Free Nids: 
>|===-===|
>   ^   ^   ^   
> ^
>Alloc Range:   |===|   
>|===|
>   hot_start   hot_end cold_start  
> cold_end
>
>5. Expand hot area range, hot/cold area boundary has been fixed:
>Free Nids: 
>|===-===|
>   ^   ^   
> ^
>Alloc Range:   
>|===|===|
>   hot_start   hot_end(cold_start) 
> cold_end
>
>Run xfstests with generic/*:
>
>before
>node_write:169660
>cp_count:  60118
>node/cp2.82
>
>after:
>node_write:159145
>cp_count:  84501
>node/cp:   2.64
>
>Signed-off-by: Chao Yu 
>---
> fs/f2fs/checkpoint.c |   4 -
> fs/f2fs/debug.c  |   6 +-
> fs/f2fs/f2fs.h   |  19 +++-
> fs/f2fs/inode.c  |   2 +-
> fs/f2fs/namei.c  |   2 +-
> fs/f2fs/node.c   | 302 ---
> fs/f2fs/node.h   |  17 +--
> fs/f2fs/segment.c|   8 +-
> fs/f2fs/shrinker.c   |   3 +-
> fs/f2fs/xattr.c  |  10 +-
> 10 files changed, 221 insertions(+), 152 deletions(-)
>
>diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
>index 96785ffc6181..c17feec72c74 100644
>--- a/fs/f2fs/checkpoint.c
>+++ b/fs/f2fs/checkpoint.c
>@@ -1029,14 +1029,10 @@ int f2fs_sync_inode_meta(struct f2fs_sb_info *sbi)
> static void __prepare_cp_block(struct f2fs_sb_info *sbi)
> {
>   struct f2fs_checkpoint *ckpt = F2FS_CKPT(sbi);
>-  struct f2fs_nm_info *nm_i = NM_I(sbi);
>-  nid_t last_nid = nm_i->next_scan_nid;
>
>-  next_free_nid(sbi, &last_nid);
>   ckpt->valid_block_count = cpu_to_le64(valid_user_blocks(sbi));
>   ckpt->valid_node_count = cpu_to_le32(valid_node_count(sbi));
>   ckpt->valid_inode_count = cpu_to_