Re: [f2fs-dev] [PATCH v4] f2fs: compress: avoid duplicate counting of valid blocks when read compressed file
On 2021/8/12 11:18, Fengnan Chang wrote:
Since cluster is basic unit of compression, one cluster is compressed or
not, so we can calculate valid blocks only for first page in cluster,
the other pages just skip.
Signed-off-by: Fengnan Chang
---
fs/f2fs/data.c | 24 +++-
1 file changed, 19 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index df5e8d8c654e..fc0115a61082 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -2299,6 +2299,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
.nr_rpages = 0,
.nr_cpages = 0,
};
+ pgoff_t nc_cluster_idx = NULL_CLUSTER;
#endif
unsigned nr_pages = rac ? readahead_count(rac) : 1;
unsigned max_nr_pages = nr_pages;
@@ -2328,15 +2329,27 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
Can we get rid of this?
if (ret)
goto set_error_page;
}
- ret = f2fs_is_compressed_cluster(inode, page->index);
- if (ret < 0)
- goto set_error_page;
- else if (!ret)
- goto read_single_page;
+ if (cc.cluster_idx == NULL_CLUSTER) {
+ if (nc_cluster_idx ==
+ page->index >> cc.log_cluster_size) {
+ goto read_single_page;
+ }
+
+ ret = f2fs_is_compressed_cluster(inode,
page->index);
+ if (ret < 0)
+ goto set_error_page;
+ else if (!ret) {
+ nc_cluster_idx =
+ page->index >>
cc.log_cluster_size;
+ goto read_single_page;
+ }
+nc_cluster_idx = NULL_CLUSTER;
+ }
ret = f2fs_init_compress_ctx();
if (ret)
goto set_error_page;
@@ -2373,6 +2386,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
Ditto,
Thanks,
}
}
#endif
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH v4] f2fs: compress: avoid duplicate counting of valid blocks when read compressed file
Since cluster is basic unit of compression, one cluster is compressed or
not, so we can calculate valid blocks only for first page in cluster,
the other pages just skip.
Signed-off-by: Fengnan Chang
---
fs/f2fs/data.c | 24 +++-
1 file changed, 19 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index df5e8d8c654e..fc0115a61082 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -2299,6 +2299,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
.nr_rpages = 0,
.nr_cpages = 0,
};
+ pgoff_t nc_cluster_idx = NULL_CLUSTER;
#endif
unsigned nr_pages = rac ? readahead_count(rac) : 1;
unsigned max_nr_pages = nr_pages;
@@ -2328,15 +2329,27 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
if (ret)
goto set_error_page;
}
- ret = f2fs_is_compressed_cluster(inode, page->index);
- if (ret < 0)
- goto set_error_page;
- else if (!ret)
- goto read_single_page;
+ if (cc.cluster_idx == NULL_CLUSTER) {
+ if (nc_cluster_idx ==
+ page->index >> cc.log_cluster_size) {
+ goto read_single_page;
+ }
+
+ ret = f2fs_is_compressed_cluster(inode,
page->index);
+ if (ret < 0)
+ goto set_error_page;
+ else if (!ret) {
+ nc_cluster_idx =
+ page->index >>
cc.log_cluster_size;
+ goto read_single_page;
+ }
+ nc_cluster_idx = NULL_CLUSTER;
+ }
ret = f2fs_init_compress_ctx();
if (ret)
goto set_error_page;
@@ -2373,6 +2386,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
}
}
#endif
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v3] f2fs: compress: avoid duplicate counting of valid blocks when read compressed file
mistake, forget this...
On 2021/8/12 11:05, Fengnan Chang wrote:
Since cluster is basic unit of compression, one cluster is compressed or
not, so we can calculate valid blocks only for first page in cluster,
the other pages just skip.
Signed-off-by: Fengnan Chang
---
fs/f2fs/data.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index df5e8d8c654e..b06a0e5de1f1 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -2299,6 +2299,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
.nr_rpages = 0,
.nr_cpages = 0,
};
+ pgoff_t nc_cluster_idx = NULL_CLUSTER;
#endif
unsigned nr_pages = rac ? readahead_count(rac) : 1;
unsigned max_nr_pages = nr_pages;
@@ -2328,15 +2329,26 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
if (ret)
goto set_error_page;
}
- ret = f2fs_is_compressed_cluster(inode, page->index);
- if (ret < 0)
- goto set_error_page;
- else if (!ret)
- goto read_single_page;
+ if (cc.cluster_idx == NULL_CLUSTER) {
+ if (nc_cluster_idx != NULL_CLUSTER &&
+ nc_cluster_idx == page->index >>
cc.log_cluster_size) {
+ goto read_single_page;
+ }
+
+ ret = f2fs_is_compressed_cluster(inode,
page->index);
+ if (ret < 0)
+ goto set_error_page;
+ else if (!ret) {
+ nc_cluster_idx = page->index >>
cc.log_cluster_size;
+ goto read_single_page;
+ }
+nc_cluster_idx = NULL_CLUSTER;
+ }
ret = f2fs_init_compress_ctx();
if (ret)
goto set_error_page;
@@ -2373,6 +2385,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
}
}
#endif
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH v3] f2fs: compress: avoid duplicate counting of valid blocks when read compressed file
Since cluster is basic unit of compression, one cluster is compressed or
not, so we can calculate valid blocks only for first page in cluster,
the other pages just skip.
Signed-off-by: Fengnan Chang
---
fs/f2fs/data.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index df5e8d8c654e..b06a0e5de1f1 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -2299,6 +2299,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
.nr_rpages = 0,
.nr_cpages = 0,
};
+ pgoff_t nc_cluster_idx = NULL_CLUSTER;
#endif
unsigned nr_pages = rac ? readahead_count(rac) : 1;
unsigned max_nr_pages = nr_pages;
@@ -2328,15 +2329,26 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
if (ret)
goto set_error_page;
}
- ret = f2fs_is_compressed_cluster(inode, page->index);
- if (ret < 0)
- goto set_error_page;
- else if (!ret)
- goto read_single_page;
+ if (cc.cluster_idx == NULL_CLUSTER) {
+ if (nc_cluster_idx != NULL_CLUSTER &&
+ nc_cluster_idx == page->index >>
cc.log_cluster_size) {
+ goto read_single_page;
+ }
+
+ ret = f2fs_is_compressed_cluster(inode,
page->index);
+ if (ret < 0)
+ goto set_error_page;
+ else if (!ret) {
+ nc_cluster_idx = page->index >>
cc.log_cluster_size;
+ goto read_single_page;
+ }
+ nc_cluster_idx = NULL_CLUSTER;
+ }
ret = f2fs_init_compress_ctx();
if (ret)
goto set_error_page;
@@ -2373,6 +2385,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
_block_in_bio,
rac != NULL, false);
f2fs_destroy_compress_ctx(, false);
+ nc_cluster_idx = NULL_CLUSTER;
}
}
#endif
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] f2fs do DIO write make file corruption
在 2021/8/11 20:57, Chao Yu 写道: On 2021/8/11 11:28, Wu Bo wrote: 在 2021/8/11 11:03, Chao Yu 写道: On 2021/8/11 10:48, Wu Bo wrote: I use the following command to create a file, the file may got corruption: f2fs_io write 2 0 512 inc_num dio $path And when I use bio or to set the chunk size to 1 block, the file is normal. The commands as following: f2fs_io write 2 0 512 inc_num buffered $path f2fs_io write 1 0 512 inc_num dio $path I find this bug on old kernel version 4.14.117, and not find on version 4.19.152. So this bug is fixed. Can anyone can tell me which patch fixed this bug? Not sure, https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y=7bae8b6b73e46c307fa355ce086800b7ad6610f8 I didn't see this patch in 4.14 stable kernel of mailine, so do you mean 4.14 kernel maintained by Android? If so, f2fs codes in between 4.14 and 4.19 Android kernel are almost the same, see below link: https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/ Sorry, it's our old Android project code.So the kernel version may have no reference value. This project code last sync to Android common kernel is: 9f892496fc0b:f2fs: readahead encrypted block during GC[2018-09-05] The code link is: https://android.googlesource.com/kernel/common/+/9f892496fc0b And after that, we applied some fixup patches. Thanks, This patch is applied. The issue occurs when f2fs dio try to preallocate multiple blocks and got scattered disk blocks. The log as following: my_f2fs_io-13425 [002] 395.583637: el0_irq_naked: 1:type:1,ino:40132,off:768,old_blk:0,new_blk:185764 my_f2fs_io-13425 [002] 395.583710: el0_irq_naked: 1:type:1,ino:40132,off:769,old_blk:0,new_blk:205824 my_f2fs_io-13425 [002] 395.583721: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len = 0x2, err = 0 my_f2fs_io-13425 [002] 395.583735: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len = 0x1, err = 0 my_f2fs_io-13425 [002] 395.583741: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 769, start blkaddr = 0x32400, len = 0x1, err = 0 And if the blocks are continuously, the file data is normal: my_f2fs_io-13425 [002] 395.584037: el0_irq_naked: 1:type:1,ino:40132,off:770,old_blk:0,new_blk:205825 my_f2fs_io-13425 [002] 395.584066: el0_irq_naked: 1:type:1,ino:40132,off:771,old_blk:0,new_blk:205826 my_f2fs_io-13425 [002] 395.584077: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len = 0x2, err = 0 my_f2fs_io-13425 [002] 395.584091: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len = 0x2, err = 0 Thanks, ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH 2/2] fs: Don't create discard thread when device not support realtime discard
On 2021/8/11 21:18, Yangtao Li wrote:
From: Fengnan Chang
Don't create discard thread when device not support realtime discard.
Signed-off-by: Fengnan Chang
Signed-off-by: Yangtao Li
---
fs/f2fs/segment.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 363779a4402d..bc4ac46f3041 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -2161,6 +2161,8 @@ static int create_discard_cmd_control(struct f2fs_sb_info
*sbi)
init_waitqueue_head(>discard_wait_queue);
SM_I(sbi)->dcc_info = dcc;
init_thread:
+ if (!f2fs_realtime_discard_enable(sbi))
How about below case:
- mount -o nodiscard
- mount -o remount,discard
Thanks,
+ return err;
dcc->f2fs_issue_discard = kthread_run(issue_discard_thread, sbi,
"f2fs_discard-%u:%u", MAJOR(dev), MINOR(dev));
if (IS_ERR(dcc->f2fs_issue_discard)) {
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v2] f2fs: warn on when fsck flag is set
On 2021/8/11 21:30, Yangtao Li wrote:
SBI_NEED_FSCK is an indicator that fsck.f2fs needs to be triggered,
this flag is set in too many places. For some scenes that are not very
reproducible,
adding stack information will help locate the problem.
Signed-off-by: Yangtao Li
---
v2:
-convert to WARN_ON
-one more blank
fs/f2fs/f2fs.h | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 67faa43cc141..2e2294234c0d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -33,7 +33,7 @@
#else
#define f2fs_bug_on(sbi, condition) \
do {\
- if (WARN_ON(condition)) \
+ if ((condition))\
Why removing WARN_ON()?
set_sbi_flag(sbi, SBI_NEED_FSCK); \
} while (0)
#endif
@@ -1999,6 +1999,8 @@ static inline bool is_sbi_flag_set(struct f2fs_sb_info
*sbi, unsigned int type)
static inline void set_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
{
+ WARN_ON(type == SBI_NEED_FSCK);
Again, please remove one unnecessary blank character before SBI_NEED_FSCK.
WARN_ON_ONCE(type == SBI_NEED_FSCK)
+
No need to add blank line.
Thanks,
set_bit(type, >s_flag);
}
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH v3] f2fs: warn on when fsck flag is set
SBI_NEED_FSCK is an indicator that fsck.f2fs needs to be triggered,
this flag is set in too many places. For some scenes that are not very
reproducible, adding stack information will help locate the problem.
Signed-off-by: Yangtao Li
---
v3:
-Remove extra parentheses
v2:
-Convert to WARN_ON
-One more blank
fs/f2fs/f2fs.h | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 67faa43cc141..0d08e31214a3 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -33,7 +33,7 @@
#else
#define f2fs_bug_on(sbi, condition)\
do {\
- if (WARN_ON(condition)) \
+ if (condition) \
set_sbi_flag(sbi, SBI_NEED_FSCK); \
} while (0)
#endif
@@ -1999,6 +1999,8 @@ static inline bool is_sbi_flag_set(struct f2fs_sb_info
*sbi, unsigned int type)
static inline void set_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
{
+ WARN_ON(type == SBI_NEED_FSCK);
+
set_bit(type, >s_flag);
}
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH 1/2] f2fs: fix potential memory leaks in create_discard_cmd_control
On 2021/8/11 21:18, Yangtao Li wrote: Release f2fs_issue_discard every time it is destroyed, otherwise it will cause memory leaks when remounting. I didn't get the problem here, could you please explain a bit more about details? Thanks, Signed-off-by: Yangtao Li --- fs/f2fs/segment.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index afe20c5c3c08..363779a4402d 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -1741,6 +1741,7 @@ void f2fs_stop_discard_thread(struct f2fs_sb_info *sbi) dcc->f2fs_issue_discard = NULL; kthread_stop(discard_thread); + put_task_struct(discard_thread); } } @@ -2168,6 +2169,7 @@ static int create_discard_cmd_control(struct f2fs_sb_info *sbi) SM_I(sbi)->dcc_info = NULL; return err; } + get_task_struct(dcc->f2fs_issue_discard); return err; } ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH v2] f2fs: warn on when fsck flag is set
SBI_NEED_FSCK is an indicator that fsck.f2fs needs to be triggered,
this flag is set in too many places. For some scenes that are not very
reproducible,
adding stack information will help locate the problem.
Signed-off-by: Yangtao Li
---
v2:
-convert to WARN_ON
-one more blank
fs/f2fs/f2fs.h | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 67faa43cc141..2e2294234c0d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -33,7 +33,7 @@
#else
#define f2fs_bug_on(sbi, condition)\
do {\
- if (WARN_ON(condition)) \
+ if ((condition))\
set_sbi_flag(sbi, SBI_NEED_FSCK); \
} while (0)
#endif
@@ -1999,6 +1999,8 @@ static inline bool is_sbi_flag_set(struct f2fs_sb_info
*sbi, unsigned int type)
static inline void set_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
{
+ WARN_ON(type == SBI_NEED_FSCK);
+
set_bit(type, >s_flag);
}
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH 2/2] fs: Don't create discard thread when device not support realtime discard
From: Fengnan Chang
Don't create discard thread when device not support realtime discard.
Signed-off-by: Fengnan Chang
Signed-off-by: Yangtao Li
---
fs/f2fs/segment.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 363779a4402d..bc4ac46f3041 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -2161,6 +2161,8 @@ static int create_discard_cmd_control(struct f2fs_sb_info
*sbi)
init_waitqueue_head(>discard_wait_queue);
SM_I(sbi)->dcc_info = dcc;
init_thread:
+ if (!f2fs_realtime_discard_enable(sbi))
+ return err;
dcc->f2fs_issue_discard = kthread_run(issue_discard_thread, sbi,
"f2fs_discard-%u:%u", MAJOR(dev), MINOR(dev));
if (IS_ERR(dcc->f2fs_issue_discard)) {
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH 1/2] f2fs: fix potential memory leaks in create_discard_cmd_control
Release f2fs_issue_discard every time it is destroyed, otherwise it will cause memory leaks when remounting. Signed-off-by: Yangtao Li --- fs/f2fs/segment.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index afe20c5c3c08..363779a4402d 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -1741,6 +1741,7 @@ void f2fs_stop_discard_thread(struct f2fs_sb_info *sbi) dcc->f2fs_issue_discard = NULL; kthread_stop(discard_thread); + put_task_struct(discard_thread); } } @@ -2168,6 +2169,7 @@ static int create_discard_cmd_control(struct f2fs_sb_info *sbi) SM_I(sbi)->dcc_info = NULL; return err; } + get_task_struct(dcc->f2fs_issue_discard); return err; } -- 2.32.0 ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v2] f2fs: compress: avoid duplicate counting of valid blocks when read compressed file
On 2021/8/11 16:40, Fengnan Chang wrote:
Since cluster is basic unit of compression, one cluster is compressed or
not, so we can calculate valid blocks only for first page in cluster,
the other pages just skip.
Signed-off-by: Fengnan Chang
---
fs/f2fs/compress.c | 1 +
fs/f2fs/data.c | 21 -
fs/f2fs/f2fs.h | 1 +
3 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index 7dbfd6965b97..71768f15752a 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -152,6 +152,7 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc,
bool reuse)
cc->rpages = NULL;
cc->nr_rpages = 0;
cc->nr_cpages = 0;
+ cc->nc_cluster_idx = NULL_CLUSTER;
if (!reuse)
cc->cluster_idx = NULL_CLUSTER;
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index df5e8d8c654e..3ee1a88d8400 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -2294,6 +2294,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
.log_cluster_size = F2FS_I(inode)->i_log_cluster_size,
.cluster_size = F2FS_I(inode)->i_cluster_size,
.cluster_idx = NULL_CLUSTER,
+ .nc_cluster_idx = NULL_CLUSTER,
.rpages = NULL,
.cpages = NULL,
.nr_rpages = 0,
pgoff_t nc_cluster_idx = NULL_CLUSTER;
@@ -2331,12 +2332,22 @@ static int f2fs_mpage_readpages(struct inode *inode,
if (ret)
goto set_error_page;
}
- ret = f2fs_is_compressed_cluster(inode, page->index);
- if (ret < 0)
- goto set_error_page;
- else if (!ret)
- goto read_single_page;
+ if (cc.cluster_idx == NULL_CLUSTER) {
+ if (cc.nc_cluster_idx != NULL_CLUSTER &&
+ cc.nc_cluster_idx == page->index >>
cc.log_cluster_size) {
if (nc_cluster_idx ==
page->index >> cc.log_cluster_size)
+ goto read_single_page;
+ }
+
+ ret = f2fs_is_compressed_cluster(inode,
page->index);
+ if (ret < 0)
+ goto set_error_page;
+ else if (!ret) {
+ cc.nc_cluster_idx = page->index >>
cc.log_cluster_size;
nc_cluster_idx =
page->index >>
cc.log_cluster_size;
+ goto read_single_page;
+ }
+cc.nc_cluster_idx = NULL_CLUSTER;
+ }
ret = f2fs_init_compress_ctx();
if (ret)
goto set_error_page;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index e97b4d8c5efc..e9b6890a3f19 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1471,6 +1471,7 @@ struct compress_data {
struct compress_ctx {
struct inode *inode;/* inode the context belong to */
pgoff_t cluster_idx;/* cluster index number */
+ pgoff_t nc_cluster_idx; /* cluster index number for
non-compressed cluster use*/
We only use this field in read flow, please add a local variable to record
last cluster index number of non-compressed cluster.
Thanks,
unsigned int cluster_size; /* page count in cluster */
unsigned int log_cluster_size; /* log of cluster size */
struct page **rpages; /* pages store raw data in cluster */
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] f2fs do DIO write make file corruption
On 2021/8/11 11:28, Wu Bo wrote: 在 2021/8/11 11:03, Chao Yu 写道: On 2021/8/11 10:48, Wu Bo wrote: I use the following command to create a file, the file may got corruption: f2fs_io write 2 0 512 inc_num dio $path And when I use bio or to set the chunk size to 1 block, the file is normal. The commands as following: f2fs_io write 2 0 512 inc_num buffered $path f2fs_io write 1 0 512 inc_num dio $path I find this bug on old kernel version 4.14.117, and not find on version 4.19.152. So this bug is fixed. Can anyone can tell me which patch fixed this bug? Not sure, https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y=7bae8b6b73e46c307fa355ce086800b7ad6610f8 I didn't see this patch in 4.14 stable kernel of mailine, so do you mean 4.14 kernel maintained by Android? If so, f2fs codes in between 4.14 and 4.19 Android kernel are almost the same, see below link: https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git/ Thanks, This patch is applied. The issue occurs when f2fs dio try to preallocate multiple blocks and got scattered disk blocks. The log as following: my_f2fs_io-13425 [002] 395.583637: el0_irq_naked: 1:type:1,ino:40132,off:768,old_blk:0,new_blk:185764 my_f2fs_io-13425 [002] 395.583710: el0_irq_naked: 1:type:1,ino:40132,off:769,old_blk:0,new_blk:205824 my_f2fs_io-13425 [002] 395.583721: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len = 0x2, err = 0 my_f2fs_io-13425 [002] 395.583735: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 768, start blkaddr = 0x2d5a4, len = 0x1, err = 0 my_f2fs_io-13425 [002] 395.583741: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 769, start blkaddr = 0x32400, len = 0x1, err = 0 And if the blocks are continuously, the file data is normal: my_f2fs_io-13425 [002] 395.584037: el0_irq_naked: 1:type:1,ino:40132,off:770,old_blk:0,new_blk:205825 my_f2fs_io-13425 [002] 395.584066: el0_irq_naked: 1:type:1,ino:40132,off:771,old_blk:0,new_blk:205826 my_f2fs_io-13425 [002] 395.584077: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len = 0x2, err = 0 my_f2fs_io-13425 [002] 395.584091: f2fs_map_blocks: dev = (259,23), ino = 40132, file offset = 770, start blkaddr = 0x32401, len = 0x2, err = 0 Thanks, ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH] f2fs: warn on when fsck flag is set
On 2021/8/11 18:16, Yangtao Li wrote:
SBI_NEED_FSCK is an indicator that fsck.f2fs needs to be triggered,
this flag is set in too many places. For some scenes that are not very
reproducible, adding stack information will help locate the problem.
Signed-off-by: Yangtao Li
---
fs/f2fs/f2fs.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 67faa43cc141..7e185446d2df 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1999,6 +1999,7 @@ static inline bool is_sbi_flag_set(struct f2fs_sb_info
*sbi, unsigned int type)
static inline void set_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
{
+ WARN_ON_ONCE(type == SBI_NEED_FSCK);
One more blank character.
set_bit(type, >s_flag);
}
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v2] f2fs: introduce blk_alloc_mode mount option
On 2021/8/11 13:41, Daeho Jeong wrote:
On Tue, Aug 10, 2021 at 6:58 PM Chao Yu wrote:
On 2021/8/10 2:54, Daeho Jeong wrote:
From: Daeho Jeong
Added a mount option to control block allocation mode for filesystem
developer to simulate filesystem fragmentation and after-GC situation
for experimental reasons to understand the filesystem behaviors well
At a glance, I think this functionality can be used to initialize filesystem
status as fragmented one, which is commonly used in the first step of aging
test, in order to make filesystem fragmented, aging program needs to fill
file or block sequentially and remove them at fixed interval, that takes a
lot of time. IMO, it will be great to support this by filesystem naturally
to save time of aging filessytem, rather just for experimental reasons to
understand filesystem behavior.
Yes, that's what I meant~ :)
Maybe we can add a new mode for filesystem allocation mode except adaptive
and lfs mode, maybe "fragment" mode... :P
Sounds good~ Could you suggest the names for both blk_random and seg_random.
We need seg_random mode,too, in order to simulate after-GC condition.
mode=fragment:block and mode=fragment:segment?
under the severe condition. This supports "normal", "seg_random" and
"blk_random:" options.
"normal" (default): f2fs allocates blocks in the normal way.
"seg_random": f2fs allocates a new segment in ramdom position.
With this, we can simulate the after-GC condition.
"blk_random:": We can make f2fs allocate only 1.. blocks
in a row and forcibly change the segment randomly.
With this, the newly allocated blocks will be scatter
throughout the whole partition and we can simulate
filesystem fragmentation condition.
Signed-off-by: Daeho Jeong
---
v2: changed the initialization way.
---
Documentation/filesystems/f2fs.rst | 16 ++
fs/f2fs/f2fs.h | 20 +
fs/f2fs/gc.c | 5 +++-
fs/f2fs/segment.c | 12
fs/f2fs/super.c| 47 ++
5 files changed, 99 insertions(+), 1 deletion(-)
diff --git a/Documentation/filesystems/f2fs.rst
b/Documentation/filesystems/f2fs.rst
index ff9e7cc97c65..a0ca963fda20 100644
--- a/Documentation/filesystems/f2fs.rst
+++ b/Documentation/filesystems/f2fs.rst
@@ -312,6 +312,22 @@ inlinecrypt When possible, encrypt/decrypt
the contents of encrypted
Documentation/block/inline-encryption.rst.
atgc Enable age-threshold garbage collection, it
provides high
effectiveness and efficiency on background GC.
+blk_alloc_mode=%s Control block allocation mode. This is a developer option
+ for experiments to simulate filesystem fragmentation and
+ after-GC situation. The developers use this mode to
understand
+ filesystem fragmentation and after-GC condition well, and
+ eventually get the insight to handle them better.
+ This supports "normal", "seg_random" and
"blk_random:" modes.
+ In "normal" mode (default), f2fs allocates blocks in the
normal way.
+ In "seg_random", f2fs allocates a new segment in ramdom
position.
+ With this, we can simulate the after-GC condition.
+ In "blk_random:", we can make f2fs allocate only
1..
+ blocks in a row and forcibly change the segment randomly.
+ You can set the within 1 .. 512 number.
+ With this, the newly allocated blocks will be scatter
throughout
+ the whole partition and we can simulate filesystem
fragmentation
+ condition. Please, use this option for your experiments
and we
+ strongly recommand a filesystem format after using this
option.
Debugfs Entries
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index fccee18ab776..aed09e8c0fce 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -155,6 +155,9 @@ struct f2fs_mount_info {
int compress_mode; /* compression mode */
unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN]; /*
extensions */
unsigned char noextensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN]; /*
extensions */
+
+ int blk_alloc_mode; /* block allocation mode */
+ int blk_alloc_random_max; /* the maximum chunk size for block
random allocation mode */
};
#define F2FS_FEATURE_ENCRYPT0x0001
@@ -1740,6 +1743,8 @@ struct f2fs_sb_info {
unsigned long seq_file_ra_mul; /* multiplier for ra_pages of
seq. files in
[f2fs-dev] [PATCH] f2fs: warn on when fsck flag is set
SBI_NEED_FSCK is an indicator that fsck.f2fs needs to be triggered,
this flag is set in too many places. For some scenes that are not very
reproducible, adding stack information will help locate the problem.
Signed-off-by: Yangtao Li
---
fs/f2fs/f2fs.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 67faa43cc141..7e185446d2df 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1999,6 +1999,7 @@ static inline bool is_sbi_flag_set(struct f2fs_sb_info
*sbi, unsigned int type)
static inline void set_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
{
+ WARN_ON_ONCE(type == SBI_NEED_FSCK);
set_bit(type, >s_flag);
}
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v4 3/6] block: change ioprio_valid() to an inline function
On 11/08/2021 10:51, Damien Le Moal wrote: > On 2021/08/11 16:56, Johannes Thumshirn wrote: >> On 11/08/2021 05:37, Damien Le Moal wrote: >>> Change the ioprio_valid() macro in include/usapi/linux/ioprio.h to an >>> uapi ~^ >> >>> inline function declared on the kernel side in include/linux/ioprio.h. >>> Also improve checks on the class value by checking the upper bound >>> value. >> >> But I think it needs to stay in include/uapi/linux/ioprio.h as it's there >> since the 2.6.x days (I've checked back to v2.6.39.4) so the chance of >> user-space using it is quite high. > > include/uapi/linux/ioprio.h is being introduced with kernel 5.15. This user > header did not exist now and in previous kernels. include/linux/ioprio.h has > been around for a while though, but that is a kernel header, not an > application > header. > > Ah ok this was the now or never one, I thought it was about one of the constants. Sorry for the noise. ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v4 3/6] block: change ioprio_valid() to an inline function
On 2021/08/11 16:56, Johannes Thumshirn wrote: > On 11/08/2021 05:37, Damien Le Moal wrote: >> Change the ioprio_valid() macro in include/usapi/linux/ioprio.h to an >> uapi ~^ > >> inline function declared on the kernel side in include/linux/ioprio.h. >> Also improve checks on the class value by checking the upper bound >> value. > > But I think it needs to stay in include/uapi/linux/ioprio.h as it's there > since the 2.6.x days (I've checked back to v2.6.39.4) so the chance of > user-space using it is quite high. include/uapi/linux/ioprio.h is being introduced with kernel 5.15. This user header did not exist now and in previous kernels. include/linux/ioprio.h has been around for a while though, but that is a kernel header, not an application header. -- Damien Le Moal Western Digital Research ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH v2] f2fs: compress: avoid duplicate counting of valid blocks when read compressed file
Since cluster is basic unit of compression, one cluster is compressed or
not, so we can calculate valid blocks only for first page in cluster,
the other pages just skip.
Signed-off-by: Fengnan Chang
---
fs/f2fs/compress.c | 1 +
fs/f2fs/data.c | 21 -
fs/f2fs/f2fs.h | 1 +
3 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index 7dbfd6965b97..71768f15752a 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -152,6 +152,7 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc,
bool reuse)
cc->rpages = NULL;
cc->nr_rpages = 0;
cc->nr_cpages = 0;
+ cc->nc_cluster_idx = NULL_CLUSTER;
if (!reuse)
cc->cluster_idx = NULL_CLUSTER;
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index df5e8d8c654e..3ee1a88d8400 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -2294,6 +2294,7 @@ static int f2fs_mpage_readpages(struct inode *inode,
.log_cluster_size = F2FS_I(inode)->i_log_cluster_size,
.cluster_size = F2FS_I(inode)->i_cluster_size,
.cluster_idx = NULL_CLUSTER,
+ .nc_cluster_idx = NULL_CLUSTER,
.rpages = NULL,
.cpages = NULL,
.nr_rpages = 0,
@@ -2331,12 +2332,22 @@ static int f2fs_mpage_readpages(struct inode *inode,
if (ret)
goto set_error_page;
}
- ret = f2fs_is_compressed_cluster(inode, page->index);
- if (ret < 0)
- goto set_error_page;
- else if (!ret)
- goto read_single_page;
+ if (cc.cluster_idx == NULL_CLUSTER) {
+ if (cc.nc_cluster_idx != NULL_CLUSTER &&
+ cc.nc_cluster_idx == page->index >>
cc.log_cluster_size) {
+ goto read_single_page;
+ }
+
+ ret = f2fs_is_compressed_cluster(inode,
page->index);
+ if (ret < 0)
+ goto set_error_page;
+ else if (!ret) {
+ cc.nc_cluster_idx = page->index >>
cc.log_cluster_size;
+ goto read_single_page;
+ }
+ cc.nc_cluster_idx = NULL_CLUSTER;
+ }
ret = f2fs_init_compress_ctx();
if (ret)
goto set_error_page;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index e97b4d8c5efc..e9b6890a3f19 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1471,6 +1471,7 @@ struct compress_data {
struct compress_ctx {
struct inode *inode;/* inode the context belong to */
pgoff_t cluster_idx;/* cluster index number */
+ pgoff_t nc_cluster_idx; /* cluster index number for
non-compressed cluster use*/
unsigned int cluster_size; /* page count in cluster */
unsigned int log_cluster_size; /* log of cluster size */
struct page **rpages; /* pages store raw data in cluster */
--
2.32.0
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v4 3/6] block: change ioprio_valid() to an inline function
On 11/08/2021 05:37, Damien Le Moal wrote: > Change the ioprio_valid() macro in include/usapi/linux/ioprio.h to an >uapi ~^ > inline function declared on the kernel side in include/linux/ioprio.h. > Also improve checks on the class value by checking the upper bound > value. But I think it needs to stay in include/uapi/linux/ioprio.h as it's there since the 2.6.x days (I've checked back to v2.6.39.4) so the chance of user-space using it is quite high. ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [Bug 206057] 5.5.0-rc2-next: f2fs is extremely slow, with ext4 system works well
https://bugzilla.kernel.org/show_bug.cgi?id=206057 David Heidelberg (okias) (da...@ixit.cz) changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |CODE_FIX --- Comment #10 from David Heidelberg (okias) (da...@ixit.cz) --- Currently Nexus 7 2012 works on F2FS well, closing. -- You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug. ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
