Re: [f2fs-dev] [PATCH] f2fs-tools: give 6 sections for overprovision buffer
On 2024/4/3 7:54, Jaegeuk Kim wrote: This addresses high GC cost at runtime. Signed-off-by: Jaegeuk Kim Reviewed-by: Chao Yu Thanks, ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH] f2fs-tools: print extension list properly
On 2024/4/8 21:11, Sheng Yong wrote: The "hot file extensions" list does not print properly. **Before** extension_count [0x 23 : 35] cold file extentsions [mp wm og jp ] [avi m4v m4p mkv ] [mov webmwav m4a ] [3gp opusflacgif ] [png svg webpjar ] [deb iso gz xz ] [zst pdf pyc ttc ] [ttf exe apk cnt ] [exo odexvdex] hot_ext_count [0x 1 : 1] hot file extentsions db ] cp_payload [0x 0 : 0] **After** extension_count [0x 23 : 35] cold file extentsions [mp wm og jp ] [avi m4v m4p mkv ] [mov webmwav m4a ] [3gp opusflacgif ] [png svg webpjar ] [deb iso gz xz ] [zst pdf pyc ttc ] [ttf exe apk cnt ] [exo odexvdex] hot_ext_count [0x 1 : 1] hot file extentsions [db ] cp_payload [0x 0 : 0] Signed-off-by: Sheng Yong Reviewed-by: Chao Yu Thanks, ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH] f2fs: don't set RO when shutting down f2fs
On 2024/4/5 3:52, Jaegeuk Kim wrote:
Shutdown does not check the error of thaw_super due to readonly, which
causes a deadlock like below.
f2fs_ioc_shutdown(F2FS_GOING_DOWN_FULLSYNC)issue_discard_thread
- bdev_freeze
- freeze_super
- f2fs_stop_checkpoint()
- f2fs_handle_critical_error - sb_start_write
- set RO - waiting
- bdev_thaw
- thaw_super_locked
- return -EINVAL, if sb_rdonly()
- f2fs_stop_discard_thread
-> wait for kthread_stop(discard_thread);
Reported-by: "Light Hsieh (謝明燈)"
Signed-off-by: Jaegeuk Kim
---
fs/f2fs/super.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index df9765b41dac..ba6288e870c5 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4135,9 +4135,16 @@ void f2fs_handle_critical_error(struct f2fs_sb_info
*sbi, unsigned char reason,
if (shutdown)
set_sbi_flag(sbi, SBI_IS_SHUTDOWN);
- /* continue filesystem operators if errors=continue */
- if (continue_fs || f2fs_readonly(sb))
+ /*
+* Continue filesystem operators if errors=continue. Should not set
+* RO by shutdown, since RO bypasses thaw_super which can hang the
+* system.
+*/
+ if (continue_fs || f2fs_readonly(sb) ||
+ reason == STOP_CP_REASON_SHUTDOWN) {
+ f2fs_warn(sbi, "Stopped filesystem due to readon: %d", reason);
return;
Do we need to set RO after bdev_thaw() in f2fs_do_shutdown()?
Thanks,
+ }
f2fs_warn(sbi, "Remounting filesystem read-only");
/*
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH] resize.f2fs: get value from new sb during rebuilding cp
On 2024/4/9 2:34, Jaegeuk Kim wrote: On 04/08, Sheng Yong wrote: Althrough old and new sb have the same value for now, it would be better to build new checkpoint according to new sb. May need to add assert, if they're different? We could add assert here, but I think it's not that necessary: 1. rebuild_checkpoint is only called by resize, and new_sb is copied directly from original sb without any changes of these basic attributes. 2. for now, new_sb has the same attributes/members with the original one. If those attributes are allowed to get changed in the future, the assert needs to be removed. So how about adding a new helper to check and show the difference between the new and original sb? many thanks, shengyong Signed-off-by: Sheng Yong --- fsck/resize.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fsck/resize.c b/fsck/resize.c index 049ddd3..1b4ae85 100644 --- a/fsck/resize.c +++ b/fsck/resize.c @@ -481,7 +481,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi, set_cp(overprov_segment_count, get_cp(rsvd_segment_count)); set_cp(overprov_segment_count, get_cp(overprov_segment_count) + - 2 * get_sb(segs_per_sec)); + 2 * get_newsb(segs_per_sec)); DBG(0, "Info: Overprovision ratio = %.3lf%%\n", c.new_overprovision); DBG(0, "Info: Overprovision segments = %u (GC reserved = %u)\n", @@ -551,11 +551,12 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi, cpu_to_le32(crc); /* Write a new checkpoint in the other set */ - new_cp_blk_no = old_cp_blk_no = get_sb(cp_blkaddr); + old_cp_blk_no = get_sb(cp_blkaddr); + new_cp_blk_no = get_newsb(cp_blkaddr); if (sbi->cur_cp == 2) old_cp_blk_no += 1 << get_sb(log_blocks_per_seg); else - new_cp_blk_no += 1 << get_sb(log_blocks_per_seg); + new_cp_blk_no += 1 << get_newsb(log_blocks_per_seg); /* write first cp */ ret = dev_write_block(new_cp, new_cp_blk_no++); -- 2.40.1 ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH] resize.f2fs: get value from new sb during rebuilding cp
On 04/08, Sheng Yong wrote: > Althrough old and new sb have the same value for now, it would be better > to build new checkpoint according to new sb. May need to add assert, if they're different? > > Signed-off-by: Sheng Yong > --- > fsck/resize.c | 7 --- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/fsck/resize.c b/fsck/resize.c > index 049ddd3..1b4ae85 100644 > --- a/fsck/resize.c > +++ b/fsck/resize.c > @@ -481,7 +481,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi, > set_cp(overprov_segment_count, get_cp(rsvd_segment_count)); > > set_cp(overprov_segment_count, get_cp(overprov_segment_count) + > - 2 * get_sb(segs_per_sec)); > + 2 * get_newsb(segs_per_sec)); > > DBG(0, "Info: Overprovision ratio = %.3lf%%\n", c.new_overprovision); > DBG(0, "Info: Overprovision segments = %u (GC reserved = %u)\n", > @@ -551,11 +551,12 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi, > cpu_to_le32(crc); > > /* Write a new checkpoint in the other set */ > - new_cp_blk_no = old_cp_blk_no = get_sb(cp_blkaddr); > + old_cp_blk_no = get_sb(cp_blkaddr); > + new_cp_blk_no = get_newsb(cp_blkaddr); > if (sbi->cur_cp == 2) > old_cp_blk_no += 1 << get_sb(log_blocks_per_seg); > else > - new_cp_blk_no += 1 << get_sb(log_blocks_per_seg); > + new_cp_blk_no += 1 << get_newsb(log_blocks_per_seg); > > /* write first cp */ > ret = dev_write_block(new_cp, new_cp_blk_no++); > -- > 2.40.1 ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH v7] f2fs: fix zoned block device information initialization
On Sun, Apr 7, 2024 at 12:23 AM Chao Yu wrote:
>
> From: Wenjie Qi
>
> If the max open zones of zoned devices are less than
> the active logs of F2FS, the device may error due to
> insufficient zone resources when multiple active logs
> are being written at the same time.
>
> Signed-off-by: Wenjie Qi
> Signed-off-by: Chao Yu
> ---
> v7:
> - check f2fs_sb_has_blkzoned() instead of multiple-device condition
> in f2fs_remount().
> fs/f2fs/f2fs.h | 1 +
> fs/f2fs/super.c | 27 +++
> 2 files changed, 28 insertions(+)
>
> diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
> index 0550929dc6e5..694f8a52cb84 100644
> --- a/fs/f2fs/f2fs.h
> +++ b/fs/f2fs/f2fs.h
> @@ -1559,6 +1559,7 @@ struct f2fs_sb_info {
>
> #ifdef CONFIG_BLK_DEV_ZONED
> unsigned int blocks_per_blkz; /* F2FS blocks per zone */
> + unsigned int max_open_zones;/* max open zone resources of
> the zoned device */
> #endif
>
> /* for node-related operations */
> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> index ba6288e870c5..fdf358c7f808 100644
> --- a/fs/f2fs/super.c
> +++ b/fs/f2fs/super.c
> @@ -2324,6 +2324,17 @@ static int f2fs_remount(struct super_block *sb, int
> *flags, char *data)
> if (err)
> goto restore_opts;
>
> +#ifdef CONFIG_BLK_DEV_ZONED
> + if (f2fs_sb_has_blkzoned(sbi) &&
> + sbi->max_open_zones < F2FS_OPTION(sbi).active_logs) {
> + f2fs_err(sbi,
> + "zoned: max open zones %u is too small, need at least
> %u open zones",
> +sbi->max_open_zones,
> F2FS_OPTION(sbi).active_logs);
> + err = -EINVAL;
> + goto restore_opts;
> + }
> +#endif
> +
> /* flush outstanding errors before changing fs state */
> flush_work(>s_error_work);
>
> @@ -3866,11 +3877,24 @@ static int init_blkz_info(struct f2fs_sb_info *sbi,
> int devi)
> sector_t nr_sectors = bdev_nr_sectors(bdev);
> struct f2fs_report_zones_args rep_zone_arg;
> u64 zone_sectors;
> + unsigned int max_open_zones;
> int ret;
>
> if (!f2fs_sb_has_blkzoned(sbi))
> return 0;
>
> + if (bdev_is_zoned(FDEV(devi).bdev)) {
> + max_open_zones = bdev_max_open_zones(bdev);
> + if (max_open_zones && (max_open_zones < sbi->max_open_zones))
> + sbi->max_open_zones = max_open_zones;
> + if (sbi->max_open_zones < F2FS_OPTION(sbi).active_logs) {
> + f2fs_err(sbi,
> + "zoned: max open zones %u is too small, need
> at least %u open zones",
> + sbi->max_open_zones,
> F2FS_OPTION(sbi).active_logs);
> + return -EINVAL;
> + }
> + }
> +
> zone_sectors = bdev_zone_sectors(bdev);
> if (sbi->blocks_per_blkz && sbi->blocks_per_blkz !=
> SECTOR_TO_BLOCK(zone_sectors))
> @@ -4191,6 +4215,9 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi)
>
> logical_blksize = bdev_logical_block_size(sbi->sb->s_bdev);
> sbi->aligned_blksize = true;
> +#ifdef CONFIG_BLK_DEV_ZONED
> + sbi->max_open_zones = UINT_MAX;
> +#endif
>
> for (i = 0; i < max_devices; i++) {
> if (i == 0)
> --
> 2.40.1
>
>
Reviewed-by: Daeho Jeong
>
> ___
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH] f2fs-tools: give 6 sections for overprovision buffer
On Tue, Apr 2, 2024 at 4:56 PM Jaegeuk Kim wrote:
>
> This addresses high GC cost at runtime.
>
> Signed-off-by: Jaegeuk Kim
> ---
> include/f2fs_fs.h | 8 +++-
> mkfs/f2fs_format.c | 5 +++--
> 2 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/include/f2fs_fs.h b/include/f2fs_fs.h
> index fc56396fa358..870a6e4823d2 100644
> --- a/include/f2fs_fs.h
> +++ b/include/f2fs_fs.h
> @@ -1775,6 +1775,12 @@ static inline uint32_t get_reserved(struct
> f2fs_super_block *sb, double ovp)
> return round_up(reserved, segs_per_sec) * segs_per_sec;
> }
>
> +static inline uint32_t overprovision_segment_buffer(struct f2fs_super_block
> *sb)
> +{
> + /* Give 6 current sections to avoid huge GC overheads. */
> + return 6 * get_sb(segs_per_sec);
> +}
> +
> static inline double get_best_overprovision(struct f2fs_super_block *sb)
> {
> double ovp, candidate, end, diff, space;
> @@ -1798,7 +1804,7 @@ static inline double get_best_overprovision(struct
> f2fs_super_block *sb)
> if (ovp < 0)
> continue;
> space = usable_main_segs - max((double)reserved, ovp) -
> - 2 * get_sb(segs_per_sec);
> + overprovision_segment_buffer(sb);
> if (max_space < space) {
> max_space = space;
> max_ovp = candidate;
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 8f632f8d74b4..e26a513ed80c 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -778,7 +778,8 @@ static int f2fs_write_check_point_pack(void)
> * In non configurable reserved section case, overprovision
> * segments are always bigger than two sections.
> */
> - if (get_cp(overprov_segment_count) < 2 *
> get_sb(segs_per_sec)) {
> + if (get_cp(overprov_segment_count) <
> + overprovision_segment_buffer(sb)) {
> MSG(0, "\tError: Not enough overprovision segments
> (%u)\n",
> get_cp(overprov_segment_count));
> goto free_cp_payload;
> @@ -787,7 +788,7 @@ static int f2fs_write_check_point_pack(void)
> get_cp(rsvd_segment_count));
> } else {
> set_cp(overprov_segment_count, get_cp(overprov_segment_count)
> +
> - 2 * get_sb(segs_per_sec));
> + overprovision_segment_buffer(sb));
> }
>
> if (f2fs_get_usable_segments(sb) <= get_cp(overprov_segment_count)) {
> --
> 2.44.0.478.gd926399ef9-goog
>
>
>
Reviewed-by: Daeho Jeong
> ___
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH] f2fs-tools: print extension list properly
The "hot file extensions" list does not print properly.
**Before**
extension_count [0x 23 : 35]
cold file extentsions
[mp wm og jp ]
[avi m4v m4p mkv ]
[mov webmwav m4a ]
[3gp opusflacgif ]
[png svg webpjar ]
[deb iso gz xz ]
[zst pdf pyc ttc ]
[ttf exe apk cnt ]
[exo odexvdex]
hot_ext_count [0x 1 : 1]
hot file extentsions
db ]
cp_payload [0x 0 : 0]
**After**
extension_count [0x 23 : 35]
cold file extentsions
[mp wm og jp ]
[avi m4v m4p mkv ]
[mov webmwav m4a ]
[3gp opusflacgif ]
[png svg webpjar ]
[deb iso gz xz ]
[zst pdf pyc ttc ]
[ttf exe apk cnt ]
[exo odexvdex]
hot_ext_count [0x 1 : 1]
hot file extentsions
[db ]
cp_payload [0x 0 : 0]
Signed-off-by: Sheng Yong
---
fsck/mount.c | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/fsck/mount.c b/fsck/mount.c
index b983920..8524335 100644
--- a/fsck/mount.c
+++ b/fsck/mount.c
@@ -429,20 +429,26 @@ void print_extention_list(struct f2fs_super_block *sb,
int cold)
printf("%s file extentsions\n", cold ? "cold" : "hot");
- for (i = start; i < end; i++) {
+ for (i = 0; i < end - start; i++) {
if (c.layout) {
printf("%-30s %-8.8s\n", "extension_list",
- sb->extension_list[i]);
+ sb->extension_list[start + i]);
} else {
if (i % 4 == 0)
printf("%-30s\t\t[", "");
- printf("%-8.8s", sb->extension_list[i]);
+ printf("%-8.8s", sb->extension_list[start + i]);
- if (i % 4 == 4 - 1 || i == end - start - 1)
+ if (i % 4 == 4 - 1)
printf("]\n");
}
}
+
+ for (; i < round_up(end - start, 4) * 4; i++) {
+ printf("%-8.8s", "");
+ if (i % 4 == 4 - 1)
+ printf("]\n");
+ }
}
static void DISP_label(const char *name)
--
2.40.1
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
[f2fs-dev] [PATCH] resize.f2fs: get value from new sb during rebuilding cp
Althrough old and new sb have the same value for now, it would be better to build new checkpoint according to new sb. Signed-off-by: Sheng Yong --- fsck/resize.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fsck/resize.c b/fsck/resize.c index 049ddd3..1b4ae85 100644 --- a/fsck/resize.c +++ b/fsck/resize.c @@ -481,7 +481,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi, set_cp(overprov_segment_count, get_cp(rsvd_segment_count)); set_cp(overprov_segment_count, get_cp(overprov_segment_count) + - 2 * get_sb(segs_per_sec)); + 2 * get_newsb(segs_per_sec)); DBG(0, "Info: Overprovision ratio = %.3lf%%\n", c.new_overprovision); DBG(0, "Info: Overprovision segments = %u (GC reserved = %u)\n", @@ -551,11 +551,12 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi, cpu_to_le32(crc); /* Write a new checkpoint in the other set */ - new_cp_blk_no = old_cp_blk_no = get_sb(cp_blkaddr); + old_cp_blk_no = get_sb(cp_blkaddr); + new_cp_blk_no = get_newsb(cp_blkaddr); if (sbi->cur_cp == 2) old_cp_blk_no += 1 << get_sb(log_blocks_per_seg); else - new_cp_blk_no += 1 << get_sb(log_blocks_per_seg); + new_cp_blk_no += 1 << get_newsb(log_blocks_per_seg); /* write first cp */ ret = dev_write_block(new_cp, new_cp_blk_no++); -- 2.40.1 ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
