Hi everyone, I want to start a discussion on using f2fs for regular desktops/workstations.
There are growing number of interests in using f2fs as the general root file-system: 2018: https://www.phoronix.com/news/GRUB-Now-Supports-F2FS 2020: https://www.phoronix.com/news/Clear-Linux-F2FS-Root-Option 2023: https://code.launchpad.net/~nexusprism/curtin/+git/curtin/+merge/439880 2023: https://code.launchpad.net/~nexusprism/grub/+git/ubuntu/+merge/440193 I've been personally running f2fs on all of my x86 Linux boxes since 2015, and I have several concerns that I think we need to collectively address for regular non-Android normies to use f2fs: A. Bootloader and installer support B. Host-side GC C. Extended node bitmap I'll go through each one. === A. Bootloader and installer support === It seems that both GRUB and systemd-boot supports f2fs without the need for a separate ext4-formatted /boot partition. Some distros are seemingly disabling f2fs module for GRUB though for security reasons: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868664 It's ultimately up to the distro folks to enable this, and still in the worst-case scenario, they can specify a separate /boot partition and format it to ext4 upon installation. The installer itself to show f2fs and call mkfs.f2fs is being worked on currently on Ubuntu. See the 2023 links above. Nothing f2fs mainline developers should do here, imo. === B. Host-side GC === f2fs relieves most of the device-side GC but introduces a new host-side GC. This is extremely confusing for people who have no background in SSDs and flash storage to understand, let alone discard/trim/erase complications. In most consumer-grade blackbox SSDs, device-side GCs are handled automatically for various workloads. f2fs, however, leaves that responsibility to the userspace with conservative tuning on the kernel-side by default. Android handles this by init.rc tunings and a separate code running in vold to trigger gc_urgent. For regular Linux desktop distros, f2fs just runs on the default configuration set on the kernel and unless it’s running 24/7 with plentiful idle time, it quickly runs out of free segments and starts triggering foreground GC. This is giving people the wrong impression that f2fs slows down far drastically than other file-systems when that’s quite the contrary (i.e., less fragmentation overtime). This is almost the equivalent of re-living the nightmare of trim. On SSDs with very small to no over-provisioned space, running a file-system with no discard what-so-ever (sadly still a common case when an external SSD is used with no UAS) will also drastically slow the performance down. On file-systems with no asynchronous discard, mounting a file-system with the discard option adds a non-negligible overhead on every remove/delete operations, so most distros now (thankfully) use a timer job registered to systemd to trigger fstrim: https://github.com/util-linux/util-linux/commits/master/sys-utils/fstrim.timer This is still far from ideal. The default file-system, ext4, slows down drastically almost to a halt when fstrim -a is called, especially on SATA. For some reason that is still a mystery for me, people seem to be happy with it. No one bothered to improve it for years ¯\_(ツ)_/¯. So here’s my proposal: As Linux distros don’t have a good mechanism for hinting when to trigger GC, introduce a new Kconfig, CONFIG_F2FS_GC_UPON_FSTRIM and enable it by default. This config will hook up ioctl(FITRIM), which is currently ignored on f2fs - https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/commit/?h=master&id=e555da9f31210d2b62805cd7faf29228af7c3cfb , to perform discard and GC on all invalid segments. Userspace configuration with enough f2fs/GC knowledge such as Android should disable it. This will ensure that Linux distros that blindly call fstrim will at least avoid constant slowdowns when free segments are depleted with the occasional (once a week) slowdown, which *people are already living with on ext4*. I'll even go further and mention that since f2fs GC is a regular R/W workload, it doesn't cause an extreme slowdown comparable to a level of a full file-system trim operation. If this is acceptable, I’ll cook up a patch. In an ideal world, all Linux distros should have an explicit f2fs GC trigger mechanism (akin to https://github.com/kdave/btrfsmaintenance#distro-integration ), but it’s practically unrealistic to expect that, given the installer doesn’t even support f2fs for now. === C. Extended node bitmap === f2fs by default have a very limited number of allowed inodes compared to other file-systems. Just 2 AOSP syncs are enough to exhaust f2fs and result in -ENOSPC. Here are some of the stats collected from me and my colleague that we use daily as a regular desktop with GUI, web-browsing and everything: 1. Laptop Utilization: 68% (182914850 valid blocks, 462 discard blocks) - Node: 10234905 (Inode: 10106526, Other: 128379) - Data: 172679945 - Inline_xattr Inode: 2004827 - Inline_data Inode: 867204 - Inline_dentry Inode: 51456 2. Desktop #1 Utilization: 55% (133310465 valid blocks, 0 discard blocks) - Node: 6389660 (Inode: 6289765, Other: 99895) - Data: 126920805 - Inline_xattr Inode: 2253838 - Inline_data Inode: 1119109 - Inline_dentry Inode: 187958 3. Desktop #2 Utilization: 83% (202222003 valid blocks, 1 discard blocks) - Node: 21887836 (Inode: 21757139, Other: 130697) - Data: 180334167 - Inline_xattr Inode: 39292 - Inline_data Inode: 35213 - Inline_dentry Inode: 1127 4. Colleague Utilization: 22% (108652929 valid blocks, 362420605 discard blocks) - Node: 5629348 (Inode: 5542909, Other: 86439) - Data: 103023581 - Inline_xattr Inode: 655752 - Inline_data Inode: 259900 - Inline_dentry Inode: 193000 5. Android phone (for reference) Utilization: 78% (36505713 valid blocks, 1074 discard blocks) - Node: 704698 (Inode: 683337, Other: 21361) - Data: 35801015 - Inline_xattr Inode: 683333 - Inline_data Inode: 237470 - Inline_dentry Inode: 112177 Chao Yu added a functionality to expand this via the -i flag passed to mkfs.f2fs back in 2018 - https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/commit/?id=baaa076b4d576042913cfe34169442dfda651ca4 I occasionally find myself in a weird position of having to tell people "Oh you should use the -i option from mkfs.f2fs" when they encounter this issue only after they’ve migrated most of the data and ask back "Why isn’t this enabled by default?". While this might not be an issue for the foreseeable future in Android, I’d argue that this is a feature that needs to be enabled by default for desktop environments with preferably a robust testing infrastructure. Guarding this with #ifndef __ANDROID__ doesn’t seem to make much sense as it introduces more complications to how fuzzing/testing should be done. I’ll also add that it’s a common practice for userspace mkfs tools to introduce breaking default changes to older kernels (with options to produce a legacy image, of course). This was a lengthy email, but I hope I was being reasonable. Jaegeuk and Chao, let me know what you think. And as always, thanks for your hard work :) Thanks, regards _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
