Dear Matthew and other filesystem developers, I've been experimenting with implementing large folio support for compressed files in F2FS locally, and I'd like to describe the situation from the F2FS perspective.
> First, I believe that all filesystems work by compressing fixed-size > plaintext into variable-sized compressed blocks. Well, yes. F2FS's current compression implementation does compress fixed-size memory into variable-sized blocks. However, F2FS operates on a fixed-size unit called a "cluster." A file is logically divided into these clusters, and each cluster corresponds to a fixed number of contiguous page indices. The cluster size is 4 << n pages, with n typically defaulting to 0 (making a 4-page cluster). F2FS can only perform compression on a per-cluster basis; it cannot operate on a unit larger than the logical size of a cluster. So, for a 16-page folio with a 4-page cluster size, we would have to split the folio into four separate clusters. We then perform compression on each cluster individually and write back each compressed result to disk separately.We cannot perform compression on the whole large chunk of folio. In fact, the fact that a large folio can span multiple clusters was the main headache in my attempt to implement large folio support for F2FS compression. Why is this the case? It's due to F2FS's current on-disk layout for compressed data. Each cluster is prefixed by a special block address, COMPRESS_ADDR, which separates one cluster from the next on disk. Furthermore, after F2FS compresses the original data in a cluster, the space freed up within that cluster remains reserved on disk; it is not released for other files to use. You may have heard that F2FS compression doesn't actually save space for the user—this is the reason. In F2FS, the model is not what we might intuitively expect—a large chunk of data being compressed into a series of tightly packed data blocks on disk (which I assume is the model other filesystems adopt). So, regarding: > So, my proposal is that filesystems tell the page cache that their minimum > folio size is the compression block size. That seems to be around 64k, > so not an unreasonable minimum allocation size. F2FS doesn't have a uniform "compression block size." It purely depends on the configured cluster size, and the resulting compressed size is determined by the compression ratio. For example, a 4-page cluster could be compressed down to a single block. Regarding the folio order, perhaps we could set its maximum order to match the cluster size, while keeping the minimum order at 0. However, for smaller cluster sizes, this would completely limit the potential of using larger folios. My own current implementation makes no assumptions about the maximum folio order. As I am a student, I lack extensive experience, so it's difficult for me to evaluate the pros and cons of these two approaches. I believe Mr Chao Yu could provide a more constructive suggestion on this point. Thinking about a possible implementation for your proposal of a 64KB size and in-place compression in the context of F2FS, I think the possible approach may be to set the maximum folio order to 4 pages. This would align with the default cluster size (especially relevant as F2FS moves to support 16K pages and blocks). We could then perform compression in-place, eliminating the need for scratch pages (which are the compressed pages/folios in the F2FS context) and also disable per-page dirty tracking for that folio. However, F2FS has fallback logic for when compression fails during writeback. The original F2FS logic still relies on per-page dirty tracking for writes. If we were to completely remove per-page tracking for the folio,then in compression failure case we would bear the cost of one write amplification. These are just my personal thoughts on your proposal. I believe Mr Chao Yu can provide more detailed insights into the specifics of F2FS. Best regards _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
