On Sun, Mar 31, 2024 at 1:37 PM Tomas Vondra <tomas.von...@enterprisedb.com> wrote: > So I decided to take a stab at Thomas' idea, i.e. reading the data to > ... > I'll see how this works on EXT4/ZFS next ...
Wow, very cool! A couple of very quick thoughts/notes: ZFS: the open source version only gained per-file block cloning in 2.2, so if you're on an older release I expect copy_file_range() to work but not really do the magic thing. On the FreeBSD version you also have to turn cloning on with a sysctl because people were worried about bugs in early versions so by default you still get actual copying, not sure if you need something like that on the Linux version... (Obviously ZFS is always completely COW-based, but before the new block cloning stuff it could only share blocks by its own magic deduplication if enabled, or by cloning/snapshotting a whole dataset/mountpoint; there wasn't a way to control it explicitly like this.) Alignment: block sharing on any fs requires it. I haven't re-checked recently but IIRC the incremental file format might have a non-block-sized header? That means that if you copy_file_range() from both the older backup and also the incremental backup, only the former will share blocks, and the latter will silently be done by copying to newly allocated blocks. If that's still true, I'm not sure how hard it would be to tweak the format to inject some padding and to make sure that there isn't any extra header before each block.
