In my continuing effort to complete the integration of ZSTD into OpenZFS, here is my third weekly status report:
https://github.com/allanjude/zfs/commit/87bb538bdbc4bb8848ed5791b7b0de84a026ebbe - Completed the rewrite of the way the compression property is handled, moving away from the initial approach of storing the compression property (enum zio_compress) and the level (uint64_t) separately. Previously we exposed the list of compression algorithms and levels to userland as the corresponding value from the enum in the lower 7 bits, and the level in the remaining upper bits. Then, as part of the property GET and SET IOCTLs, we read the separate compression= and compress_level= properties from the ZAP and returned the combined value, or split the combined value into those two separate properties. This worked but caused a lot of headache around property inheritance. Instead I've changed to doing the combine/split when reading/writing from the dataset properties ZAP, via the compression_changed_cb() function. So the properties ZAP contains the combined value (lower 7 bits are the compression algorithm, as defined in the enum zio_compress, and the upper bits are the compression level). Elsewhere in ZFS we keep the two values separate (os_compress and os_complevel, and related variables in all of the different parts of ZFS). So now, inheritance of the property is handled correctly, and avoids issues where a dataset with compression=zstd-12, would say 'inherited from' a dataset with zstd at some other compression level (since both actually just had compression=zstd, but different compress_level= values). I have also further extended zdb to inspect the compression settings when looking at an object: https://github.com/allanjude/zfs/commit/3fef3c83b8ce90149110ed989bd9fd3e289798e0 I am still working on a solution for setting the zstd feature flag to 'active' as soon as it is set, rather than only once a block is born. Additionally, I am investigating how to best handle the fact that embedded block pointers compressed with ZSTD will make 'zfs send -e' streams backwards incompatible, without a way for the user to opt-out of sending a stream that contains zstd compressed blocks that the receiving side may not be able to read. The same can be said for 'zfs send -c' as well. I am open to ideas on how best to handle this. I have thought about only sending ZSTD compressed blocks if the user specifies the -Z (--zstd) flag, but this can lead to confusion where using -c without -Z would have to either error out, or send the ZSTD compressed blocks uncompressed. This project is sponsored by the FreeBSD Foundation. -- Allan Jude
signature.asc
Description: OpenPGP digital signature
This is a multi-part message in MIME format... ------------=_1594080439-19433-1--
