Apparently your data is represented by rather small files (thus
many small data blocks), so proportion of metadata is relatively
high, and your<4k blocks are now using at least 4k disk space.
For data with small blocks (a 4k volume on an ashift=12 pool)
I saw metadata use up most of my drive - becoming equal to
Just for the sake of completeness, I brought up a similar problem
and a non-intrusive (compatibility-wise) solution in this bug:
Main idea was to let ZFS users specify a minumum data block
size and alignment, while formal ashift=9 remains in place and
metadata blocks remain 512b. This fix would be a code-only
change without on-disk-format changes. There could be some
further cleverness when working with (updating) metadata,
so that hardware 4k blocks would need to be rewritten as
rarely and as fully as possible - to reduce wear and increase
efficiency - but the main idea is hopefully simple.
zfs-discuss mailing list