> On 9. Sep 2024, at 13:41, Gea <g...@napp-it.org> wrote: > > On 08.09.2024 11:55, d wrote: >> This post enticed me to do a little research on zfs compression, which lead >> me to some interesting info an a question: >> https://indico.fnal.gov/event/16264/contributions/36466/attachments/22610/28037/Zstd__LZ4.pdf > > The simplified result seems lz4 is faster while zstd compress data more > efficient. > > When I look at my main pool, refcompressratio with lz4 is 1.04 for my main > data filesystem with mixed data, some at 1.02 with mainly iso and media and > up to 1.12 in rare cases. With expensive NVMe or hybrid pools with a special > vdev, I would opt for zstd and better compressratio as performance is better > than good enough. But I have this choice only with Open-ZFS and this for > quite a long time similar to draid with many advantages on very large pools.
tsoome@beastie:~$ zfs get compression rpool/ROOT/openindiana-2024:08:23 NAME PROPERTY VALUE SOURCE rpool/ROOT/openindiana-2024:08:23 compression zstd inherited from rpool tsoome@beastie:~$ zfs get compressratio rpool/ROOT/openindiana-2024:08:23 NAME PROPERTY VALUE SOURCE rpool/ROOT/openindiana-2024:08:23 compressratio 2.75x - tsoome@beastie:~$ zfs get compression rpool/code/illumos-gate NAME PROPERTY VALUE SOURCE rpool/code/illumos-gate compression zstd local tsoome@beastie:~$ zfs get compressratio rpool/code/illumos-gate NAME PROPERTY VALUE SOURCE rpool/code/illumos-gate compressratio 3.32x - tsoome@beastie:~$ (I probably still do have some lz4 blocks lurking around). rgds, toomas > > The upcoming Fast Dedup in Open-ZFS where you can limit dedup table size with > a quota, place the dedup table not only on a dedicated dedup vdev but also a > special vdev, can shrink dedup tables from single items or cache dedup in Arc > seems a killer feature. Unlike current dedup, I would expect that you can > enable Fast Dedup per default, either with a special vdev that also massively > improve all small file actions and metadata access or with a quota like a few > GB to limit RAM usage. Hybrid pools with a special vdev become more and more > attractive as suggested default. Fast dedup is beta but near to be ready. I > already can play with in in Open-ZFS on Windows rc. Will this appear in > Illumos in the forseeable future. I doubt. > > The stability aspect, ok. Software has bugs and you need to maintain does not > matter if Open-ZFS or Illumos-ZFS. There have also been critical bugs in > Illumos. When I remember correctly it was persistent L2Arc in Illumos where > datalosses could have happened a few years ago (luckily I was not affected as > silent errors could have happened). > > Why do anyone expects that bugs in Illumos ZFS do not happen or fixed faster > than bugs in Open-ZFS with 20x the user base and number of devs and many > large commercial companies behind. Even the additional tests on taking over > Open-ZFS code in more and more rare cases does not guarantee that there are > no bugs remained (while helpful for sure and better as using Open-ZFS master > or the stable branch directly) > ------------------------------------------ illumos: illumos-discuss Permalink: https://illumos.topicbox.com/groups/discuss/T627f77e1b29a7b53-M2f7ec9501be1d08776f21477 Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
