I feel that, while that's true and valid, it also kind of misses the point?
What I'm wondering is, are there simple enhancements that would be
beneficial in that area, or provide useful internal data.
It seems plausible that if a configurable in space map block size helps,
perhaps a configurable DDT block size could as well, and that if DDT
contents are needed on every file load/save for a deduped pool, then a way
to preload them could be beneficial in the same.way as a way to preload
spacemap data.
Clearly allocation devices will help, but gains are always layered. "Faster
storage will fix it" isnt really an answer, any more than its an answer for
any greatly bottlenecked critical pathway in a file system - and for
deduped pools, access to DDT records is *the* critical element, no user
data can be read from disk or put to txgs without them. Fundamentally there
are a few useful configurables available for spacemaps that could be
potential wins if also available for DDTs. Since analogs for these
settings already exist in the code, perhaps no great work is involved in
them, and perhaps they would give cheap but significant IO gains for pools
with dedup enabled.
Hence I think the question is valid, and remains valid both before
allocation classes and after them, and might be worth considering deeper.
On 7 July 2019 16:03:59 Richard Elling <[email protected]>
wrote:
Yes, from the ZoL zpool man page:
A device dedicated solely for deduplication tables.
-- richard
On Jul 7, 2019, at 5:41 AM, Stilez <[email protected]> wrote:
"Dedup special class"?
On 6 July 2019 16:24:27 Richard Elling <[email protected]>
wrote:
On Jul 5, 2019, at 9:11 PM, Stilez <[email protected]> wrote:
I'm one of many end-users with highly dedupable pools held back by DDT and
spacemap RW inefficiencies. There's been discussion and presentations -
Matt Ahrens' talk at BSDCan 2016 ("Dedup doesn't have to suck") was
especially useful, and allocation classes from the ZoL/ZoF work will allow
metadata-specific offload to SSD. But briad discussion of this general area
is not on the roadmap atm, probably bc so much else is a priority and seems
nobody's stepped up.
In part because dedup will always be slower than non-dedup while the cost
of storage continues to plummet (flash SSDs down 40% in the past year and
there is currently an oversupply of NAND). A good starting point for
experiments is to use the dedup special class and report back to the
community how well it works for you.
-- richard
openzfs / openzfs-developer / see
discussions +
participants +
delivery options
Permalink
------------------------------------------
openzfs: openzfs-developer
Permalink:
https://openzfs.topicbox.com/groups/developer/Td9c7189186fd24f2-Me752b098af06d58a18583cc9
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription
