Thomas Glanzmann wrote:
Hello Ric,
(1) Block level or file level dedup?
what is the difference between the two?
(2) Inband dedup (during a write) or background dedup?
I think inband dedup is way to intensive on ressources (memory) and also
would kill every performance benchmark. So I think the offline dedup is
the right way to go.
I would not categorize it as offline, but just not as inband (i.e., you can run
a low priority background process to handle dedup). Offline windows are
extremely rare in production sites these days and it could take a very long time
to do dedup at the block level over a large file system :-)
(3) How reliably can you protect the pool of blocks? How reliably can
you protect the database that maps hashes to blocks?
You have to lock down the i/o requests for the blocks in question and
compare them byte by byte anyway, just to make sure.
Yes, one advantage we had in centera was that the objects were read-only and
whole files, so this was not an issue for us.
(4) Can you give users who are somewhat jaded confidence in your
solution (this is where stats come in very handy!)
For virtual machines you can reduce the used data by 1/3. Of course it
can blow in your face when you don't watch your physical resources
closely.
Thomas
1/3 is not sufficient for dedup in my opinion - you can get that with normal
compression at the block level.
Put another way, if the baseline is bzip2 levels of compression for the block
device, when is the complexity of dedup going to pay off?
I think that Chris already mentioned that you can (for virt OS images) also
imagine using copy on write snapshots (of something that is mostly read-only
like the OS system partitions). Make 50 copy-on-write snapshots of "/"
(excluding /home) and you have an effective compression of 98% until someone
rudely starts writing at least :-)
ric
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
