On Thu, Apr 10, 2014 at 12:08:17PM +0300, Konstantinos Skarlatos wrote:
> On 10/4/2014 6:48 πμ, Liu Bo wrote:
> >Hello,
> >
> >This the 10th attempt for in-band data dedupe, based on Linux _3.14_ kernel.
> >
> >Data deduplication is a specialized data compression technique for 
> >eliminating
> >duplicate copies of repeating data.[1]
> >
> >This patch set is also related to "Content based storage" in project 
> >ideas[2],
> >it introduces inband data deduplication for btrfs and dedup/dedupe is for 
> >short.
> >
> >* PATCH 1 is a speed-up improvement, which is about dedup and quota.
> >
> >* PATCH 2-5 is the preparation work for dedup implementation.
> >
> >* PATCH 6 shows how we implement dedup feature.
> >
> >* PATCH 7 fixes a backref walking bug with dedup.
> >
> >* PATCH 8 fixes a free space bug of dedup extents on error handling.
> >
> >* PATCH 9 adds the ioctl to control dedup feature.
> >
> >* PATCH 10 targets delayed refs' scalability problem of deleting refs, which 
> >is
> >   uncovered by the dedup feature.
> >
> >* PATCH 11-16 fixes bugs of dedupe including race bug, deadlock, abnormal
> >   transaction abortion and crash.
> >
> >* btrfs-progs patch(PATCH 17) offers all details about how to control the
> >   dedup feature on progs side.
> >
> >I've tested this with xfstests by adding a inline dedup 'enable & on' in 
> >xfstests'
> >mount and scratch_mount.
> >
> >
> >***NOTE***
> >Known bugs:
> >* Mounting with options "flushoncommit" and enabling dedupe feature will end 
> >up
> >   with _deadlock_.
> >
> >
> >TODO:
> >* a bit-to-bit comparison callback.
> >
> >All comments are welcome!
> Hi Liu,
> Thanks for doing this work.
> I tested your previous patches a few months ago, and will now test
> the new ones. One question about memory requirements, are they in
> the same league as ZFS dedup (ie needing 10's of gb of RAM for multi
> TB filesystems) or are they more reasonable?
> Thanks

Hi Konstantinos,

It depends on Linux native memory management which can reclaim memory when
lacking memory, but still, it'd lead to high memory pressure according to my
experiments.

Thanks for testing it!

-liubo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reply via email to