Re: [zfs-discuss] offline dedup

2011-05-29 Thread Daniel Carosone
On Fri, May 27, 2011 at 07:28:06AM -0400, Edward Ned Harvey wrote: From: Daniel Carosone [mailto:d...@geek.com.au] Sent: Thursday, May 26, 2011 8:19 PM Once your data is dedup'ed, by whatever means, access to it is the same. You need enough memory+l2arc to indirect references via

Re: [zfs-discuss] offline dedup

2011-05-27 Thread Edward Ned Harvey
From: Daniel Carosone [mailto:d...@geek.com.au] Sent: Thursday, May 26, 2011 8:19 PM Once your data is dedup'ed, by whatever means, access to it is the same. You need enough memory+l2arc to indirect references via DDT. I don't think this is true. The reason you need arc+l2arc to store

Re: [zfs-discuss] offline dedup

2011-05-27 Thread Jim Klimov
Dan ... It would still need a complex bp_rewrite. Are you certain about that? For example, scrubbing/resilvering and fixing corrupt blocks with non-matching checksums is a post-processing operation which works on an existing pool and rewrites some blocks if needed. And it works without a

Re: [zfs-discuss] offline dedup

2011-05-27 Thread Frank Van Damme
2011/5/27 Edward Ned Harvey opensolarisisdeadlongliveopensola...@nedharvey.com: I don't think this is true.  The reason you need arc+l2arc to store your DDT is because when you perform a write, the system will need to check and see if that block is a duplicate of an already existing block.  If

[zfs-discuss] offline dedup

2011-05-26 Thread Edward Ned Harvey
Hey, I got another question for ZFS developers - Given: If you enable dedup and write a bunch of data, and then disable dedup, the formerly written data will remain dedup'd. Given: The zdb -s command, which simulates dedup to provide dedup statistics without actually enabling dedup.

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Brandon High
On Thu, May 26, 2011 at 8:37 AM, Edward Ned Harvey opensolarisisdeadlongliveopensola...@nedharvey.com wrote: Question:  Is it possible, or can it easily become possible, to periodically dedup a pool instead of keeping dedup running all the time?  It is easy to I think it's been discussed

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Daniel Carosone
On Thu, May 26, 2011 at 09:04:04AM -0700, Brandon High wrote: On Thu, May 26, 2011 at 8:37 AM, Edward Ned Harvey opensolarisisdeadlongliveopensola...@nedharvey.com wrote: Question:? Is it possible, or can it easily become possible, to periodically dedup a pool instead of keeping dedup

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Jim Klimov
2011-05-26 19:37, Edward Ned Harvey ?: Hey, I got another question for ZFS developers - Given: If you enable dedup and write a bunch of data, and then disable dedup, the formerly written data will remain dedup'd. Given: The zdb -s command, which simulates dedup to provide dedup

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Daniel Carosone
On Fri, May 27, 2011 at 04:32:03AM +0400, Jim Klimov wrote: One more rationale in this idea is that with deferred dedup in place, the DDT may be forced to hold only non-unique blocks (2+ references), and would require less storage in RAM, disk, L2ARC, etc. - in case we agree to remake the DDT