Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-21 Thread Brandon High
On Sat, Dec 19, 2009 at 8:34 AM, Colin Raven co...@clearcutnetworks.com wrote: If snapshots reside within the confines of the pool, are you saying that dedup will also count what's contained inside the snapshots? I'm not sure why, but that thought is vaguely disturbing on some level. Sure, why

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-21 Thread Erik Trimble
Brandon High wrote: On Sat, Dec 19, 2009 at 8:34 AM, Colin Raven co...@clearcutnetworks.com wrote: If snapshots reside within the confines of the pool, are you saying that dedup will also count what's contained inside the snapshots? I'm not sure why, but that thought is vaguely disturbing on

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-20 Thread Nick
Wait...whoah, hold on.brIf snapshots reside within the confines of the pool, are you saying that dedup will also count what#39;s contained inside the snapshots? I#39;m not sure why, but that thought is vaguely disturbing on some level. brThen again (not sure how gurus feel on this point)

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-20 Thread Colin Raven
On Sun, Dec 20, 2009 at 16:23, Nick nick.couch...@seakr.com wrote: IMHO, snapshots are not a replacement for backups. Backups should definitely reside outside the system, so that if you lose your entire array, SAN, controller, etc., you can recover somewhere else. Snapshots, on the other

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-20 Thread Nick Couchman
The (one and only) point that I was making was that - like backups - snapshots should be kept elsewhere whether by using zfs-send, or zipping up the whole shebang and ssh'ing it someplaceelsewhere meaning beyond the pool. Rolling 15 minute and hourly snapshotsno, they stay local, but

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Colin Raven
On Sat, Dec 19, 2009 at 05:25, Ian Collins i...@ianshome.com wrote: Stacy Maydew wrote: The commands zpool list and zpool get dedup pool both show a ratio of 1.10. So thanks for that answer. I'm a bit confused though if the dedup is applied per zfs filesystem, not zpool, why can I only see

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Bob Friesenhahn
On Sat, 19 Dec 2009, Colin Raven wrote: There is no original, there is no copy. There is one block with reference counters. - Fred can rm his file (because clearly it isn't a file, it's a filename and that's all) - result: the reference count is decremented by one - the data remains on disk.

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Colin Raven
On Sat, Dec 19, 2009 at 17:20, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Sat, 19 Dec 2009, Colin Raven wrote: There is no original, there is no copy. There is one block with reference counters. - Fred can rm his file (because clearly it isn't a file, it's a filename and

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Bob Friesenhahn
On Sat, 19 Dec 2009, Colin Raven wrote:   Wait...whoah, hold on. If snapshots reside within the confines of the pool, are you saying that dedup will also count what's contained inside the snapshots? I'm not sure why, but that thought is vaguely disturbing on some level. Yes, of course. Any

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Andrey Kuzmin
On Sat, Dec 19, 2009 at 7:20 PM, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Sat, 19 Dec 2009, Colin Raven wrote: There is no original, there is no copy. There is one block with reference counters. - Fred can rm his file (because clearly it isn't a file, it's a filename and

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Toby Thain
On 19-Dec-09, at 4:35 AM, Colin Raven wrote: ... There is no original, there is no copy. There is one block with reference counters. Many blocks, potentially shared, make up a de-dup'd file. Not sure why you write one here. - Fred can rm his file (because clearly it isn't a file,

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Toby Thain
On 19-Dec-09, at 11:34 AM, Colin Raven wrote: ... Wait...whoah, hold on. If snapshots reside within the confines of the pool, are you saying that dedup will also count what's contained inside the snapshots? Snapshots themselves are only references, so yes. I'm not sure why, but that

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Colin Raven
On Sat, Dec 19, 2009 at 19:08, Toby Thain t...@telegraphics.com.au wrote: On 19-Dec-09, at 11:34 AM, Colin Raven wrote Then again (not sure how gurus feel on this point) but I have this probably naive and foolish belief that snapshots (mostly) oughtta reside on a separate physical

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Toby Thain
On 19-Dec-09, at 2:01 PM, Colin Raven wrote: On Sat, Dec 19, 2009 at 19:08, Toby Thain t...@telegraphics.com.au wrote: On 19-Dec-09, at 11:34 AM, Colin Raven wrote Then again (not sure how gurus feel on this point) but I have this probably naive and foolish belief that snapshots

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-19 Thread Toby Thain
On 19-Dec-09, at 11:34 AM, Colin Raven wrote: ... When we are children, we are told that sharing is good. In the case or references, sharing is usually good, but if there is a huge amount of sharing, then it can take longer to delete a set of files since the mutual references create a

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-18 Thread Ian Collins
Stacy Maydew wrote: The commands zpool list and zpool get dedup pool both show a ratio of 1.10. So thanks for that answer. I'm a bit confused though if the dedup is applied per zfs filesystem, not zpool, why can I only see the dedup on a per pool basis rather than for each zfs filesystem?

[zfs-discuss] How do I determine dedupe effectiveness?

2009-12-17 Thread Stacy Maydew
I'm trying to see if zfs dedupe is effective on our datasets, but I'm having a hard time figuring out how to measure the space saved. When I sent one backup set to the filesystem, the usage reported by zfs list and zfs get used my zfs are the expected values based on the data size. When I

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-17 Thread Cyril Plisko
On Thu, Dec 17, 2009 at 8:57 PM, Stacy Maydew stacy.may...@sun.com wrote: I'm trying to see if zfs dedupe is effective on our datasets, but I'm having a hard time figuring out how to measure the space saved. When I sent one backup set to the filesystem, the usage reported by zfs list and

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-17 Thread Brandon High
On Thu, Dec 17, 2009 at 10:57 AM, Stacy Maydew stacy.may...@sun.com wrote: When I sent one backup set to the filesystem, the usage reported by zfs list and zfs get used my zfs are the expected values based on the data size. When I store a second copy, which should dedupe entirely, the zfs

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-17 Thread Stacy Maydew
The commands zpool list and zpool get dedup pool both show a ratio of 1.10. So thanks for that answer. I'm a bit confused though if the dedup is applied per zfs filesystem, not zpool, why can I only see the dedup on a per pool basis rather than for each zfs filesystem? Seems to me there

Re: [zfs-discuss] How do I determine dedupe effectiveness?

2009-12-17 Thread A Darren Dunham
On Thu, Dec 17, 2009 at 12:30:29PM -0800, Stacy Maydew wrote: So thanks for that answer. I'm a bit confused though if the dedup is applied per zfs filesystem, not zpool, why can I only see the dedup on a per pool basis rather than for each zfs filesystem? Seems to me there should be a way to