Re: [zfs-discuss] zfs diff performance

2012-02-28 Thread Ian Collins
On 02/28/12 12:53 PM, Ulrich Graef wrote: Hi Ian, On 26.02.12 23:42, Ian Collins wrote: I had high hopes of significant performance gains using zfs diff in Solaris 11 compared to my home-brew stat based version in Solaris 10. However the results I have seen so far have been disappointing.

Re: [zfs-discuss] zfs diff performance

2012-02-28 Thread Edward Ned Harvey
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Ulrich Graef Do you use de-duplication? (does not directly harm the performance, but needs memory and slows down zfs diff through that)? Yikes. That couldn't be more wrong. Yes, dedup

Re: [zfs-discuss] zfs diff performance

2012-02-27 Thread Ulrich Graef
Hi Ian, On 26.02.12 23:42, Ian Collins wrote: I had high hopes of significant performance gains using zfs diff in Solaris 11 compared to my home-brew stat based version in Solaris 10. However the results I have seen so far have been disappointing. Testing on a reasonably sized filesystem

[zfs-discuss] zfs diff performance

2012-02-26 Thread Ian Collins
I had high hopes of significant performance gains using zfs diff in Solaris 11 compared to my home-brew stat based version in Solaris 10. However the results I have seen so far have been disappointing. Testing on a reasonably sized filesystem (4TB), a diff that listed 41k changes took 77

[zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is zfs diff. Using the birthtime of the sectors, I would expect very high performance. The actual performance doesn't seems better that an standard rdiff, though. Quite

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Nico Williams
On Mon, Sep 26, 2011 at 1:55 PM, Jesus Cea j...@jcea.es wrote: I just upgraded to Solaris 10 Update 10, and one of the improvements is zfs diff. Using the birthtime of the sectors, I would expect very high performance. The actual performance doesn't seems better that an standard rdiff,

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread David Magda
On Mon, September 26, 2011 14:55, Jesus Cea wrote: [...] real10m0.272s user0m0.809s sys 2m6.693s 10 minutes to diff 7.55 GB is... disappointing. This machine uses a 2-mirror configurations, and there is no more activity going on in the machine. ZPOOL version 29, ZFS version

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/09/11 22:29, David Magda wrote: Talking about 7.55 GB is mostly useless as well. If it's a dozen video files then stat()ing them all with be done very quickly by just running find(1). If however the 7.55 GB is made up of 7,550,000 files then

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/09/11 21:31, Nico Williams wrote: atime has nothing to do with it. How much work zfs diff has to do depends on how much has changed between snapshots. That is what I thought, but look at my example: less than 20 changes and more than 10

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/09/11 22:54, Jesus Cea wrote: On 26/09/11 22:29, David Magda wrote: Talking about 7.55 GB is mostly useless as well. If it's a dozen video files then stat()ing them all with be done very quickly by just running find(1). If however the 7.55

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Bob Friesenhahn
On Mon, 26 Sep 2011, Jesus Cea wrote: rsync takes a bit less than 7 minutes. So zfs diff is actually slower!. It is important to define what is meant by rsync. For example, a common rsync operating mode is to simply compare whole-file timestamps and file size in order to determine that a

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Bill Sommerfeld
On 09/26/11 12:31, Nico Williams wrote: On Mon, Sep 26, 2011 at 1:55 PM, Jesus Cea j...@jcea.es wrote: Should I disable atime to improve zfs diff performance? (most data doesn't change, but atime of most files would change). atime has nothing to do with it. based on my experiences with

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Nico Williams
Ah yes, of course. I'd misread your original post. Yes, disabling atime updates will reduce the number of superfluous transactions. It's *all* transactions that count, not just the ones the app explicitly caused, and atime implies lots of transactions. Nico --

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Ian Collins
On 09/27/11 07:55 AM, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is zfs diff. Using the birthtime of the sectors, I would expect very high performance. The actual performance doesn't seems better that an

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Tomas Forsman
On 27 September, 2011 - Ian Collins sent me these 0,8K bytes: On 09/27/11 07:55 AM, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is zfs diff. Using the birthtime of the sectors, I would expect very

Re: [zfs-discuss] zfs diff performance disappointing

2011-09-26 Thread Ian Collins
On 09/27/11 10:59 AM, Tomas Forsman wrote: On 27 September, 2011 - Ian Collins sent me these 0,8K bytes: On 09/27/11 07:55 AM, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is zfs diff. Using the

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-14 Thread David Blasingame Oracle
a diff to list the file differences between snapshots http://arc.opensolaris.org/caselog/PSARC/2010/105/mail Dave On 10/13/10 15:48, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of dirk schelfhout Wanted to test

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-14 Thread dirk schelfhout
I had to upgrade zfs zfs upgrade -a then pfexec zfs set sharesmb=off data pfexec zfs set sharesmb=on data after this zfs diff failed with the old snapshots. But with newly created snapshots it worked. Thanks Tim, Dirk -- This message posted from opensolaris.org

[zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread dirk schelfhout
Wanted to test the zfs diff command and ran into this. I turned off all windows sharing. the rpool has normal permissions for .zfs/shares how do I fix this ? Dirk r...@osolpro:/data/.zfs# zfs diff d...@10aug2010 d...@13oct2010 Cannot stat /data/.zfs/shares/: unable to generate diffs pwd

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread Tim Haley
On 10/13/10 10:20 AM, dirk schelfhout wrote: Wanted to test the zfs diff command and ran into this. I turned off all windows sharing. the rpool has normal permissions for .zfs/shares how do I fix this ? Dirk r...@osolpro:/data/.zfs# zfs diff d...@10aug2010 d...@13oct2010 Cannot stat

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread dirk schelfhout
cd /data/.zfs sche...@osolpro:/data/.zfs$ ls -alt ls: cannot access shares: Operation not supported total 4 drwxr-xr-x 19 schelfd staff 25 2010-10-13 18:57 .. dr-xr-xr-x 2 rootroot 2 2010-10-13 17:44 snapshot dr-xr-xr-x 4 rootroot 4 2009-01-28 23:08 . ?? ? ? ? ?

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread Edward Ned Harvey
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of dirk schelfhout Wanted to test the zfs diff command and ran into this. What's zfs diff? I know it's been requested, but AFAIK, not implemented yet. Is that new feature being developed now

Re: [zfs-discuss] zfs diff

2010-03-30 Thread Edward Ned Harvey
On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams nicolas.willi...@sun.com wrote: One really good use for zfs diff would be: as a way to index zfs send backups by contents. Or to generate the list of files for incremental backups via NetBackup or similar. This is especially important

[zfs-discuss] zfs diff

2010-03-29 Thread David Magda
A new ARC case: There is a long-standing RFE for zfs to be able to describe what has changed between the snapshots of a dataset. To provide this capability, we propose a new 'zfs diff' sub-command. When run with appropriate privilege the sub-command describes what file system level changes

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Nicolas Williams
One really good use for zfs diff would be: as a way to index zfs send backups by contents. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Nicolas Williams
zfs diff is incredibly cool. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Bruno Sousa
On 30-3-2010 0:39, Nicolas Williams wrote: One really good use for zfs diff would be: as a way to index zfs send backups by contents. Nico Any prevision about the release target? snv_13x? Bruno smime.p7s Description: S/MIME Cryptographic Signature

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Mike Gerdts
On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams nicolas.willi...@sun.com wrote: One really good use for zfs diff would be: as a way to index zfs send backups by contents. Or to generate the list of files for incremental backups via NetBackup or similar. This is especially important for file

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Bart Smaalders
On 03/29/10 16:44, Mike Gerdts wrote: On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams nicolas.willi...@sun.com wrote: One really good use for zfs diff would be: as a way to index zfs send backups by contents. Or to generate the list of files for incremental backups via NetBackup or

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Daniel Carosone
On Mon, Mar 29, 2010 at 06:38:47PM -0400, David Magda wrote: A new ARC case: I read this earlier this morning. Welcome news indeed! I have some concerns about the output format, having worked with similar requirements in the past. In particular: as part of the monotone VCS when reporting

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Daniel Carosone
On Tue, Mar 30, 2010 at 12:37:15PM +1100, Daniel Carosone wrote: There will also need to be clear rules on output ordering, with respect to renames, where multiple changes have happened to renamed files. Separately, but relevant in particular to the above due to the potential for races: what

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Tim Haley
On 3/29/10 8:02 PM, Daniel Carosone wrote: On Tue, Mar 30, 2010 at 12:37:15PM +1100, Daniel Carosone wrote: There will also need to be clear rules on output ordering, with respect to renames, where multiple changes have happened to renamed files. Separately, but relevant in particular to the

[zfs-discuss] zfs diff @snap1 @snap2

2008-05-18 Thread Akhilesh Mritunjai
Hi Is it possible to see what changed between two snapshots (efficiently) ? I tried to take a look what zfs send -i does, and I found that it operates at very low (dmu) level and basically dumps the blocks. Any pointers on extracting inode info from this stream or otherwise ? - mritun