Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-15 Thread Fred Liu
> >Even with infinite wire speed, you're bound by the ability of the source server >to generate the snapshot stream and the ability of the destination server to >write the snapshots to the media. > >Our little servers in-house using ZFS don't read/write that fast when pulling >snapshot contents of

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Eric D. Mudama
On Fri, Dec 14 at 9:29, Fred Liu wrote: We have found mbuffer to be the fastest solution. Our rates for large transfers on 10GbE are: 280MB/smbuffer 220MB/srsh 180MB/sHPN-ssh unencrypted 60MB/s standard ssh The tradeoff mbuffer is a little more complicated to script; rs

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Andrew Gabriel
In my own experiments with my own equivalent of mbuffer, it's well worth giving the receiving side a buffer which is sized to hold the amount of data in a transaction commit, which allows ZFS to be banging out one tx group to disk, whilst the network is bringing the next one across for it. This

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Fred Liu
> > We have found mbuffer to be the fastest solution. Our rates for large > transfers on 10GbE are: > > 280MB/smbuffer > 220MB/srsh > 180MB/sHPN-ssh unencrypted > 60MB/s standard ssh > > The tradeoff mbuffer is a little more complicated to script; rsh is, > well, you know;

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Fred Liu
Post in the list. > -Original Message- > From: Fred Liu > Sent: 星期五, 十二月 14, 2012 23:41 > To: 'real-men-dont-cl...@gmx.net' > Subject: RE: [zfs-discuss] any more efficient way to transfer snapshot > between two hosts than ssh tunnel? > > > >

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Fred Liu
> > I've heard you could, but I've never done it. Sorry I'm not much help, > except as a cheer leader. You can do it! I think you can! Don't give > up! heheheheh > Please post back whatever you find, or if you have to figure it out for > yourself, then blog about it and post that. Aha! Gotc

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Palmer, Trey
: Freddie Cash Cc: zfs-discuss Subject: Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel? >Add the HPN patches to OpenSSH and enable the NONE cipher. We can saturate a >gigabits link (980 mbps) between two FreeBSD hosts using that. >Without it

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-14 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Fred Liu > > BTW, anyone played NDMP in solaris? Or is it feasible to transfer snapshot via > NDMP protocol? I've heard you could, but I've never done it. Sorry I'm not much help, except as

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-13 Thread Fred Liu
>Add the HPN patches to OpenSSH and enable the NONE cipher.  We can saturate a >gigabits link (980 mbps) between two FreeBSD hosts using that. >Without it, we were only able to hit ~480 mbps on a good day. >If you want 0 overhead, there's always netcat. :) 980mbps is awesome! I am thinking runnin

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-13 Thread Freddie Cash
On Dec 13, 2012 8:02 PM, "Fred Liu" wrote: > > Assuming in a secure and trusted env, we want to get the maximum transfer speed without the overhead from ssh. Add the HPN patches to OpenSSH and enable the NONE cipher. We can saturate a gigabits link (980 mbps) between two FreeBSD hosts using that

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-13 Thread Fred Liu
. Fred From: Adrian Smith [mailto:adrian.sm...@rmit.edu.au] Sent: 星期五, 十二月 14, 2012 12:08 To: Fred Liu Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel? Hi Fred, Try mbuffer (http://www.maier-komor.de

Re: [zfs-discuss] any more efficient way to transfer snapshot between two hosts than ssh tunnel?

2012-12-13 Thread Adrian Smith
Hi Fred, Try mbuffer (http://www.maier-komor.de/mbuffer.html) On 14 December 2012 15:01, Fred Liu wrote: > Assuming in a secure and trusted env, we want to get the maximum > transfer speed without the overhead from ssh. > > ** ** > > Thanks. > > ** ** > > Fred > >