On 03/11/14 17:59, Freddie Cash wrote:
On Sat, Oct 18, 2014 at 12:32 PM, Allan Jude <allanj...@freebsd.org> wrote:

On 2014-10-18 13:21, Freddie Cash wrote:
On Oct 18, 2014 3:54 AM, "Mark Martinec" <mark.martinec+free...@ijs.si>
wrote:

If the purpose of having a none cipher is to have a fast
file transfer, then one should be using  sysutils/bbcp
for that purposes. Uses ssd for authentication, and
opens unencrypted channel(s) for the actual data transfer.
It's also very fast, can use multiple TCP streams.

That's an interesting alternative to rsync, scp, and ftp, but doesn't
help
with zfs send/recv which is where the none cipher really shines.

Without the none cipher, SSH becomes the bottleneck limiting transfers to
around 400 Mbps on a gigabit LAN. With the none cipher, the network
becomes
the bottleneck limiting transfers to around 920 Mbps on the same gigabit
LAN.

This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs.

Actually, looking into it, the bbcp command can support a pipe at each
end instead of files, so you can actually do a zfs send | zfs receive
via bbcp, and use multiple concurrent connections, to get around TCP
window stuff when going transatlantic

I am going to be trying it out shortly.

Note: the other big improvement in newer ssh is the HPN stuff, that is
switched on since 9.2 I think.


​After much finagling and testing, I have managed to incorporate bbcp into
my ZFS send/recv script.  And it works much better than regular, encrypted
SSH in my setup.

Regular SSH transfers tended to top out around 400 Mbps, using 100% of 1
CPU.  Was not able to get the multi-threadded AES cipher working.

SSH connections using the NONE cipher saturated the gigabit link with
minimal CPU usage.

And a bbcp connection is currently running between 500-800 Mbps (depending
on the size of the snpashot), also with minimal CPU usage.

NOTE:  I expect this be running much better next week, as the receiving
pool is currently resilvering a drive, slowing everything down.


Got things working using the following bbcp command format:

bbcp -N io "zfs send -I pool/fs@snap1 pool/fs@snap2" username@remotesys:"zfs
recv -d pool"


Have not played with any of the myriad tuning options for bbcp.  Just
wanted to see if I could get it to work, and how an untuned connection
compared to an untuned SSH connection (with and without NONE cipher).  So
far, I'm impressed.

​Thanks for the suggestion.  It's another tool in the box.  :)​


I have also written an utility[1] to efficiently encrypt and authenticate data transferred over the network using the modern AES-GCM or chacha20-poly1305 ciphers with pre-shared keys (or passwords). It can use multiple cores at a time allowing up to 32 gigabits per second.

[1] - https://github.com/vstakhov/hpenc
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Reply via email to