Re: ssh None cipher
On 03/11/14 17:59, Freddie Cash wrote: On Sat, Oct 18, 2014 at 12:32 PM, Allan Jude wrote: On 2014-10-18 13:21, Freddie Cash wrote: On Oct 18, 2014 3:54 AM, "Mark Martinec" wrote: If the purpose of having a none cipher is to have a fast file transfer, then one should be using sysutils/bbcp for that purposes. Uses ssd for authentication, and opens unencrypted channel(s) for the actual data transfer. It's also very fast, can use multiple TCP streams. That's an interesting alternative to rsync, scp, and ftp, but doesn't help with zfs send/recv which is where the none cipher really shines. Without the none cipher, SSH becomes the bottleneck limiting transfers to around 400 Mbps on a gigabit LAN. With the none cipher, the network becomes the bottleneck limiting transfers to around 920 Mbps on the same gigabit LAN. This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. Actually, looking into it, the bbcp command can support a pipe at each end instead of files, so you can actually do a zfs send | zfs receive via bbcp, and use multiple concurrent connections, to get around TCP window stuff when going transatlantic I am going to be trying it out shortly. Note: the other big improvement in newer ssh is the HPN stuff, that is switched on since 9.2 I think. After much finagling and testing, I have managed to incorporate bbcp into my ZFS send/recv script. And it works much better than regular, encrypted SSH in my setup. Regular SSH transfers tended to top out around 400 Mbps, using 100% of 1 CPU. Was not able to get the multi-threadded AES cipher working. SSH connections using the NONE cipher saturated the gigabit link with minimal CPU usage. And a bbcp connection is currently running between 500-800 Mbps (depending on the size of the snpashot), also with minimal CPU usage. NOTE: I expect this be running much better next week, as the receiving pool is currently resilvering a drive, slowing everything down. Got things working using the following bbcp command format: bbcp -N io "zfs send -I pool/fs@snap1 pool/fs@snap2" username@remotesys:"zfs recv -d pool" Have not played with any of the myriad tuning options for bbcp. Just wanted to see if I could get it to work, and how an untuned connection compared to an untuned SSH connection (with and without NONE cipher). So far, I'm impressed. Thanks for the suggestion. It's another tool in the box. :) I have also written an utility[1] to efficiently encrypt and authenticate data transferred over the network using the modern AES-GCM or chacha20-poly1305 ciphers with pre-shared keys (or passwords). It can use multiple cores at a time allowing up to 32 gigabits per second. [1] - https://github.com/vstakhov/hpenc ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
On Sat, Oct 18, 2014 at 12:32 PM, Allan Jude wrote: > On 2014-10-18 13:21, Freddie Cash wrote: > > On Oct 18, 2014 3:54 AM, "Mark Martinec" > > wrote: > >> > >> If the purpose of having a none cipher is to have a fast > >> file transfer, then one should be using sysutils/bbcp > >> for that purposes. Uses ssd for authentication, and > >> opens unencrypted channel(s) for the actual data transfer. > >> It's also very fast, can use multiple TCP streams. > > > > That's an interesting alternative to rsync, scp, and ftp, but doesn't > help > > with zfs send/recv which is where the none cipher really shines. > > > > Without the none cipher, SSH becomes the bottleneck limiting transfers to > > around 400 Mbps on a gigabit LAN. With the none cipher, the network > becomes > > the bottleneck limiting transfers to around 920 Mbps on the same gigabit > > LAN. > > > > This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. > > Actually, looking into it, the bbcp command can support a pipe at each > end instead of files, so you can actually do a zfs send | zfs receive > via bbcp, and use multiple concurrent connections, to get around TCP > window stuff when going transatlantic > > I am going to be trying it out shortly. > > Note: the other big improvement in newer ssh is the HPN stuff, that is > switched on since 9.2 I think. After much finagling and testing, I have managed to incorporate bbcp into my ZFS send/recv script. And it works much better than regular, encrypted SSH in my setup. Regular SSH transfers tended to top out around 400 Mbps, using 100% of 1 CPU. Was not able to get the multi-threadded AES cipher working. SSH connections using the NONE cipher saturated the gigabit link with minimal CPU usage. And a bbcp connection is currently running between 500-800 Mbps (depending on the size of the snpashot), also with minimal CPU usage. NOTE: I expect this be running much better next week, as the receiving pool is currently resilvering a drive, slowing everything down. Got things working using the following bbcp command format: bbcp -N io "zfs send -I pool/fs@snap1 pool/fs@snap2" username@remotesys:"zfs recv -d pool" Have not played with any of the myriad tuning options for bbcp. Just wanted to see if I could get it to work, and how an untuned connection compared to an untuned SSH connection (with and without NONE cipher). So far, I'm impressed. Thanks for the suggestion. It's another tool in the box. :) -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
On Sun, Oct 19, 2014 at 10:35 AM, Freddie Cash wrote: > On Oct 19, 2014 12:46 AM, "John-Mark Gurney" wrote: > > > > Freddie Cash wrote this message on Sat, Oct 18, 2014 at 10:21 -0700: > > > On Oct 18, 2014 3:54 AM, "Mark Martinec" > > > > wrote: > > > > > > > > If the purpose of having a none cipher is to have a fast > > > > file transfer, then one should be using sysutils/bbcp > > > > for that purposes. Uses ssd for authentication, and > > > > opens unencrypted channel(s) for the actual data transfer. > > > > It's also very fast, can use multiple TCP streams. > > > > > > That's an interesting alternative to rsync, scp, and ftp, but doesn't > help > > > with zfs send/recv which is where the none cipher really shines. > > > > > > Without the none cipher, SSH becomes the bottleneck limiting transfers > to > > > around 400 Mbps on a gigabit LAN. With the none cipher, the network > becomes > > > the bottleneck limiting transfers to around 920 Mbps on the same > gigabit > > > LAN. > > > > > > This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. > > > > Are you running on HEAD or possibly 10.x (I believe we have OpenSSL > > 1.0.x on 10.x)? > > Nope, 9.2. And I don't think the 6200 series Opterons have AES-NI. > Correction, the AMD Opteron 6200-series of CPUs to support AES-NI. However, these storage boxes use AMD Opteron 6128 CPUs. :( They do not support AES-NI. AES-based ciphers are extremely slow on these systems; the multithreaded AES-based ciphers are better, but nowhere near what the NONE cipher provides. :) sysutils/bbcp is interesting as an alternative, but it's a lot more complex than just enabling NONE in OpenSSH. -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
On 2014-10-20 14:33, Brooks Davis wrote: > On Sat, Oct 18, 2014 at 12:10:28AM -0400, Allan Jude wrote: >> On 2014-10-17 22:43, Benjamin Kaduk wrote: >>> On Fri, 17 Oct 2014, Ben Woods wrote: >>> Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater PC on my local LAN, I came across this bug preventing use of the None cipher: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 I think I could enable the None cipher by recompiling base with a flag in /etc/src.conf. >>> >>> I agree. >>> Is there any harm in enabling this by default, but having the None cipher remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it on my default, but wouldn't have to recompile to enable it. >>> >>> I do not see any immediate and concrete harm that doing so would cause, >>> yet that is insufficient for me to think that doing so would be a good >>> idea. >> >> I've been using openssh-portable from ports with the none cipher patch >> to get around this. >> >> IIRC, upstream openssh refuses to merge the none cipher patches "because >> you shouldn't do that". But I'd vote for having it compiled in and just >> disabled by default. >> >> It will refuse to let you have a shell without encryption, and prints a >> big fat hairy warning when encryption is disabled. > > When Bjoern and I did the merge of the HPN patches we left None disable > by default out of a desire to be conservative with a change we knew some > people didn't like. I think turning it on by default would be fine given > the seatbelts in place to prevent accidental inappropriate use. > > -- Brooks > +1 to this. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: ssh None cipher
On Sat, Oct 18, 2014 at 12:10:28AM -0400, Allan Jude wrote: > On 2014-10-17 22:43, Benjamin Kaduk wrote: > > On Fri, 17 Oct 2014, Ben Woods wrote: > > > >> Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater > >> PC on my local LAN, I came across this bug preventing use of the None > >> cipher: > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 > >> > >> I think I could enable the None cipher by recompiling base with a flag in > >> /etc/src.conf. > > > > I agree. > > > >> Is there any harm in enabling this by default, but having the None cipher > >> remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it > >> on my default, but wouldn't have to recompile to enable it. > > > > I do not see any immediate and concrete harm that doing so would cause, > > yet that is insufficient for me to think that doing so would be a good > > idea. > > I've been using openssh-portable from ports with the none cipher patch > to get around this. > > IIRC, upstream openssh refuses to merge the none cipher patches "because > you shouldn't do that". But I'd vote for having it compiled in and just > disabled by default. > > It will refuse to let you have a shell without encryption, and prints a > big fat hairy warning when encryption is disabled. When Bjoern and I did the merge of the HPN patches we left None disable by default out of a desire to be conservative with a change we knew some people didn't like. I think turning it on by default would be fine given the seatbelts in place to prevent accidental inappropriate use. -- Brooks pgphyLtTmkQdL.pgp Description: PGP signature
Re: ssh None cipher
On 2014-10-19 03:46, John-Mark Gurney wrote: > Freddie Cash wrote this message on Sat, Oct 18, 2014 at 10:21 -0700: >> On Oct 18, 2014 3:54 AM, "Mark Martinec" >> wrote: >>> >>> If the purpose of having a none cipher is to have a fast >>> file transfer, then one should be using sysutils/bbcp >>> for that purposes. Uses ssd for authentication, and >>> opens unencrypted channel(s) for the actual data transfer. >>> It's also very fast, can use multiple TCP streams. >> >> That's an interesting alternative to rsync, scp, and ftp, but doesn't help >> with zfs send/recv which is where the none cipher really shines. >> >> Without the none cipher, SSH becomes the bottleneck limiting transfers to >> around 400 Mbps on a gigabit LAN. With the none cipher, the network becomes >> the bottleneck limiting transfers to around 920 Mbps on the same gigabit >> LAN. >> >> This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. > > Are you running on HEAD or possibly 10.x (I believe we have OpenSSL > 1.0.x on 10.x)? w/ modern processors w/ AES-NI and a modern version of > OpenSSL, you should be able to get much faster speeds than that... I'm > able to get ~200MB/s over lo0 on my HEAD box on a: > CPU: AMD A10-5700 APU with Radeon(tm) HD Graphics(3393.89-MHz K8-class > CPU) > > $ netstat -w 1 -I lo0 > inputlo0 output >packets errs idrops bytespackets errs bytes colls > 39162 0 0 207823548 39162 0 207823548 0 > 26327 0 0 158674156 26327 0 158674156 0 > 38254 0 0 221313096 38254 0 221313096 0 > 41362 0 0 219740344 41362 0 219740344 0 > 40271 0 0 213565272 40271 0 213565272 0 > 37698 0 0 225447008 37698 0 225447008 0 > > while running: > $ ssh 0 dd if=/dev/zero >/dev/null > > This is w/ no special patches to OpenSSL or ssh... > > It could go twice as fast if ssh could use multiple threads to do the > encryption (the processor has 4 cores, 2 would be used for sending, 2 > for receiving)... > There is a patch for threaded AES-CTR in the openssh-portable port. Might be worth benchmarking that. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: ssh None cipher
On Oct 19, 2014 12:46 AM, "John-Mark Gurney" wrote: > > Freddie Cash wrote this message on Sat, Oct 18, 2014 at 10:21 -0700: > > On Oct 18, 2014 3:54 AM, "Mark Martinec" > > wrote: > > > > > > If the purpose of having a none cipher is to have a fast > > > file transfer, then one should be using sysutils/bbcp > > > for that purposes. Uses ssd for authentication, and > > > opens unencrypted channel(s) for the actual data transfer. > > > It's also very fast, can use multiple TCP streams. > > > > That's an interesting alternative to rsync, scp, and ftp, but doesn't help > > with zfs send/recv which is where the none cipher really shines. > > > > Without the none cipher, SSH becomes the bottleneck limiting transfers to > > around 400 Mbps on a gigabit LAN. With the none cipher, the network becomes > > the bottleneck limiting transfers to around 920 Mbps on the same gigabit > > LAN. > > > > This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. > > Are you running on HEAD or possibly 10.x (I believe we have OpenSSL > 1.0.x on 10.x)? Nope, 9.2. And I don't think the 6200 series Opterons have AES-NI. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
Freddie Cash wrote this message on Sat, Oct 18, 2014 at 10:21 -0700: > On Oct 18, 2014 3:54 AM, "Mark Martinec" > wrote: > > > > If the purpose of having a none cipher is to have a fast > > file transfer, then one should be using sysutils/bbcp > > for that purposes. Uses ssd for authentication, and > > opens unencrypted channel(s) for the actual data transfer. > > It's also very fast, can use multiple TCP streams. > > That's an interesting alternative to rsync, scp, and ftp, but doesn't help > with zfs send/recv which is where the none cipher really shines. > > Without the none cipher, SSH becomes the bottleneck limiting transfers to > around 400 Mbps on a gigabit LAN. With the none cipher, the network becomes > the bottleneck limiting transfers to around 920 Mbps on the same gigabit > LAN. > > This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. Are you running on HEAD or possibly 10.x (I believe we have OpenSSL 1.0.x on 10.x)? w/ modern processors w/ AES-NI and a modern version of OpenSSL, you should be able to get much faster speeds than that... I'm able to get ~200MB/s over lo0 on my HEAD box on a: CPU: AMD A10-5700 APU with Radeon(tm) HD Graphics(3393.89-MHz K8-class CPU) $ netstat -w 1 -I lo0 inputlo0 output packets errs idrops bytespackets errs bytes colls 39162 0 0 207823548 39162 0 207823548 0 26327 0 0 158674156 26327 0 158674156 0 38254 0 0 221313096 38254 0 221313096 0 41362 0 0 219740344 41362 0 219740344 0 40271 0 0 213565272 40271 0 213565272 0 37698 0 0 225447008 37698 0 225447008 0 while running: $ ssh 0 dd if=/dev/zero >/dev/null This is w/ no special patches to OpenSSL or ssh... It could go twice as fast if ssh could use multiple threads to do the encryption (the processor has 4 cores, 2 would be used for sending, 2 for receiving)... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
On 2014-10-18 13:21, Freddie Cash wrote: > On Oct 18, 2014 3:54 AM, "Mark Martinec" > wrote: >> >> If the purpose of having a none cipher is to have a fast >> file transfer, then one should be using sysutils/bbcp >> for that purposes. Uses ssd for authentication, and >> opens unencrypted channel(s) for the actual data transfer. >> It's also very fast, can use multiple TCP streams. > > That's an interesting alternative to rsync, scp, and ftp, but doesn't help > with zfs send/recv which is where the none cipher really shines. > > Without the none cipher, SSH becomes the bottleneck limiting transfers to > around 400 Mbps on a gigabit LAN. With the none cipher, the network becomes > the bottleneck limiting transfers to around 920 Mbps on the same gigabit > LAN. > > This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. > ___ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > Actually, looking into it, the bbcp command can support a pipe at each end instead of files, so you can actually do a zfs send | zfs receive via bbcp, and use multiple concurrent connections, to get around TCP window stuff when going transatlantic I am going to be trying it out shortly. Note: the other big improvement in newer ssh is the HPN stuff, that is switched on since 9.2 I think. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: ssh None cipher
On Oct 18, 2014 3:54 AM, "Mark Martinec" wrote: > > If the purpose of having a none cipher is to have a fast > file transfer, then one should be using sysutils/bbcp > for that purposes. Uses ssd for authentication, and > opens unencrypted channel(s) for the actual data transfer. > It's also very fast, can use multiple TCP streams. That's an interesting alternative to rsync, scp, and ftp, but doesn't help with zfs send/recv which is where the none cipher really shines. Without the none cipher, SSH becomes the bottleneck limiting transfers to around 400 Mbps on a gigabit LAN. With the none cipher, the network becomes the bottleneck limiting transfers to around 920 Mbps on the same gigabit LAN. This is between two 8-core AMD Opteron 6200 systems using igb(4) NICs. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
If the purpose of having a none cipher is to have a fast file transfer, then one should be using sysutils/bbcp for that purposes. Uses ssd for authentication, and opens unencrypted channel(s) for the actual data transfer. It's also very fast, can use multiple TCP streams. Mark On 10/18/14 06:10, Allan Jude wrote: On 2014-10-17 22:43, Benjamin Kaduk wrote: On Fri, 17 Oct 2014, Ben Woods wrote: Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater PC on my local LAN, I came across this bug preventing use of the None cipher: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 I think I could enable the None cipher by recompiling base with a flag in /etc/src.conf. I agree. Is there any harm in enabling this by default, but having the None cipher remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it on my default, but wouldn't have to recompile to enable it. I do not see any immediate and concrete harm that doing so would cause, yet that is insufficient for me to think that doing so would be a good idea. -Ben ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" I've been using openssh-portable from ports with the none cipher patch to get around this. IIRC, upstream openssh refuses to merge the none cipher patches "because you shouldn't do that". But I'd vote for having it compiled in and just disabled by default. It will refuse to let you have a shell without encryption, and prints a big fat hairy warning when encryption is disabled. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ssh None cipher
On 2014-10-17 22:43, Benjamin Kaduk wrote: > On Fri, 17 Oct 2014, Ben Woods wrote: > >> Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater >> PC on my local LAN, I came across this bug preventing use of the None >> cipher: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 >> >> I think I could enable the None cipher by recompiling base with a flag in >> /etc/src.conf. > > I agree. > >> Is there any harm in enabling this by default, but having the None cipher >> remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it >> on my default, but wouldn't have to recompile to enable it. > > I do not see any immediate and concrete harm that doing so would cause, > yet that is insufficient for me to think that doing so would be a good > idea. > > -Ben > ___ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > I've been using openssh-portable from ports with the none cipher patch to get around this. IIRC, upstream openssh refuses to merge the none cipher patches "because you shouldn't do that". But I'd vote for having it compiled in and just disabled by default. It will refuse to let you have a shell without encryption, and prints a big fat hairy warning when encryption is disabled. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: ssh None cipher
On Fri, 17 Oct 2014, Ben Woods wrote: > Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater > PC on my local LAN, I came across this bug preventing use of the None > cipher: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 > > I think I could enable the None cipher by recompiling base with a flag in > /etc/src.conf. I agree. > Is there any harm in enabling this by default, but having the None cipher > remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it > on my default, but wouldn't have to recompile to enable it. I do not see any immediate and concrete harm that doing so would cause, yet that is insufficient for me to think that doing so would be a good idea. -Ben ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
ssh None cipher
Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater PC on my local LAN, I came across this bug preventing use of the None cipher: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 I think I could enable the None cipher by recompiling base with a flag in /etc/src.conf. Is there any harm in enabling this by default, but having the None cipher remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it on my default, but wouldn't have to recompile to enable it. Thoughts? -- -- From: Benjamin Woods woods...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
