[OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Ergi Thanasko
Hi all, We have a a few servers conected via 10g nic LACP, some of them have 4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver 2-3GB accumulated throughput. Iperf does about

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Bob Friesenhahn
On Fri, 16 Sep 2016, Ergi Thanasko wrote: Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size. I want to ask the

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Ergi Thanasko
Hi Bob, We were testing it between two similar servers rsynic and copy paste both ways ( read/write) was the same around 300MB/sec average. Of course the speed test on the pools provide higher throughput around 600MB/sec [cid:449A210A-451D-40E1-97EA-98BA5E7FA558@fios-router.home] On Sep 16,

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Michael Talbott
Jumbo frames are a major help. Also, try using multiple streams (break single rsync job into multiple jobs). Also be sure to use rsync native protocol and don't tunnel it over ssh. Then there's bbcp which can push a single copy operation into multiple streams to fully saturate your disks/network

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Guenther Alka
I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex max_buf=4097152 tcp send_buf=2048576 tcp recv_buf=2048576 tcp, NFS lockd servers (ex 1024), NFS number o

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Bob Friesenhahn
On Fri, 16 Sep 2016, Ergi Thanasko wrote: We were testing it between two similar servers rsynic and copy paste both ways ( read/write) was the same around 300MB/sec average. Of course the speed test on the pools provide higher throughput around 600MB/sec I am not sure what 'copy paste' means

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Davide Poletto
Hope not be wrong here but Port Trunking usage deserves its part in the whole picture too: be aware that using Port Trunking (with LACP as per IEEE 802.3ad) between your Servers' NIC and your 10Gb Switching infrastructure - and this happens by aggregating "n" identical ports together on both link's

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Dale Ghent
Are you doing the rsync over ssh? You might want to look into using HPN-SSH: https://www.psc.edu/index.php/hpn-ssh /dale > On Sep 16, 2016, at 1:43 PM, Ergi Thanasko wrote: > > Hi all, > We have a a few servers conected via 10g nic LACP, some of them have 4nic > and some have 6nic in a lin

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Ergi Thanasko
no that is even slower, just rsync over mounted nfs, a multithreaded rsync does work have better performance. [cid:449A210A-451D-40E1-97EA-98BA5E7FA558@fios-router.home] On Sep 16, 2016, at 5:24 PM, Dale Ghent mailto:da...@omniti.com>> wrote: Are you doing the rsync over ssh? You might want t

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Ergi Thanasko
HI Gea, Great info, are you seeing 1000MB/s doing iperf or actually transfer rates rsync, cp bbcp…. > On Sep 16, 2016, at 12:57 PM, Guenther Alka wrote: > > I have made some investigations into 10G and found that 300-400MB/s is > expected with default settings. Improvements are possible up t

Re: [OmniOS-discuss] Network throughout 1GB/sec

2016-09-16 Thread Stephan Budach
Am 17.09.16 um 00:40 schrieb Davide Poletto: Hope not be wrong here but Port Trunking usage deserves its part in the whole picture too: be aware that using Port Trunking (with LACP as per IEEE 802.3ad) between your Servers' NIC and your 10Gb Switching infrastructure - and this happens by aggr