Hi all. I've been transitioning a file server to OmniOS for many reasons
(abandoning zfsonlinux). But I seem to have one last issue I'd like to resolve.
It seems to be running into a performance issue with samba. I'm not using the
built in zfs smb sharing because I need more flexibility in our environment
than it offers such as sharing subfolders, shadow copies, etc.
I'm running the latest LTS version of Omni on a well equipped server with 2x
Xeon E5-2630 v2 @ 2.60GHz, 128G ECC ram with dual intel 10g NICs, and two dual
LSI 6G SAS cards. The zpool has 8 x raidz2s that have 10 4TB drives in each
totaling 290TB. Disk performance is not an issue.
I've compiled samba 4.2.1 and netatalk from source, got winbind working nicely
with our AD environment. Samba is even happily kerberized. Everything
authenticates and functions correctly. But, while netatalk gives me line speed
performance (120-150MB/s on a gigabit workstation), samba won't budge above
40-60MB/s (same speeds using MacOS and Windows 7,8,2012 clients).
Using the same hardware on CentOS 7 with zfsonlinux, samba gives me just about
the same throughput as netatalk. In linux, I could tune it with socket options
giving it a bigger buffer and it made a big difference. But using the same
options on Omni doesn't seem to have any significant affect (actually seems to
slow it down a bit).
In smb.conf on both OSs:
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=2097152
SO_RCVBUF=2097152
In CentOS7 in sysctl.conf:
net.ipv4.tcp_rmem = 10000000 10000000 10000000
net.ipv4.tcp_wmem = 10000000 10000000 10000000
net.ipv4.tcp_mem = 10000000 10000000 10000000
net.ipv4.tcp_sack = 0
net.core.rmem_max = 524287
net.core.wmem_max = 524287
net.core.rmem_default = 524287
net.core.wmem_default = 524287
net.core.optmem_max = 524287
net.core.netdev_max_backlog = 300000
And then in Omni, I've set these ip properties
root@store3:# ipadm show-prop
PROTO PROPERTY PERM CURRENT PERSISTENT DEFAULT POSSIBLE
tcp max_buf rw 16777216 16777216 1048576
8192-1073741824
tcp recv_buf rw 16777216 16777216 128000
2048-16777216
tcp send_buf rw 16777216 16777216 49152
4096-16777216
I just can't get samba on Omni to go any faster than 60MB/s. I've tried
adjusting those buffers, removing the socket options in smb.conf altogether,
but, to no avail.
Anyone else out there running samba on Omni and getting faster throughput?
Anyone have any ideas of how I could get more throughput with samba?
Thanks,
Michael_______________________________________________
OmniOS-discuss mailing list
[email protected]
http://lists.omniti.com/mailman/listinfo/omnios-discuss
