eric wang wrote:
I have been tuning my server for net performance recently and initially I found that at high inbound packet rate (6 digitals), only one CPU is loaded. Setting ip_squeue_fanout=1 and ip_squeue_bind=0 did not help much. Then I contacted Sun and learnt that there is an additional tunable called ip_squeues_per_cpu. I was recommended to set ip_squeues_per_cpu to 32 and ip_soft_rings_cnt to 2 times the value of ip_squeues_per_cpu.
Eric,
The tuning that you have done is rather weird. By default,
ip_squeues_per_cpu has a value of 1. But that does not mean that only
one squeue will be there for a CPU. More squeues upto a maximum of 32
can get allocated for a CPU and they get allocated on demand. So setting
ip_squeues_per_cpu to 32 is not very interesting.
What is interesting is that you have set ip_soft_rings_cnt to 2 times
ip_squeues_per_cpu which would be 64 in your case. This does not seem right.
If you have a 64 CPU system and want to use all of them to process
incoming packets for a NIC, then it may just be ok, but having more soft
rings than the number of CPUs does not give you any performance advantage.
My suggestion is to not change ip_squeues_per_cpu at all but tune
ip_soft_rings_cnt only. Set it to 2 or 3 for 1Gbps NIC and anywhere from
16 to 32 for 10 Gbps NIC. Again don't increase the soft ring count to
more than the number of CPUs in the system. Also note that CPU speed is
also important in the calculation. A simple rule of thumb is that a 1Ghz
CPU should be able to handle a 1Gbps NIC.
-krgopi
The trick is that ndd does not show this tunable and it can only be set in
/etc/system and examined by
mdb -k
ip_squeues_per_cpu/D
I tried it and found my server got good performance improvement. I wonder why
SUN did not mention this tunable in its documents but it is a good one and I
hope it also helps others. Also I hope that someone here who knows the magic
can give me some hint how this tunable works and whether there are additional
secret tunables that I can try
This message posted from opensolaris.org
_______________________________________________
networking-discuss mailing list
[email protected]
_______________________________________________
networking-discuss mailing list
[email protected]
