In Linux/Intel I would use following methodology for performance analysis: Hardware: * turbostat Look for C/P states for cores, frequencies, number of SMIs. [1] * cpufreq-info Look for current driver, frequencies, and governor. * atop Look for interrupt distribution across cores Look for context switches, interrupts. * ethtool -S for stats, look for errors, drops, overruns, missed interrupts, etc -k for offloads, enable GRO/GSO, rss/rps/rfs/xps[0] -g for ring sizes, increase -c for interrupt coalescing
Kernel:
* /proc/net/softirq [2] and /proc/interrupts [3]
Again, distribution, missed, delayed interrupts
(optional) NUMA-affinity
* perf top
Look where kernel spends its time.
* iptables
Look if there are rules (if any) that may affect performance.
* netstat -s, netstat -m
Look for error counters and buffer counts
* sysctl / grub
So much to tweak here. Try increasing hashtable sizes, playing with memory
buffers and other limits.
BSD has alternatives to most of these, e.g. perf -> hwpmc, ethtool -> ifconfig,
softirq -> netisr, menu.lst -> loader.conf
In your case I suppose turning on interrupt coalescing along with enabling
hardware offloads may help.
[0] Comparing mutiqueue support Linux vs FreeBSD
https://wiki.freebsd.org/201305DevSummit/NetworkReceivePerformance/ComparingMutiqueueSupportLinuxvsFreeBSD
[1] You can pin your processor to a specific C-state:
https://gist.github.com/SaveTheRbtz/f5e8d1ca7b55b6a7897b
[2] You can analyze that data with:
https://gist.github.com/SaveTheRbtz/172b2e2eb3cbd96b598d
[3] You can set affinity with:
https://gist.github.com/SaveTheRbtz/8875474
PS. Sorry for so many Linuxisms on freebsd-performance@
> On Feb 5, 2016, at 2:57 PM, Adrian Chadd <[email protected]> wrote:
>
> On 5 February 2016 at 10:15, Meyer, Wolfgang <[email protected]> wrote:
>>
>>
>>> -----Original Message-----
>>> From: [email protected] [mailto:owner-freebsd-
>>> [email protected]] On Behalf Of Allan Jude
>>> Sent: Mittwoch, 3. Februar 2016 22:50
>>> To: [email protected]
>>> Subject: Re: ixgbe: Network performance tuning (#TCP connections)
>>>
>>> On 2016-02-03 16:34, Adrian Chadd wrote:
>>>> hi,
>>>>
>>>> can you share your testing program source?
>>>>
>>>>
>>>> -a
>>>>
>>>
>>> I have a Dual E5-2630 v3 (2x8x 2.40ghz (+HT)) with the Intel X540-AT2 that I
>>> can try to replicate this one to help debug it.
>>>
>>> --
>>> Allan Jude
>>
>> I'll try to do some polishing and removal of cruft next week than I hope I
>> will feel comfortable putting it to the public :-)
>>
>> Not that they are overly sophisticated programs, just some test set-up
>> created in the past and over time one gets used to use it as a sort of
>> private "benchmark".
>
> Please do - it'd be good to see what you're doing and figure out
> what's causing the poor behaviour.
>
> Also having more public benchmarks for testing and reproducibility is
> always good. :)
>
>
>
> -adrian
>
>>
>> Regards,
>> Wolfgang Meyer
>>
>>
>> ________________________________
>>
>> Follow HOB:
>>
>> - HOB: http://www.hob.de/redirect/hob.html
>> - Xing: http://www.hob.de/redirect/xing.html
>> - LinkedIn: http://www.hob.de/redirect/linkedin.html
>> - HOBLink Mobile: http://www.hob.de/redirect/hoblinkmobile.html
>> - Facebook: http://www.hob.de/redirect/facebook.html
>> - Twitter: http://www.hob.de/redirect/twitter.html
>> - YouTube: http://www.hob.de/redirect/youtube.html
>> - E-Mail: http://www.hob.de/redirect/mail.html
>>
>>
>> HOB GmbH & Co. KG
>> Schwadermuehlstr. 3
>> D-90556 Cadolzburg
>>
>> Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic
>>
>> AG Fuerth, HRA 5180
>> Steuer-Nr. 218/163/00107
>> USt-ID-Nr. DE 132747002
>>
>> Komplementaerin HOB electronic Beteiligungs GmbH
>> AG Fuerth, HRB 3416
>> _______________________________________________
>> [email protected] mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-performance
>> To unsubscribe, send any mail to
>> "[email protected]"
> _______________________________________________
> [email protected] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "[email protected]"
signature.asc
Description: Message signed with OpenPGP using GPGMail
