Hi, On 2015-10-19, at 16:20, Luigi Rizzo <ri...@iet.unipi.it> wrote: > > i would look at the following: > - c states and clock speed - make sure you never go below C1, > and fix the clock speed to max. > Sure these parameters also affect the 10G card, but there > may be strange interaction that trigger the power saving > modes in different ways
I already have powerd_flags="-a max -b max -n max" in rc.conf, which I hope should be enough. > - interrupt moderation (may affect ping latency, > do not remember how it is set in ixl but probably a sysctl ixl(4) describes two sysctls that sound like they control AIM, and they default to off: hw.ixl.dynamic_tx_itr: 0 hw.ixl.dynamic_rx_itr: 0 > - number of queues (32 is a lot i wouldn't use more than 4-8), > may affect cpu-socket affinity With hw.ixl.max_queues=4 in loader.conf, performance is still unchanged. > - tso and flow director - i have seen bad effects of > accelerations so i would run the iperf test with > of these features disabled on both sides, and then enable > them one at a time No change with "ifconfig -tso4 -tso6 -rxcsum -txcsum -lro". How do I turn off flow director? > - queue sizes - the driver seems to use 1024 slots which is > about 1.5 MB queued, which in turn means you have 300us > (and possibly half of that) to drain the queue at 40Gbit/s. > 150-300us may seem an eternity, but if a couple of cores fall > into c7 your budget is gone and the loss will trigger a > retransmission and window halving etc. Also no change with "hw.ixl.ringsz=256" in loader.conf. This is really weird. Lars
signature.asc
Description: Message signed with OpenPGP using GPGMail
