Hi Don,
Thanks for your reply.
On Mon, Dec 8, 2014 at 10:04 PM, Skidmore, Donald C
<donald.c.skidm...@intel.com> wrote:
> Hey Stefan,
>
> I'm not sure why you are seeing more rx_no_dma_resource on one version of
> Ubuntu than another. We sadly don't validate against Ubuntu.
Actually, we haven't found a version of Ubuntu that works :).
I guess you're validating against RHEL and SLES?
[...]
> Likewise ATR stands for Application Targeted Receive. It is on by default an
> attempts to direct patch flows to the CPU which the SYN was sent out on. It
> assumes a one to one mapping between queue number and CPU number (what the
> set_irq_affinity script does). If your script is not doing the same it would
> defiantly undermined ATF and could lead to flows being directed and in
> correct queue/CPU. Could differences in your script be causing an issue?
I was talking about the "Ageing Timer Rollover" that was being
discussed in the other thread. Apparently that can't be set from the
HP BIOS. I guess you're talking about the NIC setting that identifies
identical flows. I'm not sure we need ATR, since we expect traffic
coming from 8 src IPs and we assign each src IP to its own core. I
noticed there's an AtrSampleRate setting, but that's not available in
the driver that ships with our kernel.
We don't use set_irq_affinity because we want to associate queues from
NIC 1 to cores from CPU0 (node0) and queues from NIC to cores from
CPU1; we have two processes, each running on one CPU core and each
handling one NIC. The system we run on is a 2-node NUMA with 16 cores
each, so I'm not handling cpus >= 32. As far as I can tell, we're not
assigning the same interrupt / queue to the same CPU or the other way
around.
Snipping from our script ($DEV and $NODE are received as arguments):
for i in `seq 0 7`; do
ip_digit=`expr $i + 10`
ethtool -U $DEV flow-type ip4 src-ip 10.10.10.$ip_digit action $i
done
[...]
startCpu=$(cat /sys/devices/system/node/node${NODE}/cpulist|cut -d- -f1)
IRQS=`grep $DEV-TxRx /proc/interrupts | cut -d: -f1 | tr -d " "`
if [ -z "$IRQS" ]; then
echo "No interrupts assigned to $DEV, maybe device is down?"
exit 1
fi
cpu=$startCpu
for irq in $IRQS; do
mask=$((1<<$cpu))
mask_hex=`printf "%X" $mask`
echo "mask=$mask_hex irq=$irq"
printf " %s" $mask_hex > /proc/irq/$irq/smp_affinity
cpu=`expr $cpu + 1`
done
Thanks,
Stefan.
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
