Hi, note that I've never used e1000e, but based on my experience with ixgbe:
if you use e1000e in NAPI (RX_POLLING) mode then Linux kernel will queue net_rx_action softirq task and disable RX Interrupts until net_rx_action completes. So, in effect, if you're using the NAPI, there will always be an interrupt moderation effect happening at high enough network loads. However, I can't really match your experience to anything I would expect the driver or NAPI to be doing. From what the e1000e readme[0] tells me, its Interrupt Throttling seems to be following the same idea as in ixgbe and it appears to be using the same ITR hardware feature. The ITR hardware feature is a "cooldown" timer that starts when an rx interrupt is asserted and blocks the assertion of the rx interrupt until the timer has expired. If there is a packet in the rx queue when the timer expires, an interrupt is asserted immediately. Otherwise, it will be asserted the next time a packet is received. However, there is nothing in the ITR or in the NAPI that would cause *lower* latency as the network load *rises*, except when the NAPI is constantly in polling mode because net_rx_action never manages to empty the rx queue, but this only consistently happens when the network load is extremely stable because it is a pretty instable state. Maybe this gives you some more context to think about what is happening in your network - I can't really point at anything specific. Here are some questions: 1) How big are your packets? 2) Are you using normal network sockets, or something like netmap? 3) What hardware and software are you using on the host you send packets from? 4) Do you really always wait for the reception of an echo packet before you send the next one? In general, user space network sockets will give you terrible performance compared to what is possible if you bypass the linux network system. You might want to look at dpdk and netmap, or PF_RING ZC. I don't know if e1000e supports it, but there is also LLDP / busy read sockets to coax some more performance out of linux network sockets. [1] Best, Luke [0] http://downloadmirror.intel.com/20927/eng/e1000.htm#e1000e [1] http://lwn.net/Articles/551284 On 04/21/2015 07:39 AM, Kobi Cohen-Arazi wrote: > Hi All, > > I'm using FC19, kernel 3.14, e1000e driver with I218-v NIC. > > I'm trying to disable any RX interrupt mitigation/throttling/coalescing. > > I tried using InterruptThrottleRate=0, RxIntDelay=0, SmartPowerDownEnable=0, > rx-usecs=0, > but I still see the driver throttle RX interrupts. My application requires > low latency and I'd like to have RX interrupt as soon as packet is received. > > I have a simple test app that compute RTT between 2 nodes. > All it does is to send a packet to a remote node, wait for this packet to > be echoed back and measures the rtt. > I can see that the longer I wait between 2 packets, the longer the rtt is > (until a certain point where it is stabilized). If I shorten the rtt and > send packets faster (send, wait for echo reply, send the next packet) then > the rtt is shorter. There is definitely some kind of interrupt mitigation > and the driver tries to bundle interrupts/recv packets together. > > Besides the above modules params and ethtool coalesce param (rx-usecs) is > there anything else needed in order to completely disable any coalescing / > RX interrupt mitigation? > > Thank you in advance, > - Kobi > > __o > _ `\ <,_ > (*) / (*) > > > > ------------------------------------------------------------------------------ > BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT > Develop your own process in accordance with the BPMN 2 standard > Learn Process modeling best practices with Bonita BPM through live exercises > http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ > source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF > > > > _______________________________________________ > E1000-devel mailing list > E1000-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/e1000-devel > To learn more about Intel® Ethernet, visit > http://communities.intel.com/community/wired > ------------------------------------------------------------------------------ BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
