If your interrupt is constantly moving from one core to another it causes a lot of cache thrash and scheduler thrash which makes your networking very inefficient.
You really don't want to be doing what you're trying to do, it won't work well for you. That said, what was it you thought you were getting? If you want you can enable the irqbalance daemon and let it take care of things for you, otherwise disable it and run our set_irq_affinity script and let it bind each queue to one cpu. I think you'll find that most tests/benchmarks run best if you do the above and don't set what you're trying to set with multiple queues. -----Original Message----- From: wenxu [mailto:wenx05124...@163.com] Sent: Thursday, October 16, 2014 10:56 PM To: Fujinaka, Todd Cc: e1000-devel@lists.sourceforge.net Subject: Re: [E1000-devel] igb driver irq affnity Sorry for my descripion. As you said, one interrupt will only be able to be pinned to one core. On the machine with kernel 3.10.54, An interrupt can be pinned to all the cores. I set the irq affinity to ff (all cores) # cat /proc/irq/52/smp_affinity ff And all the cores can get the interrupt. It seems an interrupt can pinned to several cores. # cat /proc/interrupts | grep eth1-TxRx-0 52: 567216 177639 139883 181744 287601 185403 144114 115098 PCI-MSI-edge eth1-TxRx-0 # cat /proc/interrupts | grep eth1-TxRx-0 52: 567216 177640 139883 181744 287601 185403 144114 115098 PCI-MSI-edge eth1-TxRx-0 ]# cat /proc/interrupts | grep eth1-TxRx-0 52: 567223 177641 139883 181744 287602 185403 144115 115098 PCI-MSI-edge eth1-TxRx-0 Are there something wrong? At 2014-10-17 01:15:52, "Fujinaka, Todd" <todd.fujin...@intel.com> wrote: Maybe I’m not understanding your question. One interrupt will only be able to be pinned to one core. If you have two interrupts, for example, you will be able to pin those to two different cores. Having two cores respond to a single interrupt would be problematic at best. Todd Fujinaka Software Application Engineer Networking Division (ND) Intel Corporation todd.fujin...@intel.com (503) 712-4565 From:蒙奇D小碗豆 [mailto:wenx05124...@163.com] Sent: Wednesday, October 15, 2014 7:16 PM To: Fujinaka, Todd Cc:e1000-devel@lists.sourceforge.net Subject: Re:RE: [E1000-devel] igb driver irq affnity Hi Todd, I can set affinity of an interrupt to multiple cores in kernel 3.10.54, why? It shows PCI-MSI-edge but not IR-PCI-MSI-edge # uname -r 3.10.54 # cat /proc/interrupts 52: 201788 64236 50792 67364 92508 63360 49267 40366 PCI-MSI-edge eth1-TxRx-0 53: 36781 10741 8005 32347 18399 9787 6708 4901 PCI-MSI-edge eth1-TxRx-1 54: 65332 15334 12524 36306 20951 11250 8874 6616 PCI-MSI-edge eth1-TxRx-2 55: 312311 177556 131037 123123 287075 168315 109039 69273 PCI-MSI-edge eth1-TxRx-3 56: 62499 33244 24862 45179 55156 34390 22097 14986 PCI-MSI-edge eth1-TxRx-4 57: 166171 75797 57069 70045 123485 79925 58036 43655 PCI-MSI-edge eth1-TxRx-5 58: 27657 6801 5086 29928 18827 6228 4610 3152 PCI-MSI-edge eth1-TxRx-6 59: 325169 184371 135124 125212 279277 164904 96636 52003 PCI-MSI-edge eth1-TxRx-7 # cat /proc/irq/52/smp_affinity ff BR Xu Wen At 2014-10-16 00:06:06, "Fujinaka, Todd" <todd.fujin...@intel.com> wrote: >Yes. You can only set affinity of an interrupt line to one core. > >Todd Fujinaka >Software Application Engineer >Networking Division (ND) >Intel Corporation >todd.fujin...@intel.com >(503) 712-4565 > >-----Original Message----- >From: 蒙奇D小碗豆 [mailto:wenx05124...@163.com] >Sent: Wednesday, October 15, 2014 3:45 AM >To: e1000-devel@lists.sourceforge.net >Subject: [E1000-devel] igb driver irq affnity > >Hi, > >1. >The netwrok card I350 running with igb driver. There are 8 tx-rx queues, each >queue has a interrupt vector. >I can only set a irq line to one core. If I set the affnity to multiple core, >the hardware interrupt only comes out on one core. ># uname -r >2.6.32-279.19.10.el6.x86_64 > ># cat /proc/interrupt | grep eth1 > 42: 29424 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-0 > 43: 107313 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-1 > 44: 56983 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-2 > 45: 218351 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-3 > 46: 24512 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-4 > 47: 221472 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-5 > 48: 13703 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-6 > 49: 86444 0 0 0 0 0 > 0 0 IR-PCI-MSI-edge eth1-TxRx-7 > ># cat /proc/irq/42/smp_affnity >ff >I also download the newest version of the igb driver, it's also like this. > > >I compiled a new kernel 3.10.54. The interrupt vector can assigned to all the >cores. It shows PCI-MSI-edge but not IR-PCI-MSI-edge # uname -r >3.10.54 > ># cat /proc/interrupt | grep eth1 > 52: 174 6 8 26 30 5 > 4 1 PCI-MSI-edge eth1-TxRx-0 > 53: 9 4 3 22 2 7 > 1 0 PCI-MSI-edge eth1-TxRx-1 > 54: 11 5 3 23 9 4 > 4 0 PCI-MSI-edge eth1-TxRx-2 > 55: 62 24 25 36 47 35 > 21 14 PCI-MSI-edge eth1-TxRx-3 > 56: 95 19 15 27 51 19 > 18 2 PCI-MSI-edge eth1-TxRx-4 > 57: 14 5 0 23 1 2 > 3 0 PCI-MSI-edge eth1-TxRx-5 > 58: 45 4 3 25 8 4 > 1 1 PCI-MSI-edge eth1-TxRx-6 > 59: 122 37 37 44 85 62 > 43 17 PCI-MSI-edge eth1-TxRx-7 > ># cat /proc/irq/42/smp_affnity >ff > >I investigate the kernel code, It seems there are something matter with >interrupt-mapping? How Can I fix this in the 2.6.32-279.19.10.el6.x86_64? > >BR >Xu Wen > > > > ------------------------------------------------------------------------------ Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
