You are correct - there is a bug in the function when used in a VM since all devices appear as part of the same bus, which leads to the driver counting all devices.
I'm not sure if there is a good way around this. When the device is direct attached in a VM we may not be able to tell if it's part of a multiport NIC. Thanks, Emil >-----Original Message----- >From: Chaitanya Lala [mailto:[email protected]] >Sent: Monday, December 23, 2013 5:19 PM >To: Tantilov, Emil S; Chaitanya Lala; e1000- >[email protected] >Subject: Re: [E1000-devel] Possible bug in ixgbe-3.19.1 >ixgbe_enumerate_functions() > >Hi, > >This is how all devices look. > >00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC >[Natoma] (rev >02) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > >00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA >[Natoma/Triton II] > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Latency: 0 > >00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE >[Natoma/Triton >II] (prog-if 80 [Master]) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Latency: 0 > Region 0: [virtual] Memory at 000001f0 (32-bit, non- >prefetchable) [size=8] > Region 1: [virtual] Memory at 000003f0 (type 3, non- >prefetchable) [size=1] > Region 2: [virtual] Memory at 00000170 (32-bit, non- >prefetchable) [size=8] > Region 3: [virtual] Memory at 00000370 (type 3, non- >prefetchable) [size=1] > Region 4: I/O ports at c000 [size=16] > Kernel driver in use: ata_piix > >00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI >(rev 03) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Interrupt: pin A routed to IRQ 9 > Kernel driver in use: piix4_smbus > Kernel modules: i2c-piix4 > >00:02.0 VGA compatible controller: Cirrus Logic GD 5446 >(prog-if 00 [VGA >controller]) > Subsystem: Red Hat, Inc Device 1100 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Region 0: Memory at f0000000 (32-bit, prefetchable) >[size=32M] > Region 1: Memory at f2000000 (32-bit, non-prefetchable) >[size=4K] > Expansion ROM at f2010000 [disabled] [size=64K] > Kernel modules: cirrusfb > >00:03.0 Ethernet controller: Red Hat, Inc Virtio network >device > Subsystem: Red Hat, Inc Device 0001 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Latency: 0 > Interrupt: pin A routed to IRQ 11 > Region 0: I/O ports at c020 [size=32] > Region 1: Memory at f2020000 (32-bit, non-prefetchable) >[size=4K] > Expansion ROM at f2030000 [disabled] [size=64K] > Capabilities: [40] MSI-X: Enable+ Count=3 Masked- > Vector table: BAR=1 offset=00000000 > PBA: BAR=1 offset=00000800 > Kernel driver in use: virtio-pci > >00:04.0 Ethernet controller: Intel Corporation Ethernet >Controller >10-Gigabit X540-AT2 (rev 01) > Subsystem: Intel Corporation Ethernet Converged Network >Adapter X540-T2 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin B routed to IRQ 10 > Region 0: Memory at f2200000 (32-bit, prefetchable) >[size=2M] > Region 4: Memory at f2400000 (32-bit, prefetchable) >[size=16K] > Expansion ROM at f2480000 [disabled] [size=512K] > Capabilities: [a0] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 512 bytes, PhantFunc 0, >Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ >FLReset- > DevCtl: Report errors: Correctable- Non-Fatal- >Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 256 bytes, MaxReadReq 512 bytes > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- >AuxPwr- TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s >L1, Latency L0 <1us, L1 ><8us > ClockPM- Surprise- LLActRep- BwNot- > LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- >Retrain- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- >AutBWInt- > LnkSta: Speed 5GT/s, Width x8, TrErr- Train- >SlotClk- DLActive- BWMgmt- >ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, >TimeoutDis+ > DevCtl2: Completion Timeout: 50us to 50ms, >TimeoutDis- > LnkCtl2: Target Link Speed: 5GT/s, >EnterCompliance- SpeedDis-, >Selectable De-emphasis: -6dB > Transmit Margin: Normal Operating Range, >EnterModifiedCompliance- >ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA >PME(D0-,D1-,D2-,D3hot-,D3cold-) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 >PME- > Capabilities: [70] MSI-X: Enable+ Count=64 Masked- > Vector table: BAR=4 offset=00000000 > PBA: BAR=4 offset=00002000 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- >64bit- > Address: 00000000 Data: 0000 > Kernel driver in use: ixgbe > Kernel modules: ixgbe > >00:05.0 Ethernet controller: Intel Corporation Ethernet >Controller >10-Gigabit X540-AT2 (rev 01) > Subsystem: Intel Corporation Ethernet Converged Network >Adapter X540-T2 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- >VGASnoop- ParErr- >Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>TAbort- <TAbort- ><MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 10 > Region 0: Memory at f2600000 (32-bit, prefetchable) >[size=2M] > Region 4: Memory at f2800000 (32-bit, prefetchable) >[size=16K] > Expansion ROM at f2880000 [disabled] [size=512K] > Capabilities: [a0] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 512 bytes, PhantFunc 0, >Latency L0s <512ns, L1 <64us > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ >FLReset- > DevCtl: Report errors: Correctable- Non-Fatal- >Fatal- Unsupported- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > MaxPayload 256 bytes, MaxReadReq 512 bytes > DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- >AuxPwr- TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s >L1, Latency L0 <1us, L1 ><8us > ClockPM- Surprise- LLActRep- BwNot- > LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- >Retrain- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- >AutBWInt- > LnkSta: Speed 5GT/s, Width x8, TrErr- Train- >SlotClk- DLActive- BWMgmt- >ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, >TimeoutDis+ > DevCtl2: Completion Timeout: 50us to 50ms, >TimeoutDis- > LnkCtl2: Target Link Speed: 2.5GT/s, >EnterCompliance- SpeedDis-, >Selectable De-emphasis: -6dB > Transmit Margin: Normal Operating Range, >EnterModifiedCompliance- >ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA >PME(D0-,D1-,D2-,D3hot-,D3cold-) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 >PME- > Capabilities: [70] MSI-X: Enable+ Count=64 Masked- > Vector table: BAR=4 offset=00000000 > PBA: BAR=4 offset=00002000 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- >64bit- > Address: 00000000 Data: 0000 > Kernel driver in use: ixgbe > Kernel modules: ixgbe > > > > > >On 12/23/13 4:13 PM, "Tantilov, Emil S" ><[email protected]> wrote: > >>Can you provide the output of lspci -vvv from the network >device in the >>VM? >> >>Thanks, >>Emil >> >>>-----Original Message----- >>>From: Chaitanya Lala [mailto:[email protected]] >>>Sent: Monday, December 23, 2013 4:03 PM >>>To: [email protected] >>>Cc: Chaitanya Lala >>>Subject: [E1000-devel] Possible bug in ixgbe-3.19.1 >>>ixgbe_enumerate_functions() >>> >>>Hello, >>> >>>I am testing ixgbe-3.19.1 with my dual port x540 10G Base- >T >>>card (PCIe >>>passthrough into a Ubuntu 12.10 VM). >>>I am seeing a warning like this - >>> >>>"This is not sufficient for optimal performance of this >>>card. For optimal >>>performance , at least 80 GT/s of bandwidth is required." >>> >>>This is not correct because I have a dual 10G port card >and >>>have a PCIe >>>Gen2 based system and the card is in a x8 slot. So B/W >>>should not be an >>>issue >>>On tracing the code it seems like a bug in -> >>> >>>static inline int ixgbe_enumerate_functions(struct >>>ixgbe_adapter *adapter) >>>{ >>> struct list_head *entry; >>> int physfns = 0; >>> >>> /* Some cards can not use the generic count PCIe >>>functions method, >>> * because they are behind a parent switch, so we >>>hardcode these to >>> * correct number of ports. >>> */ >>> if (ixgbe_pcie_from_parent(&adapter->hw)) { >>> physfns = 4; >>> } else { >>> list_for_each(entry, &adapter->pdev- >>>>bus_list) { >>><-------------- This counts all pcie devices in the system >>>not just the >>>ones matching this card/intel NIC >>>#ifdef CONFIG_PCI_IOV >>> struct pci_dev *pdev = >>> list_entry(entry, struct >>>pci_dev, >>>bus_list); >>> >>> /* don't count virtual functions >*/ >>> if (pdev->is_virtfn) >>> continue; >>>#endif >>> >>> physfns++; <------ Should only be >>>incremented >>>when an Intel NIC is found >>> } >>> >>>I can send a patch but not sure if what I am thinking is >>>the correct way >>>to solve this. >>> >>>Thanks, >>>Chaitanya >>> >>> >>>---------------------------------------------------------- >-- >>>------------------ >>>Rapidly troubleshoot problems before they affect your >>>business. Most IT >>>organizations don't have a clear picture of how >application >>>performance >>>affects their revenue. With AppDynamics, you get 100% >>>visibility into your >>>Java,.NET, & PHP application. Start your 15-day FREE TRIAL >>>of AppDynamics Pro! >>>http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu= >/4 >>>140/ostg.clktrk >>>_______________________________________________ >>>E1000-devel mailing list >>>[email protected] >>>https://lists.sourceforge.net/lists/listinfo/e1000-devel >>>To learn more about Intel® Ethernet, visit >>>http://communities.intel.com/community/wired ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
