We may be able to detect when we get number of physical functions that can't be correct (i.e. not 1 or 2) and at least log a different message rather than the current warning about insufficient bandwidth.
Thanks, -Don > -----Original Message----- > From: Rustad, Mark D [mailto:[email protected]] > Sent: Thursday, December 26, 2013 9:40 AM > To: Tantilov, Emil S > Cc: [email protected]; Chaitanya Lala > Subject: Re: [E1000-devel] Possible bug in ixgbe-3.19.1 > ixgbe_enumerate_functions() > > On Dec 24, 2013, at 2:30 PM, "Tantilov, Emil S" <[email protected]> > wrote: > > > You are correct - there is a bug in the function when used in a VM since all > devices appear as part of the same bus, which leads to the driver counting all > devices. > > > > I'm not sure if there is a good way around this. When the device is direct > attached in a VM we may not be able to tell if it's part of a multiport NIC. > > > > Thanks, > > Emil > > This is not the only problem in this area. In my experience, all functions of > a > NIC that are direct attached will have a subfunction number of 0 in a VM. This > means that there is no way to make the NIC look like it should in the VM. > > I take the issues in this area to largely be the result of deficiencies in the > hypervisor and that they really should be addressed there. Principally, there > needs to be a hypervisor supporting a PCIe guest architecture. > > >> -----Original Message----- > >> From: Chaitanya Lala [mailto:[email protected]] > >> Sent: Monday, December 23, 2013 5:19 PM > >> To: Tantilov, Emil S; Chaitanya Lala; e1000- > >> [email protected] > >> Subject: Re: [E1000-devel] Possible bug in ixgbe-3.19.1 > >> ixgbe_enumerate_functions() > >> > >> Hi, > >> > >> This is how all devices look. > >> > >> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] > >> (rev > >> 02) > >> Subsystem: Red Hat, Inc Qemu virtual machine > >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> > >> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA > >> [Natoma/Triton II] > >> Subsystem: Red Hat, Inc Qemu virtual machine > >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Latency: 0 > >> > >> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE > >> [Natoma/Triton II] (prog-if 80 [Master]) > >> Subsystem: Red Hat, Inc Qemu virtual machine > >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Latency: 0 > >> Region 0: [virtual] Memory at 000001f0 (32-bit, non- > >> prefetchable) [size=8] > >> Region 1: [virtual] Memory at 000003f0 (type 3, non- > >> prefetchable) [size=1] > >> Region 2: [virtual] Memory at 00000170 (32-bit, non- > >> prefetchable) [size=8] > >> Region 3: [virtual] Memory at 00000370 (type 3, non- > >> prefetchable) [size=1] > >> Region 4: I/O ports at c000 [size=16] > >> Kernel driver in use: ata_piix > >> > >> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > >> Subsystem: Red Hat, Inc Qemu virtual machine > >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Interrupt: pin A routed to IRQ 9 > >> Kernel driver in use: piix4_smbus > >> Kernel modules: i2c-piix4 > >> > >> 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 (prog-if 00 > >> [VGA > >> controller]) > >> Subsystem: Red Hat, Inc Device 1100 > >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Region 0: Memory at f0000000 (32-bit, prefetchable) [size=32M] > >> Region 1: Memory at f2000000 (32-bit, non-prefetchable) [size=4K] > >> Expansion ROM at f2010000 [disabled] [size=64K] > >> Kernel modules: cirrusfb > >> > >> 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device > >> Subsystem: Red Hat, Inc Device 0001 > >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Latency: 0 > >> Interrupt: pin A routed to IRQ 11 > >> Region 0: I/O ports at c020 [size=32] > >> Region 1: Memory at f2020000 (32-bit, non-prefetchable) [size=4K] > >> Expansion ROM at f2030000 [disabled] [size=64K] > >> Capabilities: [40] MSI-X: Enable+ Count=3 Masked- > >> Vector table: BAR=1 offset=00000000 > >> PBA: BAR=1 offset=00000800 > >> Kernel driver in use: virtio-pci > >> > >> 00:04.0 Ethernet controller: Intel Corporation Ethernet Controller > >> 10-Gigabit X540-AT2 (rev 01) > >> Subsystem: Intel Corporation Ethernet Converged Network Adapter > >> X540-T2 > >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx+ > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Latency: 0, Cache Line Size: 64 bytes > >> Interrupt: pin B routed to IRQ 10 > >> Region 0: Memory at f2200000 (32-bit, prefetchable) [size=2M] > >> Region 4: Memory at f2400000 (32-bit, prefetchable) [size=16K] > >> Expansion ROM at f2480000 [disabled] [size=512K] > >> Capabilities: [a0] Express (v2) Endpoint, MSI 00 > >> DevCap: MaxPayload 512 bytes, PhantFunc 0, > >> Latency L0s <512ns, L1 <64us > >> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ > >> FLReset- > >> DevCtl: Report errors: Correctable- Non-Fatal- > >> Fatal- Unsupported- > >> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > >> MaxPayload 256 bytes, MaxReadReq 512 bytes > >> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- > >> AuxPwr- TransPend- > >> LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s > >> L1, Latency L0 <1us, L1 > >> <8us > >> ClockPM- Surprise- LLActRep- BwNot- > >> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- > >> Retrain- CommClk+ > >> ExtSynch- ClockPM- AutWidDis- BWInt- > >> AutBWInt- > >> LnkSta: Speed 5GT/s, Width x8, TrErr- Train- > >> SlotClk- DLActive- BWMgmt- > >> ABWMgmt- > >> DevCap2: Completion Timeout: Range ABCD, > >> TimeoutDis+ > >> DevCtl2: Completion Timeout: 50us to 50ms, > >> TimeoutDis- > >> LnkCtl2: Target Link Speed: 5GT/s, > >> EnterCompliance- SpeedDis-, > >> Selectable De-emphasis: -6dB > >> Transmit Margin: Normal Operating Range, > >> EnterModifiedCompliance- > >> ComplianceSOS- > >> Compliance De-emphasis: -6dB > >> LnkSta2: Current De-emphasis Level: -3.5dB > >> Capabilities: [40] Power Management version 3 > >> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA > >> PME(D0-,D1-,D2-,D3hot-,D3cold-) > >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 > >> PME- > >> Capabilities: [70] MSI-X: Enable+ Count=64 Masked- > >> Vector table: BAR=4 offset=00000000 > >> PBA: BAR=4 offset=00002000 > >> Capabilities: [50] MSI: Enable- Count=1/1 Maskable- > >> 64bit- > >> Address: 00000000 Data: 0000 > >> Kernel driver in use: ixgbe > >> Kernel modules: ixgbe > >> > >> 00:05.0 Ethernet controller: Intel Corporation Ethernet Controller > >> 10-Gigabit X540-AT2 (rev 01) > >> Subsystem: Intel Corporation Ethernet Converged Network Adapter > >> X540-T2 > >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- > >> VGASnoop- ParErr- > >> Stepping- SERR- FastB2B- DisINTx+ > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast > >>> TAbort- <TAbort- > >> <MAbort- >SERR- <PERR- INTx- > >> Latency: 0, Cache Line Size: 64 bytes > >> Interrupt: pin A routed to IRQ 10 > >> Region 0: Memory at f2600000 (32-bit, prefetchable) [size=2M] > >> Region 4: Memory at f2800000 (32-bit, prefetchable) [size=16K] > >> Expansion ROM at f2880000 [disabled] [size=512K] > >> Capabilities: [a0] Express (v2) Endpoint, MSI 00 > >> DevCap: MaxPayload 512 bytes, PhantFunc 0, > >> Latency L0s <512ns, L1 <64us > >> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ > >> FLReset- > >> DevCtl: Report errors: Correctable- Non-Fatal- > >> Fatal- Unsupported- > >> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ > >> MaxPayload 256 bytes, MaxReadReq 512 bytes > >> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- > >> AuxPwr- TransPend- > >> LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s > >> L1, Latency L0 <1us, L1 > >> <8us > >> ClockPM- Surprise- LLActRep- BwNot- > >> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- > >> Retrain- CommClk+ > >> ExtSynch- ClockPM- AutWidDis- BWInt- > >> AutBWInt- > >> LnkSta: Speed 5GT/s, Width x8, TrErr- Train- > >> SlotClk- DLActive- BWMgmt- > >> ABWMgmt- > >> DevCap2: Completion Timeout: Range ABCD, > >> TimeoutDis+ > >> DevCtl2: Completion Timeout: 50us to 50ms, > >> TimeoutDis- > >> LnkCtl2: Target Link Speed: 2.5GT/s, > >> EnterCompliance- SpeedDis-, > >> Selectable De-emphasis: -6dB > >> Transmit Margin: Normal Operating Range, > >> EnterModifiedCompliance- > >> ComplianceSOS- > >> Compliance De-emphasis: -6dB > >> LnkSta2: Current De-emphasis Level: -6dB > >> Capabilities: [40] Power Management version 3 > >> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA > >> PME(D0-,D1-,D2-,D3hot-,D3cold-) > >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 > >> PME- > >> Capabilities: [70] MSI-X: Enable+ Count=64 Masked- > >> Vector table: BAR=4 offset=00000000 > >> PBA: BAR=4 offset=00002000 > >> Capabilities: [50] MSI: Enable- Count=1/1 Maskable- > >> 64bit- > >> Address: 00000000 Data: 0000 > >> Kernel driver in use: ixgbe > >> Kernel modules: ixgbe > >> > >> > >> > >> > >> > >> On 12/23/13 4:13 PM, "Tantilov, Emil S" > >> <[email protected]> wrote: > >> > >>> Can you provide the output of lspci -vvv from the network > >> device in the > >>> VM? > >>> > >>> Thanks, > >>> Emil > >>> > >>>> -----Original Message----- > >>>> From: Chaitanya Lala [mailto:[email protected]] > >>>> Sent: Monday, December 23, 2013 4:03 PM > >>>> To: [email protected] > >>>> Cc: Chaitanya Lala > >>>> Subject: [E1000-devel] Possible bug in ixgbe-3.19.1 > >>>> ixgbe_enumerate_functions() > >>>> > >>>> Hello, > >>>> > >>>> I am testing ixgbe-3.19.1 with my dual port x540 10G Base- > >> T > >>>> card (PCIe > >>>> passthrough into a Ubuntu 12.10 VM). > >>>> I am seeing a warning like this - > >>>> > >>>> "This is not sufficient for optimal performance of this card. For > >>>> optimal performance , at least 80 GT/s of bandwidth is required." > >>>> > >>>> This is not correct because I have a dual 10G port card > >> and > >>>> have a PCIe > >>>> Gen2 based system and the card is in a x8 slot. So B/W should not > >>>> be an issue On tracing the code it seems like a bug in -> > >>>> > >>>> static inline int ixgbe_enumerate_functions(struct ixgbe_adapter > >>>> *adapter) { > >>>> struct list_head *entry; > >>>> int physfns = 0; > >>>> > >>>> /* Some cards can not use the generic count PCIe functions > >>>> method, > >>>> * because they are behind a parent switch, so we hardcode > >>>> these to > >>>> * correct number of ports. > >>>> */ > >>>> if (ixgbe_pcie_from_parent(&adapter->hw)) { > >>>> physfns = 4; > >>>> } else { > >>>> list_for_each(entry, &adapter->pdev- > >>>>> bus_list) { > >>>> <-------------- This counts all pcie devices in the system not just > >>>> the ones matching this card/intel NIC #ifdef CONFIG_PCI_IOV > >>>> struct pci_dev *pdev = > >>>> list_entry(entry, struct pci_dev, > >>>> bus_list); > >>>> > >>>> /* don't count virtual functions > >> */ > >>>> if (pdev->is_virtfn) > >>>> continue; #endif > >>>> > >>>> physfns++; <------ Should only be > >>>> incremented when an Intel NIC is found > >>>> } > >>>> > >>>> I can send a patch but not sure if what I am thinking is the > >>>> correct way to solve this. > >>>> > >>>> Thanks, > >>>> Chaitanya > >>>> > >>>> > >>>> ---------------------------------------------------------- > >> -- > >>>> ------------------ > >>>> Rapidly troubleshoot problems before they affect your business. > >>>> Most IT organizations don't have a clear picture of how > >> application > >>>> performance > >>>> affects their revenue. With AppDynamics, you get 100% visibility > >>>> into your Java,.NET, & PHP application. Start your 15-day FREE > >>>> TRIAL of AppDynamics Pro! > >>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu= > >> /4 > >>>> 140/ostg.clktrk > >>>> _______________________________________________ > >>>> E1000-devel mailing list > >>>> [email protected] > >>>> https://lists.sourceforge.net/lists/listinfo/e1000-devel > >>>> To learn more about Intel® Ethernet, visit > >>>> http://communities.intel.com/community/wired > > > > > > ---------------------------------------------------------------------- > > -------- Rapidly troubleshoot problems before they affect your > > business. Most IT organizations don't have a clear picture of how > > application performance affects their revenue. With AppDynamics, you > > get 100% visibility into your Java,.NET, & PHP application. Start your > > 15-day FREE TRIAL of AppDynamics Pro! > > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c > > lktrk _______________________________________________ > > E1000-devel mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/e1000-devel > > To learn more about Intel® Ethernet, visit > > http://communities.intel.com/community/wired > > -- > Mark Rustad, Networking Division, Intel Corporation ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
