On Dec 24, 2013, at 2:30 PM, "Tantilov, Emil S" <[email protected]> wrote:
> You are correct - there is a bug in the function when used in a VM since all > devices appear as part of the same bus, which leads to the driver counting > all devices. > > I'm not sure if there is a good way around this. When the device is direct > attached in a VM we may not be able to tell if it's part of a multiport NIC. > > Thanks, > Emil This is not the only problem in this area. In my experience, all functions of a NIC that are direct attached will have a subfunction number of 0 in a VM. This means that there is no way to make the NIC look like it should in the VM. I take the issues in this area to largely be the result of deficiencies in the hypervisor and that they really should be addressed there. Principally, there needs to be a hypervisor supporting a PCIe guest architecture. >> -----Original Message----- >> From: Chaitanya Lala [mailto:[email protected]] >> Sent: Monday, December 23, 2013 5:19 PM >> To: Tantilov, Emil S; Chaitanya Lala; e1000- >> [email protected] >> Subject: Re: [E1000-devel] Possible bug in ixgbe-3.19.1 >> ixgbe_enumerate_functions() >> >> Hi, >> >> This is how all devices look. >> >> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC >> [Natoma] (rev >> 02) >> Subsystem: Red Hat, Inc Qemu virtual machine >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx- >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> >> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA >> [Natoma/Triton II] >> Subsystem: Red Hat, Inc Qemu virtual machine >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx- >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Latency: 0 >> >> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE >> [Natoma/Triton >> II] (prog-if 80 [Master]) >> Subsystem: Red Hat, Inc Qemu virtual machine >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx- >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Latency: 0 >> Region 0: [virtual] Memory at 000001f0 (32-bit, non- >> prefetchable) [size=8] >> Region 1: [virtual] Memory at 000003f0 (type 3, non- >> prefetchable) [size=1] >> Region 2: [virtual] Memory at 00000170 (32-bit, non- >> prefetchable) [size=8] >> Region 3: [virtual] Memory at 00000370 (type 3, non- >> prefetchable) [size=1] >> Region 4: I/O ports at c000 [size=16] >> Kernel driver in use: ata_piix >> >> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI >> (rev 03) >> Subsystem: Red Hat, Inc Qemu virtual machine >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx- >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Interrupt: pin A routed to IRQ 9 >> Kernel driver in use: piix4_smbus >> Kernel modules: i2c-piix4 >> >> 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 >> (prog-if 00 [VGA >> controller]) >> Subsystem: Red Hat, Inc Device 1100 >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx- >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Region 0: Memory at f0000000 (32-bit, prefetchable) >> [size=32M] >> Region 1: Memory at f2000000 (32-bit, non-prefetchable) >> [size=4K] >> Expansion ROM at f2010000 [disabled] [size=64K] >> Kernel modules: cirrusfb >> >> 00:03.0 Ethernet controller: Red Hat, Inc Virtio network >> device >> Subsystem: Red Hat, Inc Device 0001 >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx- >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Latency: 0 >> Interrupt: pin A routed to IRQ 11 >> Region 0: I/O ports at c020 [size=32] >> Region 1: Memory at f2020000 (32-bit, non-prefetchable) >> [size=4K] >> Expansion ROM at f2030000 [disabled] [size=64K] >> Capabilities: [40] MSI-X: Enable+ Count=3 Masked- >> Vector table: BAR=1 offset=00000000 >> PBA: BAR=1 offset=00000800 >> Kernel driver in use: virtio-pci >> >> 00:04.0 Ethernet controller: Intel Corporation Ethernet >> Controller >> 10-Gigabit X540-AT2 (rev 01) >> Subsystem: Intel Corporation Ethernet Converged Network >> Adapter X540-T2 >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx+ >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Latency: 0, Cache Line Size: 64 bytes >> Interrupt: pin B routed to IRQ 10 >> Region 0: Memory at f2200000 (32-bit, prefetchable) >> [size=2M] >> Region 4: Memory at f2400000 (32-bit, prefetchable) >> [size=16K] >> Expansion ROM at f2480000 [disabled] [size=512K] >> Capabilities: [a0] Express (v2) Endpoint, MSI 00 >> DevCap: MaxPayload 512 bytes, PhantFunc 0, >> Latency L0s <512ns, L1 <64us >> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ >> FLReset- >> DevCtl: Report errors: Correctable- Non-Fatal- >> Fatal- Unsupported- >> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ >> MaxPayload 256 bytes, MaxReadReq 512 bytes >> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- >> AuxPwr- TransPend- >> LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s >> L1, Latency L0 <1us, L1 >> <8us >> ClockPM- Surprise- LLActRep- BwNot- >> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- >> Retrain- CommClk+ >> ExtSynch- ClockPM- AutWidDis- BWInt- >> AutBWInt- >> LnkSta: Speed 5GT/s, Width x8, TrErr- Train- >> SlotClk- DLActive- BWMgmt- >> ABWMgmt- >> DevCap2: Completion Timeout: Range ABCD, >> TimeoutDis+ >> DevCtl2: Completion Timeout: 50us to 50ms, >> TimeoutDis- >> LnkCtl2: Target Link Speed: 5GT/s, >> EnterCompliance- SpeedDis-, >> Selectable De-emphasis: -6dB >> Transmit Margin: Normal Operating Range, >> EnterModifiedCompliance- >> ComplianceSOS- >> Compliance De-emphasis: -6dB >> LnkSta2: Current De-emphasis Level: -3.5dB >> Capabilities: [40] Power Management version 3 >> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA >> PME(D0-,D1-,D2-,D3hot-,D3cold-) >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 >> PME- >> Capabilities: [70] MSI-X: Enable+ Count=64 Masked- >> Vector table: BAR=4 offset=00000000 >> PBA: BAR=4 offset=00002000 >> Capabilities: [50] MSI: Enable- Count=1/1 Maskable- >> 64bit- >> Address: 00000000 Data: 0000 >> Kernel driver in use: ixgbe >> Kernel modules: ixgbe >> >> 00:05.0 Ethernet controller: Intel Corporation Ethernet >> Controller >> 10-Gigabit X540-AT2 (rev 01) >> Subsystem: Intel Corporation Ethernet Converged Network >> Adapter X540-T2 >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- >> VGASnoop- ParErr- >> Stepping- SERR- FastB2B- DisINTx+ >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >>> TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Latency: 0, Cache Line Size: 64 bytes >> Interrupt: pin A routed to IRQ 10 >> Region 0: Memory at f2600000 (32-bit, prefetchable) >> [size=2M] >> Region 4: Memory at f2800000 (32-bit, prefetchable) >> [size=16K] >> Expansion ROM at f2880000 [disabled] [size=512K] >> Capabilities: [a0] Express (v2) Endpoint, MSI 00 >> DevCap: MaxPayload 512 bytes, PhantFunc 0, >> Latency L0s <512ns, L1 <64us >> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ >> FLReset- >> DevCtl: Report errors: Correctable- Non-Fatal- >> Fatal- Unsupported- >> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ >> MaxPayload 256 bytes, MaxReadReq 512 bytes >> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- >> AuxPwr- TransPend- >> LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s >> L1, Latency L0 <1us, L1 >> <8us >> ClockPM- Surprise- LLActRep- BwNot- >> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- >> Retrain- CommClk+ >> ExtSynch- ClockPM- AutWidDis- BWInt- >> AutBWInt- >> LnkSta: Speed 5GT/s, Width x8, TrErr- Train- >> SlotClk- DLActive- BWMgmt- >> ABWMgmt- >> DevCap2: Completion Timeout: Range ABCD, >> TimeoutDis+ >> DevCtl2: Completion Timeout: 50us to 50ms, >> TimeoutDis- >> LnkCtl2: Target Link Speed: 2.5GT/s, >> EnterCompliance- SpeedDis-, >> Selectable De-emphasis: -6dB >> Transmit Margin: Normal Operating Range, >> EnterModifiedCompliance- >> ComplianceSOS- >> Compliance De-emphasis: -6dB >> LnkSta2: Current De-emphasis Level: -6dB >> Capabilities: [40] Power Management version 3 >> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA >> PME(D0-,D1-,D2-,D3hot-,D3cold-) >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 >> PME- >> Capabilities: [70] MSI-X: Enable+ Count=64 Masked- >> Vector table: BAR=4 offset=00000000 >> PBA: BAR=4 offset=00002000 >> Capabilities: [50] MSI: Enable- Count=1/1 Maskable- >> 64bit- >> Address: 00000000 Data: 0000 >> Kernel driver in use: ixgbe >> Kernel modules: ixgbe >> >> >> >> >> >> On 12/23/13 4:13 PM, "Tantilov, Emil S" >> <[email protected]> wrote: >> >>> Can you provide the output of lspci -vvv from the network >> device in the >>> VM? >>> >>> Thanks, >>> Emil >>> >>>> -----Original Message----- >>>> From: Chaitanya Lala [mailto:[email protected]] >>>> Sent: Monday, December 23, 2013 4:03 PM >>>> To: [email protected] >>>> Cc: Chaitanya Lala >>>> Subject: [E1000-devel] Possible bug in ixgbe-3.19.1 >>>> ixgbe_enumerate_functions() >>>> >>>> Hello, >>>> >>>> I am testing ixgbe-3.19.1 with my dual port x540 10G Base- >> T >>>> card (PCIe >>>> passthrough into a Ubuntu 12.10 VM). >>>> I am seeing a warning like this - >>>> >>>> "This is not sufficient for optimal performance of this >>>> card. For optimal >>>> performance , at least 80 GT/s of bandwidth is required." >>>> >>>> This is not correct because I have a dual 10G port card >> and >>>> have a PCIe >>>> Gen2 based system and the card is in a x8 slot. So B/W >>>> should not be an >>>> issue >>>> On tracing the code it seems like a bug in -> >>>> >>>> static inline int ixgbe_enumerate_functions(struct >>>> ixgbe_adapter *adapter) >>>> { >>>> struct list_head *entry; >>>> int physfns = 0; >>>> >>>> /* Some cards can not use the generic count PCIe >>>> functions method, >>>> * because they are behind a parent switch, so we >>>> hardcode these to >>>> * correct number of ports. >>>> */ >>>> if (ixgbe_pcie_from_parent(&adapter->hw)) { >>>> physfns = 4; >>>> } else { >>>> list_for_each(entry, &adapter->pdev- >>>>> bus_list) { >>>> <-------------- This counts all pcie devices in the system >>>> not just the >>>> ones matching this card/intel NIC >>>> #ifdef CONFIG_PCI_IOV >>>> struct pci_dev *pdev = >>>> list_entry(entry, struct >>>> pci_dev, >>>> bus_list); >>>> >>>> /* don't count virtual functions >> */ >>>> if (pdev->is_virtfn) >>>> continue; >>>> #endif >>>> >>>> physfns++; <------ Should only be >>>> incremented >>>> when an Intel NIC is found >>>> } >>>> >>>> I can send a patch but not sure if what I am thinking is >>>> the correct way >>>> to solve this. >>>> >>>> Thanks, >>>> Chaitanya >>>> >>>> >>>> ---------------------------------------------------------- >> -- >>>> ------------------ >>>> Rapidly troubleshoot problems before they affect your >>>> business. Most IT >>>> organizations don't have a clear picture of how >> application >>>> performance >>>> affects their revenue. With AppDynamics, you get 100% >>>> visibility into your >>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL >>>> of AppDynamics Pro! >>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu= >> /4 >>>> 140/ostg.clktrk >>>> _______________________________________________ >>>> E1000-devel mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/e1000-devel >>>> To learn more about Intel® Ethernet, visit >>>> http://communities.intel.com/community/wired > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > _______________________________________________ > E1000-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/e1000-devel > To learn more about Intel® Ethernet, visit > http://communities.intel.com/community/wired -- Mark Rustad, Networking Division, Intel Corporation
signature.asc
Description: Message signed with OpenPGP using GPGMail
------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
