Am Mon, 1 Oct 2018 17:38:27 +0200
schrieb Claudio Scordino <[email protected]>:
> Hi Jan,
>
> Il 28/09/2018 14:11, Jan Kiszka ha scritto:
> > On 28.09.18 12:07, Claudio Scordino wrote:
> >> Dear all,
> >>
> >> I'm implementing a minimal inmate driver for i210 and I wonder if
> >> I have understood correctly the usage of the MSI-X functions.
> >>
> >> Once the correct MSI-X bar (BAR3, in my case) has been mapped, I
> >> need to invoke both:
> >>
> >> int_set_handler(IRQ_VECTOR, irq_handler);
> >> pci_msix_set_vector(bdf, IRQ_VECTOR, 0);
> >>
> >> Is IRQ_VECTOR the value reported by lspci ("pin A routed to IRQ
> >> 18") or is it the value reported by /proc/interrupts (129, in my
> >> specific case) ?
> >
> > No, it is a free APIC vector in your setup. Can be anything >= 32.
> >
> >>
> >> BTW, what is the third argument of pci_msix_set_vector() supposed
> >> to contain ?
> >
> > If your device is able and configured to generate multiple MSI-X
> > vectors (e.g. one vector per queue, one for maintenance etc.), this
> > links them to the desired APIC vector.
>
> Thank you for the clarifications.
>
> The device seems to be able to either use a single or a multiple
> MSI-X vector (based on the value of a register). However, in both
> cases I can't get any interrupt on reception and I can't figure out a
> simple way for understanding if it is a device or a cell
> misconfiguration. The platform is x86 with Intel i210, programmed to
> use only the first queue. Polling (similar to the e1000-demo) works
> fine.
>
> This is the lspci output (MSI-X is available on BAR3):
>
> 03:00.0 Ethernet controller: Intel Corporation I210 Gigabit
> Network Connection (rev 03) Control: I/O- Mem+ BusMaster+ SpecCycle-
> MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status:
> Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
> <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to
> IRQ 18 Region 0: Memory at df100000 (32-bit, non-prefetchable)
> [size=1M] Region 2: I/O ports at e000 [disabled] [size=32]
> Region 3: Memory at df200000 (32-bit, non-prefetchable)
> [size=16K] Expansion ROM at df000000 [disabled] [size=1M]
> Capabilities: [40] Power Management version 3
> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
> PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable-
> DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1
> Maskable+ 64bit+ Address: 0000000000000000 Data: 0000
> Masking: 00000000 Pending: 00000000
> Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
> Vector table: BAR=3 offset=00000000
> PBA: BAR=3 offset=00002000
> Capabilities: [a0] Express (v2) Endpoint, MSI 00
> DevCap: MaxPayload 512 bytes, PhantFunc 0,
> Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+
> FLReset+ SlotPowerLimit 0.000W DevCtl: Report errors:
> Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag-
> PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq
> 512 bytes DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+
> AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1,
> ASPM L0s L1, Exit Latency L0s <2us, L1 <16us ClockPM- Surprise-
> LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM L1 Enabled; RCB 64
> bytes Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt-
> AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train-
> SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout:
> Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported DevCtl2: Completion
> Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2:
> Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit
> Margin: Normal Operating Range, EnterModifiedCompliance-
> ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current
> De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
> EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
> Capabilities: [100 v2] Advanced Error Reporting UESta: DLP-
> SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC-
> UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO-
> CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt-
> RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP-
> BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP-
> BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error
> Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [140 v1]
> Device Serial Number c4-00-ad-ff-ff-06-d9-72 Capabilities: [1a0 v1]
> Transaction Processing Hints Device specific mode supported Steering
> table in TPH capability structure Kernel driver in use: igb Kernel
> modules: igb
>
>
> Most of the settings of the inmate cell have been taken from the root
> cell and checked against the datasheet:
>
> .pci_devices = {
> { /* Ethernet @03:00.0 */
> .type = JAILHOUSE_PCI_TYPE_DEVICE,
> .domain = 0x0000,
> .bdf = 0x0300, // 03:00.0
> .caps_start = 0,
> .num_caps = 7,
> .num_msi_vectors = 1,
> .msi_64bits = 1, // Tested also 0, since
> the BAR is at 32-bit
>
> .num_msix_vectors = 1, // Tested also 5,
> when device is programmed to use multiple vectors .msix_region_size =
> 0x1000, .msix_address = 0xdf200000,
> },
> },
>
> .pci_caps = {
> /* Ethernet @03:00.0 */
> {
> .id = 0x5,
> .start = 0x50,
> .len = 24,
> .flags = JAILHOUSE_PCICAPS_WRITE,
> },
> {
> .id = 0x11,
> .start = 0x70,
> .len = 12,
> .flags = JAILHOUSE_PCICAPS_WRITE,
> },
> },
>
> BAR0 and BAR3 are correctly mapped, and PCI capabilities discovered.
> The following statements do not report any error:
>
> int_set_handler(32, irq_handler);
> pci_msix_set_vector(bdf, 32, 0);
>
> but the handler doesn't fire when the card receives the Ethernet
> frame.
>
> Unless there is something clearly wrong/missing, I should assume that
> I'm missing something in the device configuration.
MSI-X is DMA, you have to enable the bus master flag on the device.
https://github.com/henning-schild-work/ivshmem-guest-code/commit/6c6089f565ef39ba5dee48832f35db9751e61f11
https://elixir.bootlin.com/linux/v4.19-rc6/source/drivers/pci/pci.c#L3974
Common mistake ...
Henning
> Many thanks and best regards,
>
> Claudio
>
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
