Hi Jan,

Il 28/09/2018 14:11, Jan Kiszka ha scritto:
On 28.09.18 12:07, Claudio Scordino wrote:
Dear all,

I'm implementing a minimal inmate driver for i210 and I wonder if I have
understood correctly the usage of the MSI-X functions.

Once the correct MSI-X bar (BAR3, in my case) has been mapped, I need to invoke
both:

     int_set_handler(IRQ_VECTOR, irq_handler);
     pci_msix_set_vector(bdf, IRQ_VECTOR, 0);

Is IRQ_VECTOR the value reported by lspci ("pin A routed to IRQ 18") or is it
the value reported by /proc/interrupts (129, in my specific case) ?

No, it is a free APIC vector in your setup. Can be anything >= 32.


BTW, what is the third argument of pci_msix_set_vector() supposed to contain ?

If your device is able and configured to generate multiple MSI-X vectors (e.g.
one vector per queue, one for maintenance etc.), this links them to the desired
APIC vector.

Thank you for the clarifications.

The device seems to be able to either use a single or a multiple MSI-X vector 
(based on the value of a register).
However, in both cases I can't get any interrupt on reception and I can't 
figure out a simple way for understanding if it is a device or a cell 
misconfiguration.
The platform is x86 with Intel i210, programmed to use only the first queue. 
Polling (similar to the e1000-demo) works fine.

This is the lspci output (MSI-X  is available on BAR3):

        03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network 
Connection (rev 03)
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 18
        Region 0: Memory at df100000 (32-bit, non-prefetchable) [size=1M]
        Region 2: I/O ports at e000 [disabled] [size=32]
        Region 3: Memory at df200000 (32-bit, non-prefetchable) [size=16K]
        Expansion ROM at df000000 [disabled] [size=1M]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA 
PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
                Vector table: BAR=3 offset=00000000
                PBA: BAR=3 offset=00002000
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 
<64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ 
SlotPowerLimit 0.000W
                DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ 
Unsupported+
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr+ 
TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency 
L0s <2us, L1 <16us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ 
DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, 
OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, 
OBFF Disabled
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, 
EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, 
EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, 
LinkEqualizationRequest-
        Capabilities: [100 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- 
RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- 
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- 
RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140 v1] Device Serial Number c4-00-ad-ff-ff-06-d9-72
        Capabilities: [1a0 v1] Transaction Processing Hints
                Device specific mode supported
                Steering table in TPH capability structure
        Kernel driver in use: igb
        Kernel modules: igb


Most of the settings of the inmate cell have been taken from the root cell and 
checked against the datasheet:

       .pci_devices = {
                  { /* Ethernet @03:00.0 */
                          .type = JAILHOUSE_PCI_TYPE_DEVICE,
                          .domain = 0x0000,
                          .bdf = 0x0300, // 03:00.0
                          .caps_start = 0,
                          .num_caps = 7,
                          .num_msi_vectors = 1,
                          .msi_64bits = 1,  // Tested also 0, since the BAR is 
at 32-bit

                          .num_msix_vectors = 1, // Tested also 5, when device 
is programmed to use multiple vectors
                          .msix_region_size = 0x1000,
                          .msix_address = 0xdf200000,
                  },
          },

          .pci_caps = {
                  /* Ethernet @03:00.0 */
                  {
                          .id = 0x5,
                          .start = 0x50,
                          .len = 24,
                          .flags = JAILHOUSE_PCICAPS_WRITE,
                  },
                  {
                        .id = 0x11,
                        .start = 0x70,
                        .len = 12,
                        .flags = JAILHOUSE_PCICAPS_WRITE,
                  },
          },

BAR0 and BAR3 are correctly mapped, and PCI capabilities discovered.
The following statements do not report any error:
        
        int_set_handler(32, irq_handler);
        pci_msix_set_vector(bdf, 32, 0);

but the handler doesn't fire when the card receives the Ethernet frame.

Unless there is something clearly wrong/missing, I should assume that I'm 
missing something in the device configuration.

Many thanks and best regards,

                      Claudio


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to