Maybe the problem is with the Linux side and/or UIO driver, because when I
start both cell, I can see that both send interrupts to Linux and they are
received. However, Linux can only send interrupts to cell 1.
I also did an user space program that maps the PCI regions using mmap and sends
interrupts through the mapped addresses. I can map region 1 (0xfc100000) with
size 0x100 without program, but for region 2 (0xfc100100) I can only map using
size 4096. With size 0x100, I get an invalid mapping from mmap (maybe some
align problem?). Using 4096 in region 2, I can issue an interrupt, but it is
sent to cell 1, instead of cell 2.
Any advise on how to test this to find the error?
> On 2018-05-03 20:26, Giovani Gracioli wrote:
> > I noticed that if I set a higher bdf value (0x0 in one cell config and 0x10
> > in the another one) in the .pci_devices, I can see two different interrupts
> > mapped in Linux:
> >
> > 38: 2 0 GICv2 136 Edge uio_ivshmem
> > 39: 5 0 GICv2 138 Edge uio_ivshmem
>
> Yes, INTx interrupts are assigned according to slot numbers:
>
> Pin = (bdf >> 3) & 0x3) + 1
>
> With 1 = "Pin A", 2 = "Pin B" and so forth - following PCI
> recommendations. The vpci_irq_base you configure per cell is the
> interrupt baseline for the final SPI:
>
> SPI = vpci_irq_base + pin - 1
>
> >
> > # lspci -vv
> >
> > 00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
> > Subsystem: Red Hat, Inc Inter-VM shared memory
> > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > Stepping- SERR- FastB2B- Di-
> > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> > <TAbort- <MAbort- >SERR- <PERR- -
> > Latency: 0
> > Interrupt: pin A routed to IRQ 38
> > Region 0: Memory at fc100000 (64-bit, non-prefetchable) [size=256]
> > Kernel driver in use: uio_ivshmem
> >
> > 00:02.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
> > Subsystem: Red Hat, Inc Inter-VM shared memory
> > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > Stepping- SERR- FastB2B- Di-
> > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> > <TAbort- <MAbort- >SERR- <PERR- -
> > Latency: 0
> > Interrupt: pin C routed to IRQ 39
> > Region 0: Memory at fc100100 (64-bit, non-prefetchable) [size=256]
> > Kernel driver in use: uio_ivshmem
> >
> > Then, I loaded and started both bare metal cells. The two cells were able
> > to find PCI devices. However, cell 2 identified a wrong PCI (01:00.0) with
> > a wrong bdf (256 instead of 0x10):
> >
> > Found 1af4:1110 at 01:00.0
> > shmem is at 0x0000000800500000, shmemsz is 0x0000000000100000
> > bar0 is at 0x00000000fc100100
> > mapped shmem and bars, got position 0x0000000000000000 - bdf = 256
> >
> > Note that bdf may have changed after the call to pci_find_device(), 256
> > instead of 0x10 and 01:00.0 insteaf of 02:00.0 as shown in Linux.
>
> That might be issues of the driver or the userspace test programs. lspci
> and Linux kernel core messages tell the truth.
>
> >
> > Also, in Linux, I can send an interrupt to cell 1 by using /dev/uio0 and
> > the uio_send tool. But when I try to send an interrupt to cell 2 through
> > /dev/uio1, I get a mmap failed:
> >
> > [UIO] opening file /dev/uio1
> > [UIO] count is 1
> > mmap failed (0x0xffffffffffffffff)
> >
> > I believe something is still not right in the config files. I attached all
> > of them here. Any help is appreciated.
>
> You can find a working setup with a full-meshed ivshmem-net network
> between three cells, which implies two distinct ivshmem devices per cell
> under configs/arm64/zynqmp-zcu102{,-linux-demo,-linux-demo-2}.c. You
> should be able to transfer that to your UIO setup (change shmem_protocol
> for the existing device, for a starter).
>
> Jan
>
> >
> >> Hello,
> >>
> >> I have configured ivshmem interrupts between the root-cell (Linux) and
> >> another bare-metal cell on the ultrascale+ (arm64) platform.
> >>
> >> What I would like to have now is another bare-metal cell, and then
> >> interrupts among:
> >>
> >> Linux <-> bare metal cell 1
> >> Linux <-> bare metal cell 2
> >>
> >> The idea is that bare metal cell 1 uses the shared memory region
> >> 0x800400000 and vpci 0xfc100000 and cell 2 uses 0x800500000 and vpci
> >> 0xfc100100.
> >>
> >> In my original root-cell configuration, when I add another vPCI, I can see
> >> in Linux that both devices use the same interrupt number (38). Then, when
> >> I write to the first doorbell vpci device reg (mapped from 0xfc100000),
> >> cell 2 receives the interrupt, instead of cell 1.
> >>
> >> How can I completely separate both vPCI devices, so each one have
> >> different interrupts in Linux and are routed to different Jailhouse cells?
> >>
> >> Best regards,
> >> Giovani
> >
>
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
