I noticed that if I set a higher bdf value (0x0 in one cell config and 0x10 in the another one) in the .pci_devices, I can see two different interrupts mapped in Linux:
38: 2 0 GICv2 136 Edge uio_ivshmem
39: 5 0 GICv2 138 Edge uio_ivshmem
# lspci -vv
00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
Subsystem: Red Hat, Inc Inter-VM shared memory
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- Di-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- -
Latency: 0
Interrupt: pin A routed to IRQ 38
Region 0: Memory at fc100000 (64-bit, non-prefetchable) [size=256]
Kernel driver in use: uio_ivshmem
00:02.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
Subsystem: Red Hat, Inc Inter-VM shared memory
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- Di-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- -
Latency: 0
Interrupt: pin C routed to IRQ 39
Region 0: Memory at fc100100 (64-bit, non-prefetchable) [size=256]
Kernel driver in use: uio_ivshmem
Then, I loaded and started both bare metal cells. The two cells were able to
find PCI devices. However, cell 2 identified a wrong PCI (01:00.0) with a wrong
bdf (256 instead of 0x10):
Found 1af4:1110 at 01:00.0
shmem is at 0x0000000800500000, shmemsz is 0x0000000000100000
bar0 is at 0x00000000fc100100
mapped shmem and bars, got position 0x0000000000000000 - bdf = 256
Note that bdf may have changed after the call to pci_find_device(), 256 instead
of 0x10 and 01:00.0 insteaf of 02:00.0 as shown in Linux.
Also, in Linux, I can send an interrupt to cell 1 by using /dev/uio0 and the
uio_send tool. But when I try to send an interrupt to cell 2 through /dev/uio1,
I get a mmap failed:
[UIO] opening file /dev/uio1
[UIO] count is 1
mmap failed (0x0xffffffffffffffff)
I believe something is still not right in the config files. I attached all of
them here. Any help is appreciated.
> Hello,
>
> I have configured ivshmem interrupts between the root-cell (Linux) and
> another bare-metal cell on the ultrascale+ (arm64) platform.
>
> What I would like to have now is another bare-metal cell, and then interrupts
> among:
>
> Linux <-> bare metal cell 1
> Linux <-> bare metal cell 2
>
> The idea is that bare metal cell 1 uses the shared memory region 0x800400000
> and vpci 0xfc100000 and cell 2 uses 0x800500000 and vpci 0xfc100100.
>
> In my original root-cell configuration, when I add another vPCI, I can see in
> Linux that both devices use the same interrupt number (38). Then, when I
> write to the first doorbell vpci device reg (mapped from 0xfc100000), cell 2
> receives the interrupt, instead of cell 1.
>
> How can I completely separate both vPCI devices, so each one have different
> interrupts in Linux and are routed to different Jailhouse cells?
>
> Best regards,
> Giovani
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
zynqmp-zcu102-cpus0-1.cell
Description: Binary data
/* * Jailhouse, a Linux-based partitioning hypervisor * * Configuration for gic-demo inmate on Xilinx ZynqMP ZCU102 eval board: * 1 CPU, 64K RAM, 1 serial port * * Copyright (c) Siemens AG, 2016 * * Authors: * Jan Kiszka <[email protected]> * * This work is licensed under the terms of the GNU GPL, version 2. See * the COPYING file in the top-level directory. */ #include <jailhouse/types.h> #include <jailhouse/cell-config.h> #define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0]) struct { struct jailhouse_cell_desc cell; __u64 cpus[1]; struct jailhouse_memory mem_regions[3]; struct jailhouse_irqchip irqchips[1]; struct jailhouse_pci_device pci_devices[1]; } __attribute__((packed)) config = { .cell = { .signature = JAILHOUSE_CELL_DESC_SIGNATURE, .revision = JAILHOUSE_CONFIG_REVISION, .name = "level", .flags = JAILHOUSE_CELL_PASSIVE_COMMREG, .cpu_set_size = sizeof(config.cpus), .num_memory_regions = ARRAY_SIZE(config.mem_regions), .num_irqchips = ARRAY_SIZE(config.irqchips), .num_pci_devices = ARRAY_SIZE(config.pci_devices), .vpci_irq_base = 140-32, }, .cpus = { 0x4, }, .mem_regions = { /* UART */ { .phys_start = 0xff010000, .virt_start = 0xff010000, .size = 0x1000, .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED, }, /* RAM */ { .phys_start = 0x800700000, .virt_start = 0, .size = 0x00010000, .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_LOADABLE, }, /* IVSHMEM shared memory region for 00:01.0 */ { .phys_start = 0x800500000, .virt_start = 0x800500000, .size = 0x100000, .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED, }, }, .irqchips = { /* GIC */ { .address = 0xf9010000, .pin_base = 32, .pin_bitmap = { 1 << (54 - 32), 0, 0, (1 << (142 - 128)) }, }, }, .pci_devices = { /* 00:02.0 */ { .type = JAILHOUSE_PCI_TYPE_IVSHMEM, .bdf = 0x10, .bar_mask = { 0xffffff00, 0xffffffff, 0x00000000, 0x00000000, 0x00000000, 0x00000000, }, .shmem_region = 2, .shmem_protocol = JAILHOUSE_SHMEM_PROTO_UNDEFINED, .num_msix_vectors = 0, .domain = 0, }, }, };
/* * Jailhouse, a Linux-based partitioning hypervisor * * Configuration for gic-demo inmate on Xilinx ZynqMP ZCU102 eval board: * 1 CPU, 64K RAM, 1 serial port * * Copyright (c) Siemens AG, 2016 * * Authors: * Jan Kiszka <[email protected]> * * This work is licensed under the terms of the GNU GPL, version 2. See * the COPYING file in the top-level directory. */ #include <jailhouse/types.h> #include <jailhouse/cell-config.h> #define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0]) struct { struct jailhouse_cell_desc cell; __u64 cpus[1]; struct jailhouse_memory mem_regions[3]; struct jailhouse_irqchip irqchips[1]; struct jailhouse_pci_device pci_devices[1]; } __attribute__((packed)) config = { .cell = { .signature = JAILHOUSE_CELL_DESC_SIGNATURE, .revision = JAILHOUSE_CONFIG_REVISION, .name = "spike", .flags = JAILHOUSE_CELL_PASSIVE_COMMREG, .cpu_set_size = sizeof(config.cpus), .num_memory_regions = ARRAY_SIZE(config.mem_regions), .num_irqchips = ARRAY_SIZE(config.irqchips), .num_pci_devices = ARRAY_SIZE(config.pci_devices), .vpci_irq_base = 140-32, }, .cpus = { 0x8, }, .mem_regions = { /* UART */ { .phys_start = 0xff010000, .virt_start = 0xff010000, .size = 0x1000, .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_IO | JAILHOUSE_MEM_ROOTSHARED, }, /* RAM */ { .phys_start = 0x800600000, .virt_start = 0, .size = 0x00010000, .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_LOADABLE, }, /* IVSHMEM shared memory region for 00:00.0 */ { .phys_start = 0x800400000, .virt_start = 0x800400000, .size = 0x100000, .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED, }, }, .irqchips = { /* GIC */ { .address = 0xf9010000, .pin_base = 32, .pin_bitmap = { 1 << (54 - 32), 0, 0, (1 << (140 - 128)) }, }, }, .pci_devices = { /* 00:00.0 */ { .type = JAILHOUSE_PCI_TYPE_IVSHMEM, .bdf = 0, .bar_mask = { 0xffffff00, 0xffffffff, 0x00000000, 0x00000000, 0x00000000, 0x00000000, }, .shmem_region = 2, .shmem_protocol = JAILHOUSE_SHMEM_PROTO_UNDEFINED, .num_msix_vectors = 0, .domain = 0, }, }, };
