在 2018年4月10日星期二 UTC+8下午10:39:18，Giovani Gracioli写道：
> Updating:
>
> I added
>
> .num_msix_vectors = 1,
> .iommu = 1,
>
> to the root cell .pci_devices config, wrote a simple linux program to write
> to the shared memory (0x800500000) and a simple inmate cell code to read from
> the shared memory. It was able to read the values that were written.
>
> The next step is how to generate interrupts among the cells. I tried to use
> the pci functions (the same used in the ivshmem-demo) on the arm64 but got an
> error. The lib/pci.c is not compiled by arm64.
>
> How should I discover the pci device and send/receive interrupts on arm64?
>
> Thanks
> Giovani
>
> > Thanks for the answers.
> >
> > So, I have changed the config. In the root cell I have the following now:
> >
> > /* IVSHMEM shared memory region for 00:00.0 */ {
> > .phys_start = 0x800400000,
> > .virt_start = 0x800400000,
> > .size = 0x100000,
> > .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
> > JAILHOUSE_MEM_ROOTSHARED, //added MEM_ROOTSHARED
> > },
> >
> > .pci_devices = {
> > /* 00:00.0 */ {
> > .type = JAILHOUSE_PCI_TYPE_IVSHMEM,
> > .bdf = 0 << 3,
> > .bar_mask = {
> > 0xffffff00, 0xffffffff, 0x00000000,
> > 0x00000000, 0x00000000, 0x00000000,
> > },
> > .shmem_region = 3,
> > .shmem_protocol = JAILHOUSE_SHMEM_PROTO_UNDEFINED,
> > //changed from JAILHOUSE_SHMEM_PROTO_VETH to JAILHOUSE_SHMEM_PROTO_UNDEFINED
> > },
> > },
> >
> > In the inmate cell, I have the following:
> >
> > .mem_regions = {
> > ......
> > .....
> > /* IVSHMEM shared memory region for 00:00.0 */ {
> > .phys_start = 0x800400000,
> > .virt_start = 0x800400000,
> > .size = 0x100000,
> > .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
> > JAILHOUSE_MEM_ROOTSHARED, //added JAILHOUSE_MEM_ROOTSHARED
> > },
> > },
> >
> > Considering just the configuration of the cells, are they correct?
> >
> > So correct me if I am wrong. I will need to write a driver in the root-cell
> > (Linux) to send and receive interrupts to/from the shared ivshmem region.
> > The mapped vpci region contains only the registers of the device:
> >
> > 00:00.0 Unassigned class [ff00]: Red Hat, Inc Inter-VM shared memory
> > Subsystem: Red Hat, Inc Inter-VM shared memory
> > Flags: fast devsel, IRQ 38
> > Memory at fc100000 (64-bit, non-prefetchable) [size=256]
> >
> > In this case, the address 0xfc100000.
> >
> > Is the address (0x800400000) that I configured in the cell config file
> > actually the payload memory region?
> >
> > Also, in the another inmate, how can I access the mapped region? I noticed
> > that there is an ivshmem-demo for x86, but there isn't for arm64.
> >
> > Thanks
> >
> > > Hi,
> > >
> > > what you see there are the registers of the ivshmem device, not the
> > > payload memory region. You will have to map the paddr you have in your
> > > cell-config. That paddr can be found in the config space as well, is
> > > just not a regular PCI BAR.
> > >
> > > I would suggest to write a uio driver and base it on
> > > https://github.com/henning-schild-work/ivshmem-guest-code/tree/jailhouse
> > >
> > > Here is the bit that finds the region
> > > https://github.com/henning-schild-work/ivshmem-guest-code/blob/jailhouse/kernel_module/uio/uio_ivshmem.c#L96
> > >
> > > Also see:
> > > https://github.com/henning-schild-work/ivshmem-guest-code/blob/jailhouse/README.jailhouse
> > >
> > > And you should not use JAILHOUSE_SHMEM_PROTO_VETH because you are not
> > > running a network-device on top of your shmem. Use _UNDEFINED or
> > > _CUSTOM instead.
> > >
> > > Henning
> > >
> > > Am Tue, 3 Apr 2018 10:47:31 -0700
> > > schrieb Giovani Gracioli <>:
> > >
> > > > Just another info, after enabling the root cell, I can see the
> > > > virtual pci devices with lspci -v:
> > > >
> > > > 00:00.0 Unassigned class [ff01]: Red Hat, Inc Inter-VM shared memory
> > > > Subsystem: Red Hat, Inc Inter-VM shared memory
> > > > Flags: fast devsel
> > > > Memory at fc100000 (64-bit, non-prefetchable) [disabled]
> > > > [size=256] Memory at <unassigned> (64-bit, non-prefetchable)
> > > > [disabled] Capabilities: [50] MSI-X: Enable- Count=1 Masked-
> > > >
> > > > 00:01.0 Unassigned class [ff01]: Red Hat, Inc Inter-VM shared memory
> > > > Subsystem: Red Hat, Inc Inter-VM shared memory
> > > > Flags: fast devsel
> > > > Memory at fc100100 (64-bit, non-prefetchable) [disabled]
> > > > [size=256]
> > > >
> > > >
> > > >
> > > > > Hello,
> > > > >
> > > > > I would like to share buffers among cells on arm64 (Xilinx
> > > > > ultrascale+). The documentation suggests the use of the ivshmem.
> > > > >
> > > > > In order to use ivshmem, I changed the root cell config as follows
> > > > > (it is based on the zynqmp-zcu102.c original file):
> > > > >
> > > > > - In the .mem_regions:
> > > > >
> > > > > /* IVSHMEM shared memory region for 00:00.0 */ {
> > > > > .phys_start = 0x800400000,
> > > > > .virt_start = 0x800400000,
> > > > > .size = 0x100000,
> > > > > .flags = JAILHOUSE_MEM_READ |
> > > > > JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED, //added
> > > > > JAILHOUSE_MEM_ROOTSHARED },
> > > > >
> > > > > - In the .pci_devices:
> > > > >
> > > > > /* 00:00.0 */ {
> > > > > .type = JAILHOUSE_PCI_TYPE_IVSHMEM,
> > > > > .bdf = 0 << 3,
> > > > > .bar_mask = {
> > > > > 0xffffff00, 0xffffffff, 0x00000000,
> > > > > 0x00000000, 0x00000000, 0x00000000,
> > > > > },
> > > > > .shmem_region = 3,
> > > > > .shmem_protocol =
> > > > > JAILHOUSE_SHMEM_PROTO_VETH, .num_msix_vectors = 1,
> > > > > .iommu = 1,
> > > > > },
> > > > >
> > > > > Then, I load the cell:
> > > > >
> > > > > jailhouse enable zynqmp-zcu102-ivshmem.cell
> > > > >
> > > > > Initializing Jailhouse hypervisor v0.8 (37-g1fa9001) on CPU 1
> > > > > Code location: 0x0000ffffc0200060
> > > > > Page pool usage after early setup: mem 33/995, remap 64/131072
> > > > > Initializing processors:
> > > > > CPU 1... OK
> > > > > CPU 2... OK
> > > > > CPU 0... OK
> > > > > CPU 3... OK
> > > > > Adding virtual PCI device 00:00.0 to cell "ZynqMP-ZCU102"
> > > > > Adding virtual PCI device 00:01.0 to cell "ZynqMP-ZCU102"
> > > > > Page pool usage after late setup: mem 42/995, remap 69/131072
> > > > > Activating hypervisor
> > > > >
> > > > > Then, I wrote a simple user-space program that maps the virtual PCI
> > > > > ivshmem region to user using mmap:
> > > > >
> > > > > memfd = open("/dev/mem", O_RDWR | O_SYNC);
> > > > >
> > > > > mapped_base = mmap(0, MEM_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED,
> > > > > memfd, 0xfc100000 & ~MEM_MASK);
> > > > >
> > > > > mapped_dev_base = mapped_base + (dev_base & MEM_MASK);
> > > > >
> > > > > for(int i = 0; i < 8; i += 4) {
> > > > > *((volatile unsigned int *) (mapped_dev_base + i)) = 1;
> > > > > printf("Address: %p, Read valeu = %d\n", (void
> > > > > *)(mapped_dev_base + i), *((unsigned int *) (mapped_dev_base +
> > > > > i))); }
> > > > >
> > > > > When I ran this program in the root cell, I got the following
> > > > > Unhandled trap:
> > > > >
> > > > > ./ivshmem_test
> > > > > /dev/mem opened.
> > > > > Memory mapped at address 0x7f8dd87000.
> > > > > Unhandled data write at 0xfc100000(4)
> > > > >
> > > > > FATAL: unhandled trap (exception class 0x24)
> > > > > Cell state before exception:
> > > > > pc: 0000000000400904 lr: 00000000004008d8 spsr: 80000000 EL0
> > > > > sp: 0000007fceff8df0 esr: 24 1 1810046
> > > > > x0: 0000007f8dd87000 x1: 0000000000000001 x2: 0000000000000001
> > > > > x3: 0000000000000000 x4: 0000000040100401 x5: 5404000000000000
> > > > > x6: 000000001aec1037 x7: 0000000000000000 x8: 0000000000000040
> > > > > x9: ffffff80ffffffc8 x10: 0000007fceff8df0 x11: 0000007fceff8df0
> > > > > x12: 00000000000003f3 x13: 0000000000000000 x14: 0000000000000000
> > > > > x15: 0000007f8dd8ecc0 x16: 0000000000000000 x17: 0000007f8dc53240
> > > > > x18: 0000000000000a03 x19: 00000000004009a8 x20: 0000000000000000
> > > > > x21: 0000000000000000 x22: 0000000000000000 x23: 0000000000000000
> > > > > x24: 0000000000000000 x25: 0000000000000000 x26: 0000000000000000
> > > > > x27: 0000000000000000 x28: 0000000000000000 x29: 0000007fceff8df0
> > > > >
> > > > > Parking CPU 2 (Cell: "ZynqMP-ZCU102")
> > > > >
> > > > > Does that make sense? Am I missing something in the root cell
> > > > > config or the user-space program is wrong?
> > > > >
> > > > > Best regards
> > > >
Hello Giovani,
I am now also trying to use the shm on ZYNQ UltraScale+. Is your inmates a
baremetal application or Linux? Could you share the simple code of access the
shm? Thank you!
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
