Den torsdag 21 december 2017 kl. 23:47:39 UTC+1 skrev jonas:
> Den torsdag 21 december 2017 kl. 17:24:30 UTC+1 skrev Jan Kiszka:
> > On 2017-12-21 16:32, Luca Cuomo wrote:
> > > Il giorno giovedì 21 dicembre 2017 14:19:48 UTC+1, Jan Kiszka ha scritto:
> > >> On 2017-12-21 10:05, Luca Cuomo wrote:
> > >>> Il giorno domenica 10 dicembre 2017 17:34:24 UTC+1, jonas ha scritto:
> > >>>> Hi,
> > >>>>
> > >>>> I'll be making an effort to contribute my work to the master branch of
> > >>>> Jailhouse within the next couple of weeks.
> > >>>>
> > >>>> /Jonas
> > >>>>
> > >>>> Den fredag 8 december 2017 kl. 06:47:33 UTC+1 skrev Constantin Petra:
> > >>>>> Hi,
> > >>>>>
> > >>>>>
> > >>>>> I'm resending the patch(es) that were shared by Jonas a while ago.
> > >>>>>
> > >>>>>
> > >>>>> Best Regards,
> > >>>>> Constantin
> > >>>>>
> > >>>>>
> > >>>>> On Thu, Dec 7, 2017 at 10:08 PM, Henning Schild
> > >>>>> <[email protected]> wrote:
> > >>>>> Hi Claudio,
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Am Thu, 7 Dec 2017 17:29:45 +0100
> > >>>>>
> > >>>>> schrieb Claudio Scordino <[email protected]>:
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> Hi guys,
> > >>>>>
> > >>>>>>
> > >>>>>
> > >>>>>> 2017-08-09 15:23 GMT+02:00 Henning Schild
> > >>>>>
> > >>>>>> <[email protected]>:
> > >>>>>
> > >>>>>>
> > >>>>>
> > >>>>>>> Hey,
> > >>>>>
> > >>>>>>>
> > >>>>>
> > >>>>>>> unfortunately Jonas never published his overall changes, maybe now
> > >>>>>
> > >>>>>>> he understands why i kindly asked him to do so.
> > >>>>>
> > >>>>>>> I think Jonas maybe ran into every single problem one could
> > >>>>>
> > >>>>>>> encounter on the way, so if you read the thread you will probably
> > >>>>>
> > >>>>>>> be able to come up with a similar patch at some point. That would
> > >>>>>
> > >>>>>>> be the duplication of efforts, getting a first working patch into a
> > >>>>>
> > >>>>>>> mergeable form is another story.
> > >>>>>
> > >>>>>>>
> > >>>>>
> > >>>>>>> If there are legal reasons to not publish code on the list i suggest
> > >>>>>
> > >>>>>>> you exchange patches between each other. But of cause i would like
> > >>>>>
> > >>>>>>> to see contributions eventually ;).
> > >>>>>
> > >>>>>>>
> > >>>>>
> > >>>>>>>
> > >>>>>
> > >>>>>> We need to run IVSHMEM on the TX1.
> > >>>>>
> > >>>>>> Any chance of upstreaming those patches to not waste time
> > >>>>>
> > >>>>>> re-inventing the wheel ?
> > >>>>>
> > >>>>>> If that's not possible, please send me a copy privately.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Unfortunately i do not have those patches either. I am afraid someone
> > >>>>>
> > >>>>> will have to do that over again.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> But the whole thread is basically about enabling the demo, which is
> > >>>>>
> > >>>>> interesting for people just getting started with ivshmem. And for
> > >>>>>
> > >>>>> people that want to implement their own protocol on top of it.
> > >>>>>
> > >>>>> If you are just looking at running ivshmem-net you are good to go,
> > >>>>> that
> > >>>>>
> > >>>>> code is in a working state.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> Henning
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> Many thanks and best regards,
> > >>>>>
> > >>>>>>
> > >>>>>
> > >>>>>> Claudio
> > >>>
> > >>> Hi all,
> > >>>
> > >>> i've applied the provided patch and i'm trying to connect the linux
> > >>> root cell with the bare metal cell running the
> > >>> inmate/demo/arm/ivshmem-demo.c. I've attached the used configurations
> > >>> (jetson-tx1-ivshmem for the root cell and the other one for the bare
> > >>> metal).
> > >>> When i create the bare metal cell the connection between pci devices is
> > >>> correctly up.
> > >>> -----------------------------------------------------------------
> > >>> Initializing Jailhouse hypervisor on CPU 2
> > >>> Code location: 0x0000ffffc0200050
> > >>> Page pool usage after early setup: mem 63/16358, remap 64/131072
> > >>> Initializing processors:
> > >>> CPU 2... OK
> > >>> CPU 1... OK
> > >>> CPU 3... OK
> > >>> CPU 0... OK
> > >>> Adding virtual PCI device 00:0f.0 to cell "Jetson-TX1-ivshmem"
> > >>> Page pool usage after late setup: mem 74/16358, remap 69/131072
> > >>> Activating hypervisor
> > >>> Adding virtual PCI device 00:0f.0 to cell "jetson-tx1-demo-shmem"
> > >>> Shared memory connection established: "jetson-tx1-demo-shmem" <-->
> > >>> "Jetson-TX1-ivshmem"
> > >>> -------------------------------------------------------------------
> > >>>
> > >>> 1st problem: no PCI device appears in linux (lspci does not return
> > >>> anything)
> > >>
> > >> Are you using a Linux kernel with the Jailhouse-related patches? Did you
> > >> enable CONFIG_PCI_HOST_GENERIC and CONFIG_PCI_DOMAINS? On most ARM
> > >> systems, Jailhouse exposes the ivshmem devices via a virtual host bridge.
> > >
> > > Yes, i'm using a kernel for jailhouse on Jetson tx1. The kernel has the
> > > above CONFIGS. In root cell configuration if i put ".pci_is_virtual = 1,"
> > > dmesg shows the message:
> > > [ 102.100405] jailhouse: CONFIG_OF_OVERLAY disabled
> > > [ 102.100417] jailhouse: failed to add virtual host controller
> > > [ 102.100422] The Jailhouse is opening.
> > >
> > > If i put it to 0 there is no error message but not pci device is set as
> > > before.
> > >
> > >>
> > >>>
> > >>> Then i launch the ivshmem-demo.bin. I've made some modification:
> > >>> * in inmate/lib/arm-common/pci.c the #define PCI_CFG_BASE
> > >>> (0x48000000)
> > >>> as defined in jetson-tx-ivshmem.c:
> > >>> config.header.platform_info.pci_mmconfig_base.
> > >>> In the same file i've enabled the print of pci_read/write_config
> > >>> * in inmate/demos/arm/ivshmem-demo.c i've removed the filter on
> > >>> class/revision in order to get a suitable pci device with proper
> > >>> deviceId:vendorId
> > >>>
> > >>> When the bare metal starts, it iterate on the memory with a lot of read
> > >>>
> > >>> pci_read_config(bdf:0x0, addr:0x0000000000000000, size:0x2),
> > >>> reg_addr0x48000000
> > >>> pci_read_config(bdf:0x1, addr:0x0000000000000000, size:0x2),
> > >>> reg_addr0x48000100
> > >>> pci_read_config(bdf:0x2, addr:0x0000000000000000, size:0x2),
> > >>> reg_addr0x48000200
> > >>> pci_read_config(bdf:0x3, addr:0x0000000000000000, size:0x2),
> > >>> reg_addr0x48000300
> > >>>
> > >>> .... after a while something happens (follow <---)
> > >>>
> > >>> IVSHMEM: Found 1af4:1110 at 07:10.0 <---
> > >>> pci_read_config(bdf:0x780, addr:0x0000000000000008, size:0x4),
> > >>> reg_addr0x48078008
> > >>> IVSHMEM: class/revision ff010000, not supported skipping device <---
> > >>> //IGNORED
> > >>> pci_read_config(bdf:0x780, addr:0x0000000000000006, size:0x2),
> > >>> reg_addr0x48078004
> > >>> pci_read_config(bdf:0x780, addr:0x0000000000000034, size:0x1),
> > >>> reg_addr0x48078034
> > >>> IVSHMEM ERROR: device is not MSI-X capable <---
> > >>> pci_read_config(bdf:0x780, addr:0x000000000000004c, size:0x4),
> > >>> reg_addr0x4807804c
> > >>> pci_read_config(bdf:0x780, addr:0x0000000000000048, size:0x4),
> > >>> reg_addr0x48078048
> > >>> pci_read_config(bdf:0x780, addr:0x0000000000000044, size:0x4),
> > >>> reg_addr0x48078044
> > >>> pci_read_config(bdf:0x780, addr:0x0000000000000040, size:0x4),
> > >>> reg_addr0x48078040
> > >>> IVSHMEM: shmem is at 0x000000007bf00000 <---
> > >>> pci_write_config(bdf:0x780, addr:0x0000000000000014, value:0x0,
> > >>> size:0x4), reg_addr0x48078014
> > >>> pci_write_config(bdf:0x780, addr:0x0000000000000010, value:0x7c000000,
> > >>> size:0x4), reg_addr0x48078010
> > >>> IVSHMEM: bar0 is at 0x000000007c000000 <---
> > >>> ....
> > >>> IVSHMEM: mapped shmem and bars, got position 0x0000000000000001 <---
> > >>> IVSHMEM: Enabled IRQ:0x9b
> > >>> IVSHMEM: Enabling IVSHMEM_IRQs
> > >>> ...
> > >>>
> > >>> 2nd problem:
> > >>>
> > >>> At the end of devices scanning here is the error:
> > >>>
> > >>>
> > >>> Unhandled data read at 0x10000(1)
> > >>>
> > >>> FATAL: unhandled trap (exception class 0x24)
> > >>> Cell state before exception:
> > >>> pc: 0000000000001a80 lr: 0000000000001ac0 spsr: 20000005 EL1
> > >>> sp: 0000000000003da0 esr: 24 1 1130007
> > >>> x0: 0000000070006000 x1: 00000000000000ff x2: 0000000000002238
> > >>> x3: ffffffffffffffff x4: 0000000000003de0 x5: 0000000000000002
> > >>> x6: 0000000000000000 x7: 0000000000000008 x8: 0000000000003de8
> > >>> x9: 0000000000000000 x10: 0000000000000000 x11: 0000000000002690
> > >>> x12: 0000000000000000 x13: 0000000000000000 x14: 0000000000000020
> > >>> x15: 0000000000000000 x16: 0000000000000000 x17: 0000000000000000
> > >>> x18: 0000000000000000 x19: 00000000000000ff x20: 0000000000010000
> > >>> x21: 0000000000001000 x22: 00000000000011b8 x23: 00000000ffffffd0
> > >>> x24: 0000000000003f70 x25: 000000000000288b x26: 000000000000272c
> > >>> x27: 0000000000000001 x28: 0000000000000780 x29: 0000000000000000
> > >>>
> > >>> Parking CPU 3 (Cell: "jetson-tx1-demo-shmem")
> > >>>
> > >>
> > >> That indicates a mismatch between the cell configuration and the
> > >> hardware information that is either hard-coded into the inmate code or
> > >> otherwise provided to it. I didn't check the ivshmem-demo code but I
> > >> suspect the location of the virtual PCI host controller is hard-coded. A
> > >> Linux inmate would get it from a device tree provided to its boot.
> > >>
> > >> Jan
> > > I've checked and the .pci_mmconfig_base is equal to
> > > inmate/lib/arm-common/pci.c:#define PCI_CFG_BASE.
> > > The previous error was on a read on a not terminated string. I've fixed
> > > the problem and the new output looks like this
> > >
> > > IVSHMEM: Found 1af4:1110 at 07:10.0
> > > IVSHMEM: class/revision ff010000, not supported skipping device //forced
> > > to be true
> >
> > This device was configured in the cell config to be used as virtual
> > Ethernet (over ivshmem). You should set shmem_protocol on both ends to 0
> > in order to define the right type.
> >
>
> That's right. I added some printouts in the hypervisor for this case in the
> patch indicating protocol mismatch between root-cell and bare-metal cell
> configs.
>
> > > IVSHMEM ERROR: device is not MSI-X capable
> >
> > Not sure is this is now only a warning, but if the demo code still does
> > not support INTx, event signaling via interrupts will not work on this
> > host platform.
> >
> > Jan
> >
>
> Yes, the printout originates from the ivshmem-demo for x86, which requires
> support for MSI-X (which is available and supported/used/required by
> Jailhouse on x86, but not on ARM, if I've understood it correctly). The
> printout should have been removed in the patch...
>
> /Jonas
>
You also need to enable the INTx of the IVSHMEM virtual PCI device in the cell
configuration (of the bare-metal inmate) in order for interrupts be forwarded
by Jailhouse to the inmate (as a result of writing to either the 'Doorbell' or
the 'Local state' register, located at offsets 12 and 16 in the BAR0 area of
the IVSHMEM virtual PCI device).
Adding something like this to the cell configuration (of the bare-metal inmate)
should do the trick:
8<--- Start of snippet --->8
.irqchips = {
/* GIC */ {
.address = 0x01c81000,
.pin_base = 32,
/* Interrupts:
52 of UART 7,
155 for IVSHMEM,
belong to the client */
.pin_bitmap = {
1ULL<<(52-32),
0,
0,
1 << (155-128),
},
},
},
8<--- End of snippet --->8
/Jonas
> > > IVSHMEM: shmem is at 0x000000007bf00000
> > > IVSHMEM: bar0 is at 0x000000007c000000
> > > IVSHMEM: mapped shmem and bars, got position 0x0000000000000001
> > > IVSHMEM: Enabled IRQ:0x9b
> > > IVSHMEM: Enabling IVSHMEM_IRQs
> > > IVSHMEM: Done setting up...
> > > IVSHMEM: 07:10.0 sending IRQ (by writing 1 to 0x7c00000c)
> > > IVSHMEM: waiting for interrupt.
> > >
> > > It looks like a misconfigured area (bdf, class/revision are not as
> > > expected).
> > >
> > > --Luca
> > >
> > >
> > >
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
