Den torsdag 21 december 2017 kl. 17:24:30 UTC+1 skrev Jan Kiszka: > On 2017-12-21 16:32, Luca Cuomo wrote: > > Il giorno giovedì 21 dicembre 2017 14:19:48 UTC+1, Jan Kiszka ha scritto: > >> On 2017-12-21 10:05, Luca Cuomo wrote: > >>> Il giorno domenica 10 dicembre 2017 17:34:24 UTC+1, jonas ha scritto: > >>>> Hi, > >>>> > >>>> I'll be making an effort to contribute my work to the master branch of > >>>> Jailhouse within the next couple of weeks. > >>>> > >>>> /Jonas > >>>> > >>>> Den fredag 8 december 2017 kl. 06:47:33 UTC+1 skrev Constantin Petra: > >>>>> Hi, > >>>>> > >>>>> > >>>>> I'm resending the patch(es) that were shared by Jonas a while ago. > >>>>> > >>>>> > >>>>> Best Regards, > >>>>> Constantin > >>>>> > >>>>> > >>>>> On Thu, Dec 7, 2017 at 10:08 PM, Henning Schild > >>>>> <[email protected]> wrote: > >>>>> Hi Claudio, > >>>>> > >>>>> > >>>>> > >>>>> Am Thu, 7 Dec 2017 17:29:45 +0100 > >>>>> > >>>>> schrieb Claudio Scordino <[email protected]>: > >>>>> > >>>>> > >>>>> > >>>>>> Hi guys, > >>>>> > >>>>>> > >>>>> > >>>>>> 2017-08-09 15:23 GMT+02:00 Henning Schild > >>>>> > >>>>>> <[email protected]>: > >>>>> > >>>>>> > >>>>> > >>>>>>> Hey, > >>>>> > >>>>>>> > >>>>> > >>>>>>> unfortunately Jonas never published his overall changes, maybe now > >>>>> > >>>>>>> he understands why i kindly asked him to do so. > >>>>> > >>>>>>> I think Jonas maybe ran into every single problem one could > >>>>> > >>>>>>> encounter on the way, so if you read the thread you will probably > >>>>> > >>>>>>> be able to come up with a similar patch at some point. That would > >>>>> > >>>>>>> be the duplication of efforts, getting a first working patch into a > >>>>> > >>>>>>> mergeable form is another story. > >>>>> > >>>>>>> > >>>>> > >>>>>>> If there are legal reasons to not publish code on the list i suggest > >>>>> > >>>>>>> you exchange patches between each other. But of cause i would like > >>>>> > >>>>>>> to see contributions eventually ;). > >>>>> > >>>>>>> > >>>>> > >>>>>>> > >>>>> > >>>>>> We need to run IVSHMEM on the TX1. > >>>>> > >>>>>> Any chance of upstreaming those patches to not waste time > >>>>> > >>>>>> re-inventing the wheel ? > >>>>> > >>>>>> If that's not possible, please send me a copy privately. > >>>>> > >>>>> > >>>>> > >>>>> Unfortunately i do not have those patches either. I am afraid someone > >>>>> > >>>>> will have to do that over again. > >>>>> > >>>>> > >>>>> > >>>>> But the whole thread is basically about enabling the demo, which is > >>>>> > >>>>> interesting for people just getting started with ivshmem. And for > >>>>> > >>>>> people that want to implement their own protocol on top of it. > >>>>> > >>>>> If you are just looking at running ivshmem-net you are good to go, that > >>>>> > >>>>> code is in a working state. > >>>>> > >>>>> > >>>>> > >>>>> Henning > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>> Many thanks and best regards, > >>>>> > >>>>>> > >>>>> > >>>>>> Claudio > >>> > >>> Hi all, > >>> > >>> i've applied the provided patch and i'm trying to connect the linux root > >>> cell with the bare metal cell running the inmate/demo/arm/ivshmem-demo.c. > >>> I've attached the used configurations (jetson-tx1-ivshmem for the root > >>> cell and the other one for the bare metal). > >>> When i create the bare metal cell the connection between pci devices is > >>> correctly up. > >>> ----------------------------------------------------------------- > >>> Initializing Jailhouse hypervisor on CPU 2 > >>> Code location: 0x0000ffffc0200050 > >>> Page pool usage after early setup: mem 63/16358, remap 64/131072 > >>> Initializing processors: > >>> CPU 2... OK > >>> CPU 1... OK > >>> CPU 3... OK > >>> CPU 0... OK > >>> Adding virtual PCI device 00:0f.0 to cell "Jetson-TX1-ivshmem" > >>> Page pool usage after late setup: mem 74/16358, remap 69/131072 > >>> Activating hypervisor > >>> Adding virtual PCI device 00:0f.0 to cell "jetson-tx1-demo-shmem" > >>> Shared memory connection established: "jetson-tx1-demo-shmem" <--> > >>> "Jetson-TX1-ivshmem" > >>> ------------------------------------------------------------------- > >>> > >>> 1st problem: no PCI device appears in linux (lspci does not return > >>> anything) > >> > >> Are you using a Linux kernel with the Jailhouse-related patches? Did you > >> enable CONFIG_PCI_HOST_GENERIC and CONFIG_PCI_DOMAINS? On most ARM > >> systems, Jailhouse exposes the ivshmem devices via a virtual host bridge. > > > > Yes, i'm using a kernel for jailhouse on Jetson tx1. The kernel has the > > above CONFIGS. In root cell configuration if i put ".pci_is_virtual = 1," > > dmesg shows the message: > > [ 102.100405] jailhouse: CONFIG_OF_OVERLAY disabled > > [ 102.100417] jailhouse: failed to add virtual host controller > > [ 102.100422] The Jailhouse is opening. > > > > If i put it to 0 there is no error message but not pci device is set as > > before. > > > >> > >>> > >>> Then i launch the ivshmem-demo.bin. I've made some modification: > >>> * in inmate/lib/arm-common/pci.c the #define PCI_CFG_BASE > >>> (0x48000000) > >>> as defined in jetson-tx-ivshmem.c: > >>> config.header.platform_info.pci_mmconfig_base. > >>> In the same file i've enabled the print of pci_read/write_config > >>> * in inmate/demos/arm/ivshmem-demo.c i've removed the filter on > >>> class/revision in order to get a suitable pci device with proper > >>> deviceId:vendorId > >>> > >>> When the bare metal starts, it iterate on the memory with a lot of read > >>> > >>> pci_read_config(bdf:0x0, addr:0x0000000000000000, size:0x2), > >>> reg_addr0x48000000 > >>> pci_read_config(bdf:0x1, addr:0x0000000000000000, size:0x2), > >>> reg_addr0x48000100 > >>> pci_read_config(bdf:0x2, addr:0x0000000000000000, size:0x2), > >>> reg_addr0x48000200 > >>> pci_read_config(bdf:0x3, addr:0x0000000000000000, size:0x2), > >>> reg_addr0x48000300 > >>> > >>> .... after a while something happens (follow <---) > >>> > >>> IVSHMEM: Found 1af4:1110 at 07:10.0 <--- > >>> pci_read_config(bdf:0x780, addr:0x0000000000000008, size:0x4), > >>> reg_addr0x48078008 > >>> IVSHMEM: class/revision ff010000, not supported skipping device <--- > >>> //IGNORED > >>> pci_read_config(bdf:0x780, addr:0x0000000000000006, size:0x2), > >>> reg_addr0x48078004 > >>> pci_read_config(bdf:0x780, addr:0x0000000000000034, size:0x1), > >>> reg_addr0x48078034 > >>> IVSHMEM ERROR: device is not MSI-X capable <--- > >>> pci_read_config(bdf:0x780, addr:0x000000000000004c, size:0x4), > >>> reg_addr0x4807804c > >>> pci_read_config(bdf:0x780, addr:0x0000000000000048, size:0x4), > >>> reg_addr0x48078048 > >>> pci_read_config(bdf:0x780, addr:0x0000000000000044, size:0x4), > >>> reg_addr0x48078044 > >>> pci_read_config(bdf:0x780, addr:0x0000000000000040, size:0x4), > >>> reg_addr0x48078040 > >>> IVSHMEM: shmem is at 0x000000007bf00000 <--- > >>> pci_write_config(bdf:0x780, addr:0x0000000000000014, value:0x0, > >>> size:0x4), reg_addr0x48078014 > >>> pci_write_config(bdf:0x780, addr:0x0000000000000010, value:0x7c000000, > >>> size:0x4), reg_addr0x48078010 > >>> IVSHMEM: bar0 is at 0x000000007c000000 <--- > >>> .... > >>> IVSHMEM: mapped shmem and bars, got position 0x0000000000000001 <--- > >>> IVSHMEM: Enabled IRQ:0x9b > >>> IVSHMEM: Enabling IVSHMEM_IRQs > >>> ... > >>> > >>> 2nd problem: > >>> > >>> At the end of devices scanning here is the error: > >>> > >>> > >>> Unhandled data read at 0x10000(1) > >>> > >>> FATAL: unhandled trap (exception class 0x24) > >>> Cell state before exception: > >>> pc: 0000000000001a80 lr: 0000000000001ac0 spsr: 20000005 EL1 > >>> sp: 0000000000003da0 esr: 24 1 1130007 > >>> x0: 0000000070006000 x1: 00000000000000ff x2: 0000000000002238 > >>> x3: ffffffffffffffff x4: 0000000000003de0 x5: 0000000000000002 > >>> x6: 0000000000000000 x7: 0000000000000008 x8: 0000000000003de8 > >>> x9: 0000000000000000 x10: 0000000000000000 x11: 0000000000002690 > >>> x12: 0000000000000000 x13: 0000000000000000 x14: 0000000000000020 > >>> x15: 0000000000000000 x16: 0000000000000000 x17: 0000000000000000 > >>> x18: 0000000000000000 x19: 00000000000000ff x20: 0000000000010000 > >>> x21: 0000000000001000 x22: 00000000000011b8 x23: 00000000ffffffd0 > >>> x24: 0000000000003f70 x25: 000000000000288b x26: 000000000000272c > >>> x27: 0000000000000001 x28: 0000000000000780 x29: 0000000000000000 > >>> > >>> Parking CPU 3 (Cell: "jetson-tx1-demo-shmem") > >>> > >> > >> That indicates a mismatch between the cell configuration and the > >> hardware information that is either hard-coded into the inmate code or > >> otherwise provided to it. I didn't check the ivshmem-demo code but I > >> suspect the location of the virtual PCI host controller is hard-coded. A > >> Linux inmate would get it from a device tree provided to its boot. > >> > >> Jan > > I've checked and the .pci_mmconfig_base is equal to > > inmate/lib/arm-common/pci.c:#define PCI_CFG_BASE. > > The previous error was on a read on a not terminated string. I've fixed the > > problem and the new output looks like this > > > > IVSHMEM: Found 1af4:1110 at 07:10.0 > > IVSHMEM: class/revision ff010000, not supported skipping device //forced to > > be true > > This device was configured in the cell config to be used as virtual > Ethernet (over ivshmem). You should set shmem_protocol on both ends to 0 > in order to define the right type. >
That's right. I added some printouts in the hypervisor for this case in the patch indicating protocol mismatch between root-cell and bare-metal cell configs. > > IVSHMEM ERROR: device is not MSI-X capable > > Not sure is this is now only a warning, but if the demo code still does > not support INTx, event signaling via interrupts will not work on this > host platform. > > Jan > Yes, the printout originates from the ivshmem-demo for x86, which requires support for MSI-X (which is available and supported/used/required by Jailhouse on x86, but not on ARM, if I've understood it correctly). The printout should have been removed in the patch... /Jonas > > IVSHMEM: shmem is at 0x000000007bf00000 > > IVSHMEM: bar0 is at 0x000000007c000000 > > IVSHMEM: mapped shmem and bars, got position 0x0000000000000001 > > IVSHMEM: Enabled IRQ:0x9b > > IVSHMEM: Enabling IVSHMEM_IRQs > > IVSHMEM: Done setting up... > > IVSHMEM: 07:10.0 sending IRQ (by writing 1 to 0x7c00000c) > > IVSHMEM: waiting for interrupt. > > > > It looks like a misconfigured area (bdf, class/revision are not as > > expected). > > > > --Luca > > > > > > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
