Hi,

On 6/12/19 9:32 PM, Wayne wrote:
> I'm sorry about the formatting.  It didn't look like that before I sent it.
> 
> I will double check my linux-x86-demo cell config again for the serial
> IRQ mapping. Also I will build in the CONFIG_CRASH_CELL_ON_PANIC define
> for help going forward.
> 
> Can the "jailhouse cell linux" command take an initramfs image (.img)
> through the "-i option" or does it need to be an initrd?  If initrd

Good question. The -i option does nothing else than loading the image -
as a binary - to a RAM region and propagate its location. So I'd assume
that support for different formats only depend if your kernel has
builtin support for the eventual format.

> only, can you point me to an example x86 rootfs.cpio image?  I'm having
> trouble installing Buildroot on my machine.

Buildroot is the perfect swiss army knife for non-root initrds.

  Ralf

> 
> Thanks again,
> Wayne
> 
> On Wed, Jun 12, 2019 at 1:17 PM Ralf Ramsauer
> <[email protected]
> <mailto:[email protected]>> wrote:
> 
>     Hi Wayne,
> 
>     On 6/12/19 3:55 PM, Wayne wrote:
>     > Ralf/Jan,
>     >
>     >  Are you sure you remove the device from the kernel cmdline? *Yes,
>     there
>     > is no ttyS* device in my root linux cmdline*
>     > - Are you sure that there doesn't spawn a tty? Check with
>     systemctl. *No
>     > tty as far as I can tell, verified with "lsof | grep ttyS".  The lsof
>     > command stops working after the crash though*
> 
>     please use plain text mails. It's getting hard to read :-)
> 
>     > - lsof | grep ttyS
>     > - The root-cell might receive the serial device's interrupt, while its
>     >   ports are assigned to the non-root cell. Does the crash happen when
>     >   you type in a character to the serial line? *This is possible,
>     do you
>     > have an example of how I would accomplish assigning the interrupt
>     (IRQ 4
>     > for ttyS0) to the non-root?  To rule out the interrupt concern.*
> 
>     Compare linux-x86-demo.c. The .pin_bitmap field denotes the interrupts
>     that are routed to the cell. If you use the original linux-x86-demo.c,
>     then interrupt 4 should already be routed to the cell.
> 
>     AFAICT, in linux-x86-demo.c, we assign IRQ 3 & 4. In your case, we also
>     remove the other IRQ from the root cell - IRQ 3 should be commented out.
> 
>     BTW, a helpful debugging feature:
>     echo "#define CONFIG_CRASH_CELL_ON_PANIC 1" >>
>     include/hypervisor/config.h
> 
>     This will let the root cell print a stackdump which gives you a hint
>     what codepath caused the crash.
> 
>     >
>     >   In this case, assign the interrupt to the non-root cell. You
>     will need
>     >   it in any case there once you have a initrd and want to type in
>     stuff.
>     >
>     > So there must be some reason why the crash happens. The hard way is to
>     > add 8250.nr_uarts=0 in your kernel cmdline of the root cell. *I
>     can use
>     > this method for now to continue booting the non-root linux.*
>     >
>     >> 
>     >>        if I leave out the pio mapping in the non-root linux, and
>     >> alolothat block in the root cell instead then the non-root linux
>     gets a
>     >> PIO read fault on "0x3F9" while booting.       
>     >>        Ralf mentioned that Linux would not try enumerate the
>     device if
>     >> its set to -1 ( [0x3f8/8 ... 0x3ff/8] = -1)  , but the non-root
>     seems to
>     >> attempt to do something with it anyway:
>     >> 
>     >> "[    0.956146]Serial: 8250/16550 driver, 1 ports, IRQ sharing
>     enabled"
>     >> "FATAL: Invalid PIO read, port: 3f9: size 1"  
>     >
>     > Hmm. At least that's what I thought. I don't know how enumeration of
>     > platform devices works exactly, but reconsidering this, the behaviour
>     > above makes sense: How would it know that a platform device is not
>     > present, depending of the configuration of the pio_bitmap. 
>      *Since I do
>     > technically have two serial ports, is there a way (via command line or
>     > kernel config for ex) to hardcode what the non-root would
>     enumerate for
>     > the UART when its booting up?  Meaning, force it to "see" the 0x2f8
>     > serial port as ttyS0 instead of 0x3f8 when booting up 
>     > and** CONFIG_SERIAL_8250_RUNTIME_UARTS = 1**.  That way if you had say
>     > three serial ports and three cells (1 root, 2 non-root linux) then
>     each
>     > could have their own separate dedicated ttyS0.*
> 
>     Have a look at the documentation of the 8250 driver, don't know by
>     heart.
> 
>     HTH,
>       Ralf
> 
>     > *
>     > *
>     > As far as just booting the non-root besides the serial concern, I
>     > haven't provided an initramfs to my non-root linux yet.  I would
>     like to
>     > probably boot it with a simple ramdisk if possible to get a point
>     where
>     > I can log in. 
>     > *
>     > *
>     > Thanks,
>     > Wayne
>     >
>     > On Wed, Jun 12, 2019 at 6:43 AM Ralf Ramsauer
>     > <[email protected]
>     <mailto:[email protected]>
>     > <mailto:[email protected]
>     <mailto:[email protected]>>> wrote:
>     >
>     >     Hi,
>     >
>     >     On 6/11/19 11:05 PM, Wayne wrote:
>     >     > Jan/Ralf,
>     >     >
>     >     > I was able to get past issues 1 and 3 above by using Jan's
>     kernel
>     >     config
>     >     > from this thread:
>     >     >
>     >     >
>     >   
>      
> https://groups.google.com/forum/#!searchin/jailhouse-dev/Re$3A$20Failed$20to$20boot$20jailhouse%7Csort:relevance/jailhouse-dev/M7UO89XFIk0/Qi40DDuMBAAJ";.
>  
>     >     >
>     >     > The DMA issue was resolved by disabling kernel
>     >     option CONFIG_ISA_DMA_API
>     >     > as stated in another thread, and reflected in the config
>     file above.
>     >     >
>     >     > So now I think i've almost got the non-root linux booting.  I'm
>     >     stuck at
>     >     > the point where its trying to mount a filesystem and since
>     i'm not
>     >     > providing an initramfs it panics.
>     >
>     >     Perfect!
>     >
>     >     >
>     >     > Similar to the output below:
>     >     >
>     >     > [ 1.080178] VFS: Cannot open root device "(null)" or
>     >     unknown-block(0,0):
>     >     > error -6
>     >     > [ 1.087662] Please append a correct "root=" boot option;
>     here are the
>     >     > available partitions:
>     >     > [ 1.096013] Kernel panic - not syncing: VFS: Unable to mount
>     root
>     >     fs on
>     >     > unknown-block(0,0)
>     >     >
>     >     > I have just a couple more questions:
>     >     > 1. In general, is it true that an inmate will takeover control
>     >     from the
>     >     > root of any resource in the following arrays, or are there
>     some cases
>     >     > where they get "shared" between the root and inmate?
>     >
>     >     Depends.
>     >
>     >     >      A. ".mem_regions" -> Shared?
>     >
>     >     May be shared with the root cell (i.e., if the region has the flag
>     >     JAILHOUSE_MEM_ROOTSHARED set).
>     >
>     >     >      B. ".irqchips" -> Shared?
>     >
>     >     Shared with whom? Only one guest runs on a CPU. This guest
>     will have
>     >     full access to the irqchip. Some accesses must be intercepted.
>     >
>     >     >      C. ".pio_bitmap" -> Inmate takes control of these if
>     matching
>     >     > between inmate and root
>     >
>     >     Exclusive control.
>     >
>     >     >      D. ".pci_devices" -> Inmate takes control?
>     >
>     >     Exclusive control.
>     >
>     >     >      E. ".pci_caps" -> Inmate takes control?
>     >
>     >     Exclusive control.
>     >
>     >     >
>     >     > 2. Along the same lines, going back to my ttyS0 issue
>     mentioned above
>     >     > while trying to share it:
>     >     >
>     >     >        If I allocate the following to the linux non root
>     cell (and set
>     >     > to -1 in the root)
>     >
>     >     BTW, there's generally no need to set -1 in the root cell --
>     it will be
>     >     taken away when the non-root cell is created.
>     >
>     >     >      .pio_bitmap = {
>     >     >           [0x3f8/8 ... 0x3ff/8] = 0x00   /* serial 2*/
>     >     >        }
>     >     >
>     >     >        Then on the root cell I randomly get a PIO read fault on
>     >     "0x3FE"
>     >     > when the non-root is booting, i'm not sure what process is
>     causing
>     >     this.
>     >
>     >     - Are you sure you remove the device from the kernel cmdline?
>     >     - Are you sure that there doesn't spawn a tty? Check with
>     systemctl.
>     >     - lsof | grep ttyS
>     >     - The root-cell might receive the serial device's interrupt,
>     while its
>     >       ports are assigned to the non-root cell. Does the crash
>     happen when
>     >       you type in a character to the serial line?
>     >
>     >       In this case, assign the interrupt to the non-root cell. You
>     will need
>     >       it in any case there once you have a initrd and want to type
>     in stuff.
>     >
>     >     So there must be some reason why the crash happens. The hard
>     way is to
>     >     add 8250.nr_uarts=0 in your kernel cmdline of the root cell.
>     >
>     >     >
>     >     >        if I leave out the pio mapping in the non-root linux, and
>     >     > alolothat block in the root cell instead then the non-root linux
>     >     gets a
>     >     > PIO read fault on "0x3F9" while booting.       
>     >     >        Ralf mentioned that Linux would not try enumerate the
>     device if
>     >     > its set to -1 ( [0x3f8/8 ... 0x3ff/8] = -1)  , but the non-root
>     >     seems to
>     >     > attempt to do something with it anyway:
>     >     >
>     >     > "[    0.956146]Serial: 8250/16550 driver, 1 ports, IRQ sharing
>     >     enabled"
>     >     > "FATAL: Invalid PIO read, port: 3f9: size 1" 
>     >
>     >     Hmm. At least that's what I thought. I don't know how
>     enumeration of
>     >     platform devices works exactly, but reconsidering this, the
>     behaviour
>     >     above makes sense: How would it know that a platform device is not
>     >     present, depending of the configuration of the pio_bitmap.
>     >
>     >     Anyway, 8250 drivers have some cmdline parameters that can be
>     used to
>     >     control initialisation (e.g., see the nr_uarts parameters above).
>     >
>     >     >
>     >     >        Is it possible to map the same pio block in both the
>     root and
>     >     > non-root with a different mask?  Or does the non-root just
>     >     override it? 
>     >     > It seems like the latter is true.   
>     >
>     >     Exactly. That's not a) possible, and b) does not really make
>     sense.
>     >
>     >       Ralf
>     >
>     >     >
>     >     > Thanks again Jan and Ralf for your help getting to this point.
>     >     >
>     >     >
>     >     > On Tue, Jun 11, 2019 at 2:13 PM Wayne
>     <[email protected] <mailto:[email protected]>
>     >     <mailto:[email protected] <mailto:[email protected]>>
>     >     > <mailto:[email protected]
>     <mailto:[email protected]> <mailto:[email protected]
>     <mailto:[email protected]>>>>
>     >     wrote:
>     >     >
>     >     >     Jan & Ralf:
>     >     >
>     >     >     I have a little good news, I have successfully gotten
>     the non-root
>     >     >     linux to display some startup output based on your
>     suggestions. 
>     >     >     Also, there must have been some small difference between
>     the stock
>     >     >     linux-x86-demo cell config and the one I tweaked.  I
>     went back to
>     >     >     the original and made some minor changes and it started
>     displaying
>     >     >     the serial output now for the non-root node.
>     >     >
>     >     >     I ended up making the root cell the primary user of the
>     ttyS0
>     >     >     device, and just having the non-root linux output to the
>     >     hypervisor
>     >     >     virtual console (console=jailhouse).  In my setup, there
>     >     appears to
>     >     >     be some process still attempting to use the ttyS0
>     randomly on the
>     >     >     root linux, and that causes the hypervisor to halt it if
>     I don't
>     >     >     leave it in the PIO array.  Not sure who is doing it.
>     >     >
>     >     >
>     >     >     This is where i'm at now.  The non-root linux keeps
>     running into
>     >     >     points where it tries to access PIO and gets parked:
>     >     >
>     >     >     1. First crash:
>     >     >
>     >     >     [    0.424925]kworker/u6:0 (27) used greated stack
>     depth: 14656
>     >     >     bytes left
>     >     >     [    0.425807]futex has table entries: 1024 (order: 4, 65536
>     >     bytes)
>     >     >     "FATAL: Invalid PIO write, port: 70: size 1"
>     >     >     ...
>     >     >     Parking CPU 3 (Cell: "linux-x86-demo")
>     >     >
>     >     >     According to /proc/ioports, 70 has to do with the
>     Realtime clock
>     >     >     (rtc0).  I was able to temporarily add the 70 mapping to the
>     >     >     non-root linux, but should it be there at all?  I tried
>     to disable
>     >     >     RTC support in the guest kernel config.
>     >     >
>     >     >
>     >     >     2. On the next attempt, I then got further with a crash
>     trying to
>     >     >     initialize Ser device ttyS0:
>     >     >
>     >     >     "[    0.956146]Serial: 8250/16550 driver, 1 ports, IRQ
>     sharing
>     >     enabled"
>     >     >     "FATAL: Invalid PIO read, port: 3f9: size 1"
>     >     >     ...
>     >     >     Parking CPU 3 (Cell: "linux-x86-demo")
>     >     >
>     >     >     I can get around that if I allow the non-root access in
>     its pio
>     >     >     table, but then I have a problem above with the root linux
>     >     randomly
>     >     >     trying to use it.
>     >     >
>     >     >
>     >     >     3. Now with the temporary adjustments to the non-root
>     pio table
>     >     >     above I get here:
>     >     >
>     >     >     "[    0.972399]clocksource:Switched to clocksource tsc"
>     >     >     "FATAL: Invalid PIO read, port: 87: size 1"
>     >     >     ...
>     >     >     Parking CPU 3 (Cell: "linux-x86-demo")
>     >     >
>     >     >     According to proc/ioports this one has to do with "dma page
>     >     >     request". It isn't mapped in my root linux PIO or non-root
>     >     linux array.
>     >     >
>     >     >
>     >     >     Aside from the serial conflict, it seems like these
>     should remain
>     >     >     controlled by the root linux or hypervisor.   Do you use a
>     >     >     particular bare minimum x86 kernel config on your
>     machine for the
>     >     >     guest to get around these?  I know Jan mentioned the
>     >     >     jailhouse-images project might have something, but I
>     couldn't find
>     >     >     it digging around quick.
>     >     >
>     >     >     Thanks to you both for your responses and patience.  I
>     got the
>     >     root
>     >     >     linux up and going on my own, but the non-root guest is
>     proving to
>     >     >     be more difficult.
>     >     >
>     >     >     Wayne
>     >     >
>     >     >
>     >     >
>     >     >     On Tue, Jun 11, 2019 at 11:31 AM Ralf Ramsauer
>     >     >     <[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >     <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>> wrote:
>     >     >
>     >     >         Hi,
>     >     >
>     >     >         On 6/10/19 7:45 PM, Wayne wrote:
>     >     >         > Thank you for your responses Ralf and Jan.
>     >     >         >
>     >     >         > Yes, I have successfully gotten both the apic-demo and
>     >     >         tiny-demo cells
>     >     >         > to work on my system before attempting to load the
>     non-root
>     >     >         linux.  I
>     >     >         > can see serial output from them.
>     >     >         >
>     >     >         > However, if I try to share the UART between root
>     and inmate
>     >     >         for them I
>     >     >         > run into a crash on the hypervisor if the root linux
>     >     attempts
>     >     >         to write
>     >     >         > to the UART (after creating/starting a demo). 
>     Does the root
>     >     >         linux lose
>     >     >         > access once an inmate is created/started?  Or am I
>     missing
>     >     >         something
>     >     >         > from my System config or tiny-demo config that allows
>     >     them to
>     >     >         share?  I
>     >     >         > attached the crash in hypervisor_output.txt.
>     >     >         >
>     >     >         > Also, I noticed the tiny-demo output on the UART gets
>     >     mirrored
>     >     >         on the
>     >     >
>     >     >         Yes, according to your config,
>     >     >         JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE is
>     >     >         set. This means that the guest will use the virtual
>     >     console. The
>     >     >         virtual
>     >     >         console uses the hypervisor output which depends
>     whatever
>     >     is set
>     >     >         in your
>     >     >         system configuration. In your case the hypervisor
>     uses 3f8.
>     >     >
>     >     >         And additionally, the inmate also a valid .console
>     set. Again,
>     >     >         3f8. This
>     >     >         means, it will use the console *and* the hypervisor
>     debug
>     >     >         hypercall. In
>     >     >         your case, both outputs are routed to the same
>     device which
>     >     >         explains the
>     >     >         mirroring.
>     >     >
>     >     >         > Jailhouse virtual console.  Does that mean that its
>     >     writing to the
>     >     >         > serial through the hypervisor and not directly
>     accessing
>     >     ttyS0
>     >     >         itself?
>     >     >
>     >     >         Exactly.
>     >     >
>     >     >         > If so, is there a way to create the cell to write
>     ttyS0
>     >     >         directly as a
>     >     >         > test?  To see if it exhibits same behavior as the
>     >     non-root linux.
>     >     >
>     >     >         Sure, simply align the .address field of .console,
>     and allow
>     >     >         access to
>     >     >         that port via .pio_bitmap.
>     >     >
>     >     >         >
>     >     >         > FYI, I create the tiny-demo like this:
>     >     >         >
>     >     >         > jailhouse cell create configs/x86/tiny-demo.cell
>     >     >         > jailhouse cell load tiny-demo
>     >     inmates/demos/x86/tiny-demo.cell  
>     >     >         > jailhouse cell start tiny-demo
>     >     >         >
>     >     >         > Going back to my original non-root linux question, I
>     >     don't see any
>     >     >         > output when connected over ttyS0.  Its address is
>     0x3f8
>     >     >         ("serial 1") in
>     >     >         > the PIO, and that seems to work fine for the other
>     simple
>     >     >         demos, minus
>     >     >         > the hypervisor crash issue above when sharing.  I
>     know there
>     >     >         is the
>     >     >         > other "serial 2" port but I haven't tried to use
>     that one.
>     >     >
>     >     >         I see that port on our Dell server as well. It's
>     present,
>     >     but I
>     >     >         don't
>     >     >         know where it's connected to.
>     >     >
>     >     >         Let me just summarise your issue:
>     >     >
>     >     >         You only have one serial line available, right? From the
>     >     root-cell's
>     >     >         POV, it's ttyS0 on 3f8.
>     >     >
>     >     >         I would propose to share ttyS0 between the
>     hypervisor and
>     >     non-root
>     >     >         Linux. This means:
>     >     >
>     >     >           - Don't let the root-cell use ttyS0. Remove any
>     >     console=ttyS0
>     >     >             parameters from you commandline. Ensure that noone
>     >     else accesses
>     >     >             ttyS0 (e.g., getty@ttyS0 or other friends)
>     >     >
>     >     >           - Set 3f8 as debug_console in your master.c (seems you
>     >     already
>     >     >         have)
>     >     >
>     >     >           - Allow 3f8 access in the linux-demo.c (already set in
>     >     the default
>     >     >             linux-demo
>     >     >
>     >     >           - Disallow 2f8 access in linux-demo.c:
>     >     >         -               [ 0x2f8/8 ...  0x2ff/8] = 0, /*
>     serial2 */
>     >     >         +               [ 0x2f8/8 ...  0x2ff/8] = -1, /*
>     serial2 */
>     >     >
>     >     >             I don't know how Linux enumerates serial
>     devices, but this
>     >     >         ensures
>     >     >             that Linux won't see the unconnected serial line
>     and map
>     >     >         ttyS0 to
>     >     >             3f8
>     >     >
>     >     >         >
>     >     >         > non root linux launched with:
>     >     >         > /tools/jailhouse cell linux configs/x86/linux-x86.cell
>     >     >         > /boot/vmlinux-4.1.16-Guest -c "console=ttyS0,115200"
>     >     >
>     >     >         Then, this should actually work...
>     >     >
>     >     >         And if not, then console=jailhouse0 should work.
>     >     >
>     >     >         >
>     >     >         > I can try the 4.19 siemens kernel and "next"
>     branch for
>     >     >         jailhouse you
>     >     >         > suggested.  Do you think that combination will resolve
>     >     the missing
>     >     >         > serial for the non-root linux?  Otherwise, did my
>     kernel
>     >     >         config, system
>     >     >         > config, and non-root linux cell configs look OK?
>     >     >
>     >     >         I didn't look at it, but we should at least - even
>     if some
>     >     flags are
>     >     >         still incorrect - see the "Uncompressing kernel..."
>     thing.
>     >     >
>     >     >         Wait -- one thing you could try: Deactivate
>     CONFIG_EFI and
>     >     >         especially
>     >     >         CONFIG_EFI_STUBS. I'm not sure, but it could be that
>     EFI_STUBS
>     >     >         change
>     >     >         the header format that our bootloader patches.
>     >     >
>     >     >         >
>     >     >         > Thanks again for your help.
>     >     >
>     >     >         No problem. But it's not yet working. ;-)
>     >     >
>     >     >           Ralf
>     >     >
>     >     >         >
>     >     >         >
>     >     >         > On Fri, Jun 7, 2019 at 5:30 PM Ralf Ramsauer
>     >     >         > <[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >         <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>
>     >     >         > <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >         <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>>> wrote:
>     >     >         >
>     >     >         >     Hi Wayne,
>     >     >         >
>     >     >         >     On 6/7/19 10:04 PM, Wayne wrote:
>     >     >         >     > Hi Ralf,
>     >     >         >     >
>     >     >         >     > Thank you for your responses.  I have
>     attached the
>     >     >         requested config
>     >     >         >     > files.  I am now just trying to use the same
>     >     bzImage for
>     >     >         the root and
>     >     >         >     > the non-root linux node.
>     >     >         >
>     >     >         >     One question that you didn't answer: do
>     apic-demo or
>     >     >         tiny-demo work?
>     >     >         >     They need to to have correct .console
>     parameter set, so
>     >     >         ensure that
>     >     >         >     parameters are set if you run those demos in your
>     >     >         linux-demo cell. Just
>     >     >         >     compare it to tiny-demo.c or apic-demo.c
>     >     >         >
>     >     >         >     >
>     >     >         >     > Yes, I am using vanilla 4.16 because the
>     documentation
>     >     >         indicated
>     >     >         >     that it
>     >     >         >     > is the first mainline release to officially
>     support
>     >     >         Jailhouse Guest
>     >     >         >     > images.  Do you suggest that I use something
>     else?
>     >     >         >
>     >     >         >     As Jan already wrote: Try to use Siemens' 4.19
>     branch.
>     >     >         >
>     >     >         >     >
>     >     >         >     > Right now i'm just trying to get some serial
>     >     output from
>     >     >         the non-root
>     >     >         >     > Linux.  Hopefully, I have that configured
>     correctly. 
>     >     >         Ideally, I would
>     >     >         >     > like to share the same serial console for
>     the Root
>     >     node
>     >     >         and non-root
>     >     >         >     > node eventually if possible.  I have been OK
>     with
>     >     using
>     >     >         the virtual
>     >     >         >     > console (jailhouse console -f) for the
>     Hypervisor
>     >     output.
>     >     >         >
>     >     >         >     In addition to Jan's comment:
>     >     >         >
>     >     >         >     Currently you only have access to the
>     jailhouse console
>     >     >         via 'jailhouse
>     >     >         >     console' from your root cell. Imagine your
>     root cell
>     >     >         crashes for some
>     >     >         >     reason. You will never see the fault reason,
>     which makes
>     >     >         things hard to
>     >     >         >     debug.
>     >     >         >
>     >     >         >     I would suggest to configure the hypervisor to
>     use the
>     >     >         serial console.
>     >     >         >     You can share the device with the non-root
>     Linux, that's
>     >     >         no problem.
>     >     >         >
>     >     >         >     In this case, the non-root kernel's output will
>     >     always be
>     >     >         printed to the
>     >     >         >     serial console. Directly, if you choose
>     >     console=ttySx, and
>     >     >         indirectly
>     >     >         >     via the hypervisor if you choose
>     console=jailhouse.
>     >     >         >
>     >     >         >     BTW: According to your config, your system is a
>     >     PowerEdge,
>     >     >         and the
>     >     >         >     non-root cell gets both, 0x2f8 and 0x3f8. Are
>     you sure
>     >     >         that ttyS1 is the
>     >     >         >     correct console that you are connected to?
>     >     >         >
>     >     >         >     Just mentioning this as we have a similar
>     machine here,
>     >     >         and, afair, both
>     >     >         >     platform serial device are 'usable', but one
>     ends in the
>     >     >         nirvana while
>     >     >         >     it is accessible.
>     >     >         >
>     >     >         >     Ah, and one last thing: try to switch to the
>     current
>     >     next
>     >     >         branch for
>     >     >         >     jailhouse. Jan just integrated a few patches
>     from me
>     >     that
>     >     >         might also be
>     >     >         >     relevant for your machine.
>     >     >         >
>     >     >         >     HTH
>     >     >         >       Ralf
>     >     >         >
>     >     >         >     >
>     >     >         >     > Wayne
>     >     >         >     >
>     >     >         >     > On Fri, Jun 7, 2019 at 9:10 AM Ralf Ramsauer
>     >     >         >     > <[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >         <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>
>     >     >         >     <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >         <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>>
>     >     >         >     > <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >         <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>
>     >     >         >     <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>
>     >     >         <mailto:[email protected]
>     <mailto:[email protected]>
>     >     <mailto:[email protected]
>     <mailto:[email protected]>>>>>> wrote:
>     >     >         >     >
>     >     >         >     >     Hi,
>     >     >         >     >
>     >     >         >     >     On 6/7/19 2:28 PM, Wayne wrote:
>     >     >         >     >     > Hello,
>     >     >         >     >     >
>     >     >         >     >     > I am new to Linux development and
>     >     Jailhouse.  I have
>     >     >         >     successfully
>     >     >         >     >     booted
>     >     >         >     >     > the Jailhouse Hypervisor and root cell
>     on a bare
>     >     >         metal X86 Linux
>     >     >         >     >     system
>     >     >         >     >     > (No QEMU).  I am now trying to load a
>     non-root
>     >     >         Linux cell
>     >     >         >     and I have a
>     >     >         >     >     > few questions.  Jailhouse accepts and
>     starts my
>     >     >         non-root
>     >     >         >     linux cell
>     >     >         >     >     > configuration and I see it as "running"
>     >     through the
>     >     >         >     "jailhouse cell
>     >     >         >     >     > list" command.  However, I don't see
>     any serial
>     >     >         output from the
>     >     >         >     >     > "non-root linux" cell booting up.  I’m
>     not sure
>     >     >         what the
>     >     >         >     non-root node
>     >     >         >     >     > is doing at this point.
>     >     >         >     >
>     >     >         >     >     Ok, first of all, can you see any output
>     from
>     >     >         Jailhouse's demo
>     >     >         >     inmates
>     >     >         >     >     (e.g., tiny-demo or apic-demo)?
>     >     >         >     >
>     >     >         >     >     >
>     >     >         >     >     >                My root node is a 4.16
>     kernel
>     >     >         configured this
>     >     >         >     way:
>     >     >         >     >     >                               1.
>     >     >         CONFIG_JAILHOUSE_GUEST is
>     >     >         >     not set
>     >     >         >     >     >                               2.
>     CONFIG_SERIO=y
>     >     >         >     >     >                               3.
>     >     >         >     CONFIG_SERIAL_8250_RUNTIME_UARTS=4
>     >     >         >     >     >
>     >     >         >     >     >                My non-root node is a
>     4.16 kernel
>     >     >         configured
>     >     >         >     this way:
>     >     >         >     >     >                               1.
>     >     >         CONFIG_JAILHOUSE_GUEST=y
>     >     >         >     >     >                               2.
>     CONFIG_SERIO=m
>     >     >         (can't seem
>     >     >         >     to disable
>     >     >         >     >     > completely in my config for 4.16)
>     >     >         >     >     >                               3.
>     >     >         >     CONFIG_SERIAL_8250_RUNTIME_UARTS=1
>     >     >         >     >
>     >     >         >     >     Could you please provide a full .config?
>     What
>     >     branch
>     >     >         are you
>     >     >         >     exactly
>     >     >         >     >     using? Vanilla 4.16?
>     >     >         >     >
>     >     >         >     >     Please attach your system config and your
>     >     non-root linux
>     >     >         >     config as well.
>     >     >         >     >
>     >     >         >     >     You may also want to try
>     >     >         https://github.com/siemens/linux . See
>     >     >         >     >     jailhouse-enabling branches for some
>     releases.
>     >     >         >     >
>     >     >         >     >     >
>     >     >         >     >     > In general, do the kernel config settings
>     >     have to
>     >     >         match between
>     >     >         >     >     the root
>     >     >         >     >     > node and non-linux or is the above fine?
>     >     >         >     >
>     >     >         >     >     No, they do not have to be the same, but
>     they can.
>     >     >         Still, an
>     >     >         >     analysis
>     >     >         >     >     requires your .config.
>     >     >         >     >
>     >     >         >     >     >
>     >     >         >     >     > The vmlinux-4.1.16-Guest bzImage is
>     approx 7MB,
>     >     >         and the
>     >     >         >     inmate node is
>     >     >         >     >     > allocated ~75MB of RAM.
>     >     >         >     >     >
>     >     >         >     >     > I have a single UART, so I have
>     configured the
>     >     >         root cell
>     >     >         >     system config
>     >     >         >     >     > to output to the virtual hypervisor
>     console:
>     >     >         >     >
>     >     >         >     >     Wait... You configured your root-cell to
>     use the
>     >     >         hypervisor debug
>     >     >         >     >     console? How? It's only available once the
>     >     hypervisor is
>     >     >         >     enabled. How
>     >     >         >     >     should this work?
>     >     >         >     >
>     >     >         >     >     But let's have a look at your
>     configuration first.
>     >     >         >     >
>     >     >         >     >     So you want to:
>     >     >         >     >       - Have the UART (0x3f8) available in the
>     >     non-root cell
>     >     >��        >     >       - Use 0x3f8 as hypervisor debug
>     console as well
>     >     >         >     >       - No console for root-cell
>     >     >         >     >
>     >     >         >     >     Did I get this right?
>     >     >         >     >
>     >     >         >     >       Ralf
>     >     >         >     >
>     >     >         >     >     >
>     >     >         >     >     > .flags =
>     JAILHOUSE_SYS_VIRTUAL_DEBUG_CONSOLE,
>     >     >         >     >     >
>     >     >         >     >     > .debug_console = {
>     >     >         >     >     > .type = JAILHOUSE_CON_TYPE_NONE,
>     >     >         >     >     > },
>     >     >         >     >     >
>     >     >         >     >     > and I have configured the non-root
>     linux cell to
>     >     >         output to
>     >     >         >     the UART:
>     >     >         >     >     >
>     >     >         >     >     > (Added serial 0x3f8 to pio bitmap for
>     non-root
>     >     >         linux) and
>     >     >         >     started the
>     >     >         >     >     > node with this:
>     >     >         >     >     >
>     >     >         >     >     > ./tools/jailhouse cell linux
>     >     >         configs/x86/linux-x86.cell
>     >     >         >     >     > /boot/vmlinux-4.1.16-Guest -c
>     >     "console=ttyS0,115200"
>     >     >         >     >     > (Note I also tried "console=jailhouse"
>     in the
>     >     >         command above,
>     >     >         >     and that
>     >     >         >     >     > produces the same result)
>     >     >         >     >     >
>     >     >         >     >     > I then see the following on my hypervisor
>     >     console
>     >     >         >     (./tools/jailhouse
>     >     >         >     >     > console -f):
>     >     >         >     >     >
>     >     >         >     >     > Created cell "linux-x86-demo"
>     >     >         >     >     > ...
>     >     >         >     >     > Cell "linux-x86-demo" can be loaded
>     >     >         >     >     > Started cell "linux-x86-demo"
>     >     >         >     >     >
>     >     >         >     >     > After a little while I do get a parked
>     CPU error
>     >     >         on the root
>     >     >         >     node,
>     >     >         >     >     looks
>     >     >         >     >     > like its trying to do something with the
>     >     UART as well:
>     >     >         >     >     > FATAL: Invalid PIO read, port: 3fe size: 1
>     >     >         >     >     >
>     >     >         >     >     > I would expect something to pop out on
>     the UART
>     >     >         from the
>     >     >         >     non-root
>     >     >         >     >     linux
>     >     >         >     >     > node first.  Note that root node has
>     serial
>     >     0x3f8
>     >     >         disabled
>     >     >         >     in its pio
>     >     >         >     >     > bitmap.
>     >     >         >     >     >
>     >     >         >     >     > I verifed that the UART is functioning by
>     >     allowing the
>     >     >         >     hypervisor to
>     >     >         >     >     > print to it and also performed an echo
>     test over
>     >     >         ttyS0.
>     >     >         >     >     >
>     >     >         >     >     > I have tried several configurations of
>     >     >         kernel.....including your
>     >     >         >     >     current
>     >     >         >     >     > "queues/jailhouse" branch head kernel
>     for the
>     >     >         non-root node,
>     >     >         >     along
>     >     >         >     >     with
>     >     >         >     >     > the kernel config for 4.7 posted in
>     this thread
>     >     >         below (but I
>     >     >         >     get same
>     >     >         >     >     > result as above when I start it, no kernel
>     >     output):
>     >     >         >     >     >              
>     >     >         >     >     >
>     >     >         >     >   
>     >     >         >   
>     >     >       
>     >   
>          
> "https://groups.google.com/forum/#!searchin/jailhouse-dev/Re$3A$20Failed$20to$20boot$20jailhouse%7Csort:relevance/jailhouse-dev/M7UO89XFIk0/Qi40DDuMBAAJ";.
>     >     >         >     >     >
>     >     >         >     >     > Any information you can provide to me
>     will be
>     >     >         helpful.  I'm
>     >     >         >     not sure
>     >     >         >     >     > what might be going wrong here.
>     >     >         >     >     >
>     >     >         >     >     > Thanks,
>     >     >         >     >     > Wayne
>     >     >         >     >     >
>     >     >         >     >     > --
>     >     >         >     >     > You received this message because you are
>     >     >         subscribed to the
>     >     >         >     Google
>     >     >         >     >     > Groups "Jailhouse" group.
>     >     >         >     >     > To unsubscribe from this group and stop
>     >     receiving
>     >     >         emails
>     >     >         >     from it, send
>     >     >         >     >     > an email to
>     >     >         [email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >         <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>
>     >     >         >   
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >       
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>>
>     >     >         >     >   
>     >      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >       
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>
>     >     >         >   
>     >      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >       
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>>>
>     >     >         >     >     >
>     >     <mailto:[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >         <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>
>     >     >         >   
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >       
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>>
>     >     >         >     >   
>     >      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >       
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>
>     >     >         >   
>     >      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     >       
>      <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>>>>.
>     >     >         >     >     > To view this discussion on the web visit
>     >     >         >     >     >
>     >     >         >     >   
>     >     >         >   
>     >     >       
>     >   
>         
> https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc2iKk1J6%2B0huh5__dS4HyujXzV9r%2BLbKLzuVZ4K3Bt5eA%40mail.gmail.com
>     >     >         >     >     >
>     >     >         >     >   
>     >     >         >   
>     >     >       
>     >   
>         
> <https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc2iKk1J6%2B0huh5__dS4HyujXzV9r%2BLbKLzuVZ4K3Bt5eA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>     >     >         >     >     > For more options, visit
>     >     >         https://groups.google.com/d/optout.
>     >     >         >     >
>     >     >         >
>     >     >
>     >     > --
>     >     > You received this message because you are subscribed to the
>     Google
>     >     > Groups "Jailhouse" group.
>     >     > To unsubscribe from this group and stop receiving emails
>     from it, send
>     >     > an email to [email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>
>     >     > <mailto:[email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     >     <mailto:jailhouse-dev%[email protected]
>     <mailto:jailhouse-dev%[email protected]>>>.
>     >     > To view this discussion on the web visit
>     >     >
>     >   
>      
> https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc0O2zSeuLLY3MaeRW7cQrWbq-6Y2BHJg%2Bx_j6nk%3DECa_A%40mail.gmail.com
>     >     >
>     >   
>      
> <https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc0O2zSeuLLY3MaeRW7cQrWbq-6Y2BHJg%2Bx_j6nk%3DECa_A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>     >     > For more options, visit https://groups.google.com/d/optout.
>     >
>     > --
>     > You received this message because you are subscribed to the Google
>     > Groups "Jailhouse" group.
>     > To unsubscribe from this group and stop receiving emails from it, send
>     > an email to [email protected]
>     <mailto:jailhouse-dev%[email protected]>
>     > <mailto:[email protected]
>     <mailto:jailhouse-dev%[email protected]>>.
>     > To view this discussion on the web visit
>     >
>     
> https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc36sP7ABEsR5Bp%2Bgpts0DUBENqV6eFDPazfs5kR_QRGaw%40mail.gmail.com
>     >
>     
> <https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc36sP7ABEsR5Bp%2Bgpts0DUBENqV6eFDPazfs5kR_QRGaw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>     > For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Jailhouse" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected]
> <mailto:[email protected]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc13nuahbog_WVD-BJi0asHEa8TxMAvB5M5damJGJ_5inA%40mail.gmail.com
> <https://groups.google.com/d/msgid/jailhouse-dev/CA%2B%2BKhc13nuahbog_WVD-BJi0asHEa8TxMAvB5M5damJGJ_5inA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/d2e2b271-4e43-4126-5433-8304d4984e7a%40oth-regensburg.de.
For more options, visit https://groups.google.com/d/optout.

Reply via email to