On Wed, Jun 17, 2020 at 12:14 PM Bertrand Marquis <[email protected]> wrote: > > > > > On 17 Jun 2020, at 13:48, Bruce Ashfield via lists.yoctoproject.org > > <[email protected]> wrote: > > > > On Wed, Jun 17, 2020 at 8:45 AM Bruce Ashfield via > > lists.yoctoproject.org > > <[email protected]> wrote: > >> > >> On Wed, Jun 17, 2020 at 4:29 AM Bertrand Marquis > >> <[email protected]> wrote: > >>> > >>> Hi, > >>> > >>>> On 16 Jun 2020, at 22:43, Christopher Clark via lists.yoctoproject.org > >>>> <[email protected]> wrote: > >>>> > >>>> On Mon, Jun 15, 2020 at 5:45 AM Bruce Ashfield > >>>> <[email protected]> wrote: > >>>>> On Mon, Jun 15, 2020 at 5:11 AM Christopher Clark > >>>>> <[email protected]> wrote: > >>>>>> > >>>>>> On Fri, Jun 5, 2020 at 5:58 PM Corey Minyard <[email protected]> > >>>>>> wrote: > >>>>>>> > >>>>>>> On Fri, Jun 05, 2020 at 07:53:53PM -0400, Bruce Ashfield wrote: > >>>>>>>> On Fri, Jun 5, 2020 at 5:16 PM Corey Minyard <[email protected]> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> On Fri, Jun 05, 2020 at 07:55:37PM +0000, Stewart Hildebrand wrote: > >>>>>>>>>> + Corey > >>>>>>>>>> > >>>>>>>>>> On Friday, June 5, 2020 3:19 PM, Bruce Ashfield wrote: > >>>>>>>>>>> On Fri, Jun 5, 2020 at 3:12 PM Stewart Hildebrand wrote: > >>>> > >>>>>>>>>> Corey: you are hereby encouraged to submit patches to > >>>>>>>>>> meta-virtualization. > >>>>>>>>> > >>>>>>>>> Ok. The layer has the following basic pieces: > >>>>>> > >>>>>> Before we get to the other pieces, I'd like to cover the new "rpixen" > >>>>>> MACHINE type that the layer introduces. > >>>>>> > >>>>>> My preference is for avoiding introduction of another MACHINE to > >>>>>> reconfigure an existing one to run Xen, if possible, and use the > >>>>>> existing "raspberrypi4-64". I'm hoping to avoid the pattern of > >>>>>> creating a new machine for Xen for each board that we add support for. > >>>>>> In meta-virtualization, there's the "xen" DISTRO_FEATURE, which is > >>>>>> used to turn on Xen-specific functionality and compatibility, and I'd > >>>>>> like to explore whether that can be made to be sufficient to enable > >>>>>> what is needed. > >>>>>> > >>>>>> To that end, I've done an initial pass to see what it might take and > >>>>>> the work-in-progress from that is posted here: > >>>>>> https://github.com/dozylynx/meta-virtualization/tree/raspberry-pi4-initial-xen > >>>>>> > >>>>>> Some minor changes to other layers could assist - eg. to remove the > >>>>>> need for a guest filesystem to contain the hypervisor binary - and > >>>>>> there's still some tidying to do. > >>>>>> > >>>>>>>>> 1 The xen patches for the Pi4, just a few patches. As the Xen group > >>>>>>>>> fixes things, I keep adding :). > >>>>>> > >>>>>> Unfortunately I had to drop these patches from my local test to get it > >>>>>> to boot. It could easily be a local build issue or a fault with the > >>>>>> one test I've run so far, but I did have success booting with just Xen > >>>>>> 4.13 and I'd like to get a bit more understanding and confidence in > >>>>>> them before we bring them in. > >>>>>> > >>>>>>>>> 2 Hacks for getting the Pi4 kernel config right for xen. This > >>>>>>>>> should go > >>>>>>>>> away if you don't use the kernel from the Pi4 yocto layer, as it > >>>>>>>>> doesn't work like most kernels in yocto. > >>>>>>>> > >>>>>>>> I should take a look at the configs and see if I can create a > >>>>>>>> fragment > >>>>>>>> or two, but I can take care of that. > >>>>>>> > >>>>>>> That shouldn't be necessary. The standard fragments work, it's just > >>>>>>> that the Pi kernel does not use them. So this is really Pi+Xen > >>>>>>> specific, and hopefully they can fix the Pi kernel to use the normal > >>>>>>> fragments in the future. > >>>>>> > >>>>>> The linux-raspberrypi kernel does use linux-yocto; it's just that > >>>>>> meta-virtualization > >>>>>> needs a matching .inc file to be present for the kernel version that > >>>>>> you're using. > >>>>>> Assuming you're using Linux 4.19 (which is what I've tested with) add > >>>>>> this file: > >>>>>> meta-virtualization/recipes-kernel/linux/linux-yocto_4.19_virtualization.inc > >>>>>> containing just: > >>>>>> include linux-yocto_virtualization.inc > >>>>>> which will then enable the linux-raspberrypi kernel to add the > >>>>>> meta-virt Xen fragment. > >>>>>> > >>>>>> There are also two Linux patches: > >>>>>> 1) Disable DMA in the SDHCI driver > >>>>>> This one needs more information in the commit text to understand what > >>>>>> is motivating doing this and what the effects of it are. Should it go > >>>>>> into the standard Raspberry Pi kernel? > >>>>>> 2) Fix PCIe in dom0 for RPi4 > >>>>>> Is this fixed upstream in more recent kernels? It would be good to > >>>>>> have a pointer to that if so. > >>>>>> > >>>>>> Bruce: To apply these to just the Raspberry Pi kernel when it's being > >>>>>> used with Xen, a kernel bbappend in a raspberrypi dynamic-layers might > >>>>>> be an option to consider - eg: > >>>>>> https://github.com/dozylynx/meta-virtualization/tree/raspberry-pi4-initial-xen/dynamic-layers/raspberrypi/recipes-kernel/linux > >>>>> > >>>>> I can most likely live with that. I obviously make sure that the > >>>>> reference linux-yocto kernel doesn't need anything to work out of the > >>>>> box, but we can't (and shouldn't) enforce that choice on everyone. I'd > >>>>> rather have patches centralized in a topic layer like > >>>>> meta-virtualization, so if we need to add a dynamic layer and a few > >>>>> patches, that's a good place to be. > >>>> > >>>> please see my new comment below. > >>>> > >>>>>>>>> 5 A few Pi-specific hacks for config and u-boot. > >>>>>>>> > >>>>>>>> #5 does sound like BSP stuff. Is any of it destined for the rpi > >>>>>>>> layers > >>>>>>>> ? Or is it both rpi AND xen specific, so doesn't really make sense > >>>>>>>> there either ? > >>>>>>> > >>>>>>> It's both Pi and Xen specific. If everything gets put where it should > >>>>>>> be, that would be the only thing left in this layer :). > >>>>>> > >>>>>> These are in recipes-bsp and I agree that they're both Pi and Xen > >>>>>> specific. I think these are small enough pieces that keeping them in > >>>>>> meta-virtualization could be a reasonable call, since that's the layer > >>>>>> where Xen support is focussed, and so they can be added to: > >>>>>> dynamic-layers/raspberrypi/recipes-bsp > >>>>>> which indicates their status as amendments to the meta-raspberrypi > >>>>>> layer. Changes to them would then be easily coordinated with the Xen > >>>>>> recipes. > >>>>> > >>>>> Agreed. > >>>> > >>>> Since I wrote this, we've seen some expressed interest in support for > >>>> running Xen on the NVIDIA Jetson Nano and Xavier NX boards, and on the > >>>> xen-devel mailing list, a report of success running Xen on the RockPro64 > >>>> board: > >>>> https://lists.xenproject.org/archives/html/xen-devel/2020-06/msg01067.html > >>>> > >>>> I've personally built Xen with meta-virtualization to run on the > >>>> Cubietruck for > >>>> development and testing; the Odroid C2 and XU4 are of interest, as is the > >>>> Xilinx Ultra-96-V2 board; and the NVidia Jetson TX1 at one point had some > >>>> non-upstream patches available to enable Xen on it. PCEngines maintains > >>>> Xen compatibility for their APU2 in meta-pcengines. > >>>> > >>>> There are also some challenges being encountered in getting a more > >>>> recent Linux > >>>> kernel working as a dom0 on the Raspberry Pi 4 - eg. the Linux > >>>> Foundation Eve > >>>> Project run their own patches for 5.6 here: > >>>> https://github.com/lf-edge/eve/tree/master/pkg/new-kernel/patches-5.6.x > >>>> > >>>> All of this points towards it being a reasonable proposition to have a > >>>> dedicated Xen hardware support Yocto layer, so that board-specific > >>>> tweaks for > >>>> hardware compatibility with Xen can be maintained without complicating > >>>> meta-virtualization, while continuing to pool Xen-aware contributions in > >>>> a centralized layer. > >>>> > >>>> I'd like to propose creating: 'meta-xen-bsp'; and I'm willing to work on > >>>> maintaining it. Feedback to this suggestion is welcome. > >>>> > >>>> Bruce: how does this sound to you? > >>> > >>> I had this problem working on some other BSPs to support Xen with > >>> meta-virtualization (you can check this in meta-arm[1]). > >>> > >>> I would think the best way to handle this case is to use dynamic layers > >>> and push the required support inside the layer containing the BSP. > >>> This would make the maintenance easier (for example when the kernel is > >>> changed). > >> > >> We've already been talking about dynamic layers, but they don't > >> actually solve the issue of one source of truth for an entire stack > >> that I'm talking about. > > > > And of course, at a glance, I already see two things that could have > > easily been submitted to meta-virtualization. Which shows the point, > > that once you have layers as a place to stash fixes/features, they > > typically don't get submitted to the centralized layers. > > This is something we plan to fix in the next months and move some parts from > meta-arm-autonomy to meta-virtualization. > > By curiosity, what are the 2 things you have in mind ?
The two that I noticed were the qemu split (I did something similar for runX, but it would be nice to consolidate them). There were also some config/console changes for Xen that I noticed which could be in the main layer. We also haven't recently revisited the image definitions and it would be interesting to see if we could provide some easily extendable base images .. but that's not what I noticed in my quick glance earlier, just something I thought of since. I'm currently having some build issues with qemuarm64 and Xen in master, so I'm trying to fight through that before I can try much else out. Cheers, Bruce > > Cheers > Bertrand > > > > > Cheers, > > > > Bruce > > > >> > >> Cheers, > >> > >> Bruce > >> > >>> > >>> If you want an example of how to use dynamic layers you can check > >>> meta-arm-autonomy sub layer of meta-arm [1] (which also contains other > >>> interesting bits related to using Xen with Yocto). > >>> > >>> Cheers > >>> Bertrand > >>> > >>> > >>> > >>> IMPORTANT NOTICE: The contents of this email and any attachments are > >>> confidential and may also be privileged. If you are not the intended > >>> recipient, please notify the sender immediately and do not disclose the > >>> contents to any other person, use it for any purpose, or store or copy > >>> the information in any medium. Thank you. > >> > >> > >> > >> -- > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> thee at its end > >> - "Use the force Harry" - Gandalf, Star Trek II > >> > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#5432): https://lists.yoctoproject.org/g/meta-virtualization/message/5432 Mute This Topic: https://lists.yoctoproject.org/mt/74701134/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
