> On 17 Jun 2020, at 18:00, Bruce Ashfield via lists.yoctoproject.org > <[email protected]> wrote: > > On Wed, Jun 17, 2020 at 12:14 PM Bertrand Marquis > <[email protected]> wrote: >> >> >> >>> On 17 Jun 2020, at 13:48, Bruce Ashfield via lists.yoctoproject.org >>> <[email protected]> wrote: >>> >>> On Wed, Jun 17, 2020 at 8:45 AM Bruce Ashfield via >>> lists.yoctoproject.org >>> <[email protected]> wrote: >>>> >>>> On Wed, Jun 17, 2020 at 4:29 AM Bertrand Marquis >>>> <[email protected]> wrote: >>>>> >>>>> Hi, >>>>> >>>>>> On 16 Jun 2020, at 22:43, Christopher Clark via lists.yoctoproject.org >>>>>> <[email protected]> wrote: >>>>>> >>>>>> On Mon, Jun 15, 2020 at 5:45 AM Bruce Ashfield >>>>>> <[email protected]> wrote: >>>>>>> On Mon, Jun 15, 2020 at 5:11 AM Christopher Clark >>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>> On Fri, Jun 5, 2020 at 5:58 PM Corey Minyard <[email protected]> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> On Fri, Jun 05, 2020 at 07:53:53PM -0400, Bruce Ashfield wrote: >>>>>>>>>> On Fri, Jun 5, 2020 at 5:16 PM Corey Minyard <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> On Fri, Jun 05, 2020 at 07:55:37PM +0000, Stewart Hildebrand wrote: >>>>>>>>>>>> + Corey >>>>>>>>>>>> >>>>>>>>>>>> On Friday, June 5, 2020 3:19 PM, Bruce Ashfield wrote: >>>>>>>>>>>>> On Fri, Jun 5, 2020 at 3:12 PM Stewart Hildebrand wrote: >>>>>> >>>>>>>>>>>> Corey: you are hereby encouraged to submit patches to >>>>>>>>>>>> meta-virtualization. >>>>>>>>>>> >>>>>>>>>>> Ok. The layer has the following basic pieces: >>>>>>>> >>>>>>>> Before we get to the other pieces, I'd like to cover the new "rpixen" >>>>>>>> MACHINE type that the layer introduces. >>>>>>>> >>>>>>>> My preference is for avoiding introduction of another MACHINE to >>>>>>>> reconfigure an existing one to run Xen, if possible, and use the >>>>>>>> existing "raspberrypi4-64". I'm hoping to avoid the pattern of >>>>>>>> creating a new machine for Xen for each board that we add support for. >>>>>>>> In meta-virtualization, there's the "xen" DISTRO_FEATURE, which is >>>>>>>> used to turn on Xen-specific functionality and compatibility, and I'd >>>>>>>> like to explore whether that can be made to be sufficient to enable >>>>>>>> what is needed. >>>>>>>> >>>>>>>> To that end, I've done an initial pass to see what it might take and >>>>>>>> the work-in-progress from that is posted here: >>>>>>>> https://github.com/dozylynx/meta-virtualization/tree/raspberry-pi4-initial-xen >>>>>>>> >>>>>>>> Some minor changes to other layers could assist - eg. to remove the >>>>>>>> need for a guest filesystem to contain the hypervisor binary - and >>>>>>>> there's still some tidying to do. >>>>>>>> >>>>>>>>>>> 1 The xen patches for the Pi4, just a few patches. As the Xen group >>>>>>>>>>> fixes things, I keep adding :). >>>>>>>> >>>>>>>> Unfortunately I had to drop these patches from my local test to get it >>>>>>>> to boot. It could easily be a local build issue or a fault with the >>>>>>>> one test I've run so far, but I did have success booting with just Xen >>>>>>>> 4.13 and I'd like to get a bit more understanding and confidence in >>>>>>>> them before we bring them in. >>>>>>>> >>>>>>>>>>> 2 Hacks for getting the Pi4 kernel config right for xen. This >>>>>>>>>>> should go >>>>>>>>>>> away if you don't use the kernel from the Pi4 yocto layer, as it >>>>>>>>>>> doesn't work like most kernels in yocto. >>>>>>>>>> >>>>>>>>>> I should take a look at the configs and see if I can create a >>>>>>>>>> fragment >>>>>>>>>> or two, but I can take care of that. >>>>>>>>> >>>>>>>>> That shouldn't be necessary. The standard fragments work, it's just >>>>>>>>> that the Pi kernel does not use them. So this is really Pi+Xen >>>>>>>>> specific, and hopefully they can fix the Pi kernel to use the normal >>>>>>>>> fragments in the future. >>>>>>>> >>>>>>>> The linux-raspberrypi kernel does use linux-yocto; it's just that >>>>>>>> meta-virtualization >>>>>>>> needs a matching .inc file to be present for the kernel version that >>>>>>>> you're using. >>>>>>>> Assuming you're using Linux 4.19 (which is what I've tested with) add >>>>>>>> this file: >>>>>>>> meta-virtualization/recipes-kernel/linux/linux-yocto_4.19_virtualization.inc >>>>>>>> containing just: >>>>>>>> include linux-yocto_virtualization.inc >>>>>>>> which will then enable the linux-raspberrypi kernel to add the >>>>>>>> meta-virt Xen fragment. >>>>>>>> >>>>>>>> There are also two Linux patches: >>>>>>>> 1) Disable DMA in the SDHCI driver >>>>>>>> This one needs more information in the commit text to understand what >>>>>>>> is motivating doing this and what the effects of it are. Should it go >>>>>>>> into the standard Raspberry Pi kernel? >>>>>>>> 2) Fix PCIe in dom0 for RPi4 >>>>>>>> Is this fixed upstream in more recent kernels? It would be good to >>>>>>>> have a pointer to that if so. >>>>>>>> >>>>>>>> Bruce: To apply these to just the Raspberry Pi kernel when it's being >>>>>>>> used with Xen, a kernel bbappend in a raspberrypi dynamic-layers might >>>>>>>> be an option to consider - eg: >>>>>>>> https://github.com/dozylynx/meta-virtualization/tree/raspberry-pi4-initial-xen/dynamic-layers/raspberrypi/recipes-kernel/linux >>>>>>> >>>>>>> I can most likely live with that. I obviously make sure that the >>>>>>> reference linux-yocto kernel doesn't need anything to work out of the >>>>>>> box, but we can't (and shouldn't) enforce that choice on everyone. I'd >>>>>>> rather have patches centralized in a topic layer like >>>>>>> meta-virtualization, so if we need to add a dynamic layer and a few >>>>>>> patches, that's a good place to be. >>>>>> >>>>>> please see my new comment below. >>>>>> >>>>>>>>>>> 5 A few Pi-specific hacks for config and u-boot. >>>>>>>>>> >>>>>>>>>> #5 does sound like BSP stuff. Is any of it destined for the rpi >>>>>>>>>> layers >>>>>>>>>> ? Or is it both rpi AND xen specific, so doesn't really make sense >>>>>>>>>> there either ? >>>>>>>>> >>>>>>>>> It's both Pi and Xen specific. If everything gets put where it should >>>>>>>>> be, that would be the only thing left in this layer :). >>>>>>>> >>>>>>>> These are in recipes-bsp and I agree that they're both Pi and Xen >>>>>>>> specific. I think these are small enough pieces that keeping them in >>>>>>>> meta-virtualization could be a reasonable call, since that's the layer >>>>>>>> where Xen support is focussed, and so they can be added to: >>>>>>>> dynamic-layers/raspberrypi/recipes-bsp >>>>>>>> which indicates their status as amendments to the meta-raspberrypi >>>>>>>> layer. Changes to them would then be easily coordinated with the Xen >>>>>>>> recipes. >>>>>>> >>>>>>> Agreed. >>>>>> >>>>>> Since I wrote this, we've seen some expressed interest in support for >>>>>> running Xen on the NVIDIA Jetson Nano and Xavier NX boards, and on the >>>>>> xen-devel mailing list, a report of success running Xen on the RockPro64 >>>>>> board: >>>>>> https://lists.xenproject.org/archives/html/xen-devel/2020-06/msg01067.html >>>>>> >>>>>> I've personally built Xen with meta-virtualization to run on the >>>>>> Cubietruck for >>>>>> development and testing; the Odroid C2 and XU4 are of interest, as is the >>>>>> Xilinx Ultra-96-V2 board; and the NVidia Jetson TX1 at one point had some >>>>>> non-upstream patches available to enable Xen on it. PCEngines maintains >>>>>> Xen compatibility for their APU2 in meta-pcengines. >>>>>> >>>>>> There are also some challenges being encountered in getting a more >>>>>> recent Linux >>>>>> kernel working as a dom0 on the Raspberry Pi 4 - eg. the Linux >>>>>> Foundation Eve >>>>>> Project run their own patches for 5.6 here: >>>>>> https://github.com/lf-edge/eve/tree/master/pkg/new-kernel/patches-5.6.x >>>>>> >>>>>> All of this points towards it being a reasonable proposition to have a >>>>>> dedicated Xen hardware support Yocto layer, so that board-specific >>>>>> tweaks for >>>>>> hardware compatibility with Xen can be maintained without complicating >>>>>> meta-virtualization, while continuing to pool Xen-aware contributions in >>>>>> a centralized layer. >>>>>> >>>>>> I'd like to propose creating: 'meta-xen-bsp'; and I'm willing to work on >>>>>> maintaining it. Feedback to this suggestion is welcome. >>>>>> >>>>>> Bruce: how does this sound to you? >>>>> >>>>> I had this problem working on some other BSPs to support Xen with >>>>> meta-virtualization (you can check this in meta-arm[1]). >>>>> >>>>> I would think the best way to handle this case is to use dynamic layers >>>>> and push the required support inside the layer containing the BSP. >>>>> This would make the maintenance easier (for example when the kernel is >>>>> changed). >>>> >>>> We've already been talking about dynamic layers, but they don't >>>> actually solve the issue of one source of truth for an entire stack >>>> that I'm talking about. >>> >>> And of course, at a glance, I already see two things that could have >>> easily been submitted to meta-virtualization. Which shows the point, >>> that once you have layers as a place to stash fixes/features, they >>> typically don't get submitted to the centralized layers. >> >> This is something we plan to fix in the next months and move some parts from >> meta-arm-autonomy to meta-virtualization. >> >> By curiosity, what are the 2 things you have in mind ? > > The two that I noticed were the qemu split (I did something similar > for runX, but it would be nice to consolidate them). There were also > some config/console changes for Xen that I noticed which could be in > the main layer. > > We also haven't recently revisited the image definitions and it would > be interesting to see if we could provide some easily extendable base > images .. but that's not what I noticed in my quick glance earlier, > just something I thought of since. > > I'm currently having some build issues with qemuarm64 and Xen in > master, so I'm trying to fight through that before I can try much else > out.
I fully agree with those. I would also think that introducing the idea of “xen dom0” vs “xen guest” could make sense. I also have some ideas for the xen recipe itself as current status does not allow to have a devel version. I noted down your comments in our backlog for the next months Cheers Bertrand > > Cheers, > > Bruce > > >> >> Cheers >> Bertrand >> >>> >>> Cheers, >>> >>> Bruce >>> >>>> >>>> Cheers, >>>> >>>> Bruce >>>> >>>>> >>>>> If you want an example of how to use dynamic layers you can check >>>>> meta-arm-autonomy sub layer of meta-arm [1] (which also contains other >>>>> interesting bits related to using Xen with Yocto). >>>>> >>>>> Cheers >>>>> Bertrand >>>>> >>>>> >>>>> >>>>> IMPORTANT NOTICE: The contents of this email and any attachments are >>>>> confidential and may also be privileged. If you are not the intended >>>>> recipient, please notify the sender immediately and do not disclose the >>>>> contents to any other person, use it for any purpose, or store or copy >>>>> the information in any medium. Thank you. >>>> >>>> >>>> >>>> -- >>>> - Thou shalt not follow the NULL pointer, for chaos and madness await >>>> thee at its end >>>> - "Use the force Harry" - Gandalf, Star Trek II >>>> >>> >>> >>> >>> -- >>> - Thou shalt not follow the NULL pointer, for chaos and madness await >>> thee at its end >>> - "Use the force Harry" - Gandalf, Star Trek II >>> >> > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#5436): https://lists.yoctoproject.org/g/meta-virtualization/message/5436 Mute This Topic: https://lists.yoctoproject.org/mt/74701134/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
