On 28/10/2022 17:45, Bertrand Marquis wrote: > > > Hi Julien, > >> On 28 Oct 2022, at 16:01, Julien Grall <jul...@xen.org> wrote: >> >> >> >> On 28/10/2022 15:37, Bertrand Marquis wrote: >>> Hi Julien, >> >> Hi Bertrand, >> >>>> On 28 Oct 2022, at 14:27, Julien Grall <jul...@xen.org> wrote: >>>> >>>> >>>> >>>> On 28/10/2022 14:13, Bertrand Marquis wrote: >>>>> Hi Julien, >>>> >>>> Hi Bertrand, >>>> >>>>>> On 28 Oct 2022, at 14:06, Julien Grall <jul...@xen.org> wrote: >>>>>> >>>>>> Hi Rahul, >>>>>> >>>>>> On 28/10/2022 13:54, Rahul Singh wrote: >>>>>>>>>>>> For ACPI, I would have expected the information to be found in the >>>>>>>>>>>> IOREQ. >>>>>>>>>>>> >>>>>>>>>>>> So can you add more context why this is necessary for everyone? >>>>>>>>>>> We have information for IOMMU and Master-ID but we don’t have >>>>>>>>>>> information for linking vMaster-ID to pMaster-ID. >>>>>>>>>> >>>>>>>>>> I am confused. Below, you are making the virtual master ID optional. >>>>>>>>>> So shouldn't this be mandatory if you really need the mapping with >>>>>>>>>> the virtual ID? >>>>>>>>> vMasterID is optional if user knows pMasterID is unique on the >>>>>>>>> system. But if pMasterId is not unique then user needs to provide the >>>>>>>>> vMasterID. >>>>>>>> >>>>>>>> So the expectation is the user will be able to know that the pMasterID >>>>>>>> is uniq. This may be easy with a couple of SMMUs, but if you have 50+ >>>>>>>> (as suggested above). This will become a pain on larger system. >>>>>>>> >>>>>>>> IHMO, it would be much better if we can detect that in libxl (see >>>>>>>> below). >>>>>>> We can make the vMasterID compulsory to avoid complexity in libxl to >>>>>>> solve this >>>>>> >>>>>> In general, complexity in libxl is not too much of problem. >>>>> I am a bit unsure about this strategy. >>>>> Currently xl has one configuration file where you put all Xen parameters. >>>>> The device tree is only needed by some guests to have a description of >>>>> the system they run on. >>>>> If we change the model and say that Xen configuration parameters are both >>>>> in the configuration and the device tree, we somehow enforce to have a >>>>> device tree even though some guests do not need it at all (for example >>>>> Zephyr). >>>> >>>> I think my approach was misunderstood because there is no change in the >>>> existing model. >>>> >>>> What I am suggesting is to not introduce iommu_devid_map but instead let >>>> libxl allocate the virtual Master-ID and create the mapping with the >>>> physical Master-ID. >>>> >>>> Libxl would then update the property "iommus" in the device-tree with the >>>> allocated virtual Master-ID. >>> Ok I understand now. >>>> >>>> Each node in the partial device-tree would need to have a property >>>> to refer to the physical device just so we know how to update the >>>> "iommus". The list of device passthrough will still be specified in the >>>> configuration file. IOW, the partial device-tree is not directly involved >>>> in the configuration of the guest. >>> But we will generate it. How would something like Zephyr guest work ? >>> Zephyr is not using the device tree we pass, it has an embedded one. >> >> In general, guest that don't use the device-tree/ACPI table to detect the >> layout are already in a bad situation because we don't guarantee that the >> layout (memory, interrupt...) will be stable across Xen version. Although, >> there are a implicit agreement that the layout will not change for minor >> release (i.e. 4.14.x). > > Well right now we have no ACPI support. > But I still think that a non dtb guest is definitely a use case we need to > keep in mind for embedded and safety as most proprietary RTOS are not using a > device tree. > >> >> But see below for some suggestions how this could be handled. >> >>>> >>>> So far, I don't see a particular issue with this approach because the >>>> vMaster ID algorithm allocation should be generic. But please let me know >>>> if you think there are bits I am missing. >>> I am a bit afraid of things that are “automatic”. >>> For everything else we let the user in control (IPA for mapping, virtual >>> interrupt number) and in this case we switch to a model where we >>> automatically generated a vMaster ID. >> >> We only let the user control where the device is mapped. But this is quite >> fragile... I think this should be generated at runtime. >> >>> With this model, guest not using the device tree will have to guess the >>> vMaster ID or somehow know how the tools are generating it to use the right >>> one. >> >> To be honest, this is already the case today because the layout exposed to >> the guest is technically not fixed. Yes, so far, we haven't changed it too >> much. But sooner or later, this is going to bite because we made clear that >> the layout is not stable. >> >> Now, if those projects are willing to rebuild for each version, then we >> could use the following approach: >> 1) Write the xl.cfg >> 2) Ask libxl to generate the device-tree >> 3) Build Zephyr >> 4) Create the domain >> >> The expectation is for a given Xen version (and compatible), libxl will >> always generate the same Device-Tree. > > This is a good idea yes :-)
Zephyr still uses a device tree but in a static way - everything must be defined in a .dts before building it. The steps mentioned by Julien are already followed by Zephyr when building it to run as a Xen VM. You can take a look at the "Updating configuration" section at the bottom of the following site: https://docs.zephyrproject.org/latest/boards/arm64/xenvm/doc/index.html So, as we tend to use Zephyr as a de facto RTOS for Xen, it is already aware of possible changes to the layout. > > Cheers > Bertrand > >> >> Cheers, >> >> -- >> Julien Grall > ~Michal
