Re: [Xen-devel] Xen PVH support in grub2
On Tue, Nov 07, 2017 at 11:10:29AM -0500, Boris Ostrovsky wrote: > On 11/07/2017 02:42 AM, Juergen Gross wrote: > > On 06/11/17 17:42, Boris Ostrovsky wrote: > >> On 11/06/2017 10:05 AM, Juergen Gross wrote: > >>> On 06/11/17 15:51, Boris Ostrovsky wrote: > On 11/06/2017 02:16 AM, Juergen Gross wrote: > > On 03/11/17 20:00, Boris Ostrovsky wrote: > >> On 11/03/2017 02:40 PM, Juergen Gross wrote: > >>> On 03/11/17 19:35, Boris Ostrovsky wrote: > On 11/03/2017 02:23 PM, Juergen Gross wrote: > > On 03/11/17 19:19, Boris Ostrovsky wrote: > >> On 11/03/2017 02:05 PM, Juergen Gross wrote: > >>> So again the question: how to tell whether we are PVH or HVM in > >>> init_hypervisor_platform()? ACPi tables are scanned way later... > >> Can we make grub/OVMF append a boot option? > >> > >> Or set setup_header.hardware_subarch to something? We already have > >> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to > >> use > >> hardware_subarch_data (will need to get a buy-in from x86 > >> maintainers, I > >> think). > > But wouldn't this break the idea to reuse the native boot paths in > > grub/OVMF without further modifications? > WDYM? We will have to have some sort of a plugin in either one to > build > the zeropage anyway. So we'd set hardware_subarch there, in addition > to > other things like setting memory and such. > >>> But isn't the zeropage already being built? I admit that setting > >>> subarch > >>> isn't a big deal, but using another entry with a passed-through pvh > >>> start struct isn't either... > >> I don't follow, sorry. My understanding is that zeropage will be built > >> by PVH-enlightened grub so part of this process would be setting the > >> subarch bit. > > My reasoning was based on Roger's remark: > > > > "OTOH if Linux is capable of booting from the native entry point inside > > of a PVH container, we would only have to port OVMF and grub in order > > to work inside of a PVH container, leaving the rest of the logic > > untouched." > Right, and in my mind porting OVMF/grub includes creating proper > zeropage. > >>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run > >>> in PVH environment without touching the parts setting up anything for > >>> the new kernel. > >> Someone needs to do what xen_prepare_pvh() does. > > As the loader is filling in the memory map information > > That's the thing that I thought may need to be done by us (setting > commandline too) . But I haven't looked at Xen support in grub so maybe > it's already there. > > > the only thing > > remaining would be setting xen_pvh. And this could be delayed as my test > > have shown, so we only need to detect the PVH case from inside the > > kernel. One possibility would be the flags in the ACPI FADT table as > > Roger mentioned, another idea would be a flag in zeropage set by the > > loader. > > > >> And, for 64-bit, we also may need to build early pagetables since > >> startup_64() (unlike startup_32()) expects paging to be on. (I don't > >> know whether this is already part of standard FW codepath) > > This would be done the same way as for a native kernel. > > This is done by Linux trampoline code. AFAIK grub loads kernel in realmode. Adding in Maran who is looking at re-using PVH for qemu launching the kernels. > > > -boris > > ___ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/07/2017 02:42 AM, Juergen Gross wrote: > On 06/11/17 17:42, Boris Ostrovsky wrote: >> On 11/06/2017 10:05 AM, Juergen Gross wrote: >>> On 06/11/17 15:51, Boris Ostrovsky wrote: On 11/06/2017 02:16 AM, Juergen Gross wrote: > On 03/11/17 20:00, Boris Ostrovsky wrote: >> On 11/03/2017 02:40 PM, Juergen Gross wrote: >>> On 03/11/17 19:35, Boris Ostrovsky wrote: On 11/03/2017 02:23 PM, Juergen Gross wrote: > On 03/11/17 19:19, Boris Ostrovsky wrote: >> On 11/03/2017 02:05 PM, Juergen Gross wrote: >>> So again the question: how to tell whether we are PVH or HVM in >>> init_hypervisor_platform()? ACPi tables are scanned way later... >> Can we make grub/OVMF append a boot option? >> >> Or set setup_header.hardware_subarch to something? We already have >> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to >> use >> hardware_subarch_data (will need to get a buy-in from x86 >> maintainers, I >> think). > But wouldn't this break the idea to reuse the native boot paths in > grub/OVMF without further modifications? WDYM? We will have to have some sort of a plugin in either one to build the zeropage anyway. So we'd set hardware_subarch there, in addition to other things like setting memory and such. >>> But isn't the zeropage already being built? I admit that setting subarch >>> isn't a big deal, but using another entry with a passed-through pvh >>> start struct isn't either... >> I don't follow, sorry. My understanding is that zeropage will be built >> by PVH-enlightened grub so part of this process would be setting the >> subarch bit. > My reasoning was based on Roger's remark: > > "OTOH if Linux is capable of booting from the native entry point inside > of a PVH container, we would only have to port OVMF and grub in order > to work inside of a PVH container, leaving the rest of the logic > untouched." Right, and in my mind porting OVMF/grub includes creating proper zeropage. >>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run >>> in PVH environment without touching the parts setting up anything for >>> the new kernel. >> Someone needs to do what xen_prepare_pvh() does. > As the loader is filling in the memory map information That's the thing that I thought may need to be done by us (setting commandline too) . But I haven't looked at Xen support in grub so maybe it's already there. > the only thing > remaining would be setting xen_pvh. And this could be delayed as my test > have shown, so we only need to detect the PVH case from inside the > kernel. One possibility would be the flags in the ACPI FADT table as > Roger mentioned, another idea would be a flag in zeropage set by the > loader. > >> And, for 64-bit, we also may need to build early pagetables since >> startup_64() (unlike startup_32()) expects paging to be on. (I don't >> know whether this is already part of standard FW codepath) > This would be done the same way as for a native kernel. This is done by Linux trampoline code. AFAIK grub loads kernel in realmode. -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 06/11/17 12:36, Juergen Gross wrote: > On 03/11/17 13:17, Roger Pau Monné wrote: >> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: >>> On 29/09/17 17:51, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > On 29/09/17 17:24, Roger Pau Monné wrote: >> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >> Then, I also wonder whether it would make sense for this grub to load >> the kernel using the PVH entry point or the native entry point. Would >> it be possible to boot a Linux kernel up to the point where cpuid can >> be used inside of a PVH container? > > I don't think today's Linux allows that. This has been discussed > very thoroughly at the time Boris added PVH V2 support to the kernel. OK, I'm not going to insist on that, but my plans for FreeBSD is to make the native entry point capable of booting inside of a PVH container up to the point where cpuid (or whatever method) can be used to detect the environment. >>> >>> Looking more thoroughly into the Linux boot code I think this could >>> work for Linux, too. But only if we can tell PVH from HVM in the guest. >>> How would you do that in FreeBSD? Via flags in the boot params? This >>> would the have to be done in the boot loader (e.g. grub or OVMF). >> >> My plan was not to differentiate between HVM and PVH, but rather to >> make use of the ACPI information in order to decide which devices are >> available and which are not inside of a PVH guest. >> >> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH >> we already set "VGA Not Present" and "CMOS RTC Not Present". There >> might be other flags/fields that must be set, but I would like to >> avoid having a CPUID bit or similar saying "PVH", because then Xen >> will be tied to always providing the same set of devices in PVH >> containers. > > I looked through the xen_pvh_domain() use cases in the Linux kernel > again. > > Maybe we can really manage to not need differentiating PVH from HVM > until ACPI table scan. We'd need another hook for Xen, but this should > be easy as KVM already has a hook where we'd need one. So this can be > made more general and we are fine. > > I even think we can drop some of the PVH tests, as the PVH-specific > handling (e.g. for grant table initialization) should work for HVM, too. So I did a little test now: with some small patches added I've managed to boot a PVH Linux kernel without any special handling in the early PVH paths of the kernel (only setting up initial page tables and faking a memory map similar to the one grub would deliver). PVH detection is done via ACPI table ("VGA Not Present" and "CMOS RTC Not Present" in a HVM domain will result in PVH assumed). If nobody objects to this handling I'll send patches soon. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 06/11/17 17:42, Boris Ostrovsky wrote: > On 11/06/2017 10:05 AM, Juergen Gross wrote: >> On 06/11/17 15:51, Boris Ostrovsky wrote: >>> On 11/06/2017 02:16 AM, Juergen Gross wrote: On 03/11/17 20:00, Boris Ostrovsky wrote: > On 11/03/2017 02:40 PM, Juergen Gross wrote: >> On 03/11/17 19:35, Boris Ostrovsky wrote: >>> On 11/03/2017 02:23 PM, Juergen Gross wrote: On 03/11/17 19:19, Boris Ostrovsky wrote: > On 11/03/2017 02:05 PM, Juergen Gross wrote: >> So again the question: how to tell whether we are PVH or HVM in >> init_hypervisor_platform()? ACPi tables are scanned way later... > Can we make grub/OVMF append a boot option? > > Or set setup_header.hardware_subarch to something? We already have > X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use > hardware_subarch_data (will need to get a buy-in from x86 > maintainers, I > think). But wouldn't this break the idea to reuse the native boot paths in grub/OVMF without further modifications? >>> WDYM? We will have to have some sort of a plugin in either one to build >>> the zeropage anyway. So we'd set hardware_subarch there, in addition to >>> other things like setting memory and such. >> But isn't the zeropage already being built? I admit that setting subarch >> isn't a big deal, but using another entry with a passed-through pvh >> start struct isn't either... > I don't follow, sorry. My understanding is that zeropage will be built > by PVH-enlightened grub so part of this process would be setting the > subarch bit. My reasoning was based on Roger's remark: "OTOH if Linux is capable of booting from the native entry point inside of a PVH container, we would only have to port OVMF and grub in order to work inside of a PVH container, leaving the rest of the logic untouched." >>> Right, and in my mind porting OVMF/grub includes creating proper zeropage. >> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run >> in PVH environment without touching the parts setting up anything for >> the new kernel. > > Someone needs to do what xen_prepare_pvh() does. As the loader is filling in the memory map information the only thing remaining would be setting xen_pvh. And this could be delayed as my test have shown, so we only need to detect the PVH case from inside the kernel. One possibility would be the flags in the ACPI FADT table as Roger mentioned, another idea would be a flag in zeropage set by the loader. > And, for 64-bit, we also may need to build early pagetables since > startup_64() (unlike startup_32()) expects paging to be on. (I don't > know whether this is already part of standard FW codepath) This would be done the same way as for a native kernel. >>> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which >>> is what init_pvh_bootparams() does. In fact, whatever is done in the >>> firmware should probably match what that routine does. >> So it wouldn't be possible any longer to tell whether the kernel has >> been booted directly or via grub. I don't like this. The loader type >> is accessible via sysfs after all. > > I didn't know that. What is the path? /proc/sys/kernel/bootloader_type Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/06/2017 10:05 AM, Juergen Gross wrote: > On 06/11/17 15:51, Boris Ostrovsky wrote: >> On 11/06/2017 02:16 AM, Juergen Gross wrote: >>> On 03/11/17 20:00, Boris Ostrovsky wrote: On 11/03/2017 02:40 PM, Juergen Gross wrote: > On 03/11/17 19:35, Boris Ostrovsky wrote: >> On 11/03/2017 02:23 PM, Juergen Gross wrote: >>> On 03/11/17 19:19, Boris Ostrovsky wrote: On 11/03/2017 02:05 PM, Juergen Gross wrote: > So again the question: how to tell whether we are PVH or HVM in > init_hypervisor_platform()? ACPi tables are scanned way later... Can we make grub/OVMF append a boot option? Or set setup_header.hardware_subarch to something? We already have X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use hardware_subarch_data (will need to get a buy-in from x86 maintainers, I think). >>> But wouldn't this break the idea to reuse the native boot paths in >>> grub/OVMF without further modifications? >> WDYM? We will have to have some sort of a plugin in either one to build >> the zeropage anyway. So we'd set hardware_subarch there, in addition to >> other things like setting memory and such. > But isn't the zeropage already being built? I admit that setting subarch > isn't a big deal, but using another entry with a passed-through pvh > start struct isn't either... I don't follow, sorry. My understanding is that zeropage will be built by PVH-enlightened grub so part of this process would be setting the subarch bit. >>> My reasoning was based on Roger's remark: >>> >>> "OTOH if Linux is capable of booting from the native entry point inside >>> of a PVH container, we would only have to port OVMF and grub in order >>> to work inside of a PVH container, leaving the rest of the logic >>> untouched." >> Right, and in my mind porting OVMF/grub includes creating proper zeropage. > Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run > in PVH environment without touching the parts setting up anything for > the new kernel. Someone needs to do what xen_prepare_pvh() does. And, for 64-bit, we also may need to build early pagetables since startup_64() (unlike startup_32()) expects paging to be on. (I don't know whether this is already part of standard FW codepath) > >> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which >> is what init_pvh_bootparams() does. In fact, whatever is done in the >> firmware should probably match what that routine does. > So it wouldn't be possible any longer to tell whether the kernel has > been booted directly or via grub. I don't like this. The loader type > is accessible via sysfs after all. I didn't know that. What is the path? -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 06/11/17 15:51, Boris Ostrovsky wrote: > On 11/06/2017 02:16 AM, Juergen Gross wrote: >> On 03/11/17 20:00, Boris Ostrovsky wrote: >>> On 11/03/2017 02:40 PM, Juergen Gross wrote: On 03/11/17 19:35, Boris Ostrovsky wrote: > On 11/03/2017 02:23 PM, Juergen Gross wrote: >> On 03/11/17 19:19, Boris Ostrovsky wrote: >>> On 11/03/2017 02:05 PM, Juergen Gross wrote: So again the question: how to tell whether we are PVH or HVM in init_hypervisor_platform()? ACPi tables are scanned way later... >>> Can we make grub/OVMF append a boot option? >>> >>> Or set setup_header.hardware_subarch to something? We already have >>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use >>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I >>> think). >> But wouldn't this break the idea to reuse the native boot paths in >> grub/OVMF without further modifications? > WDYM? We will have to have some sort of a plugin in either one to build > the zeropage anyway. So we'd set hardware_subarch there, in addition to > other things like setting memory and such. But isn't the zeropage already being built? I admit that setting subarch isn't a big deal, but using another entry with a passed-through pvh start struct isn't either... >>> I don't follow, sorry. My understanding is that zeropage will be built >>> by PVH-enlightened grub so part of this process would be setting the >>> subarch bit. >> My reasoning was based on Roger's remark: >> >> "OTOH if Linux is capable of booting from the native entry point inside >> of a PVH container, we would only have to port OVMF and grub in order >> to work inside of a PVH container, leaving the rest of the logic >> untouched." > > Right, and in my mind porting OVMF/grub includes creating proper zeropage. Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run in PVH environment without touching the parts setting up anything for the new kernel. > BTW, another option might be to "type_of_loader = (9 << 4) | 0", which > is what init_pvh_bootparams() does. In fact, whatever is done in the > firmware should probably match what that routine does. So it wouldn't be possible any longer to tell whether the kernel has been booted directly or via grub. I don't like this. The loader type is accessible via sysfs after all. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/06/2017 02:16 AM, Juergen Gross wrote: > On 03/11/17 20:00, Boris Ostrovsky wrote: >> On 11/03/2017 02:40 PM, Juergen Gross wrote: >>> On 03/11/17 19:35, Boris Ostrovsky wrote: On 11/03/2017 02:23 PM, Juergen Gross wrote: > On 03/11/17 19:19, Boris Ostrovsky wrote: >> On 11/03/2017 02:05 PM, Juergen Gross wrote: >>> So again the question: how to tell whether we are PVH or HVM in >>> init_hypervisor_platform()? ACPi tables are scanned way later... >> Can we make grub/OVMF append a boot option? >> >> Or set setup_header.hardware_subarch to something? We already have >> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use >> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I >> think). > But wouldn't this break the idea to reuse the native boot paths in > grub/OVMF without further modifications? WDYM? We will have to have some sort of a plugin in either one to build the zeropage anyway. So we'd set hardware_subarch there, in addition to other things like setting memory and such. >>> But isn't the zeropage already being built? I admit that setting subarch >>> isn't a big deal, but using another entry with a passed-through pvh >>> start struct isn't either... >> I don't follow, sorry. My understanding is that zeropage will be built >> by PVH-enlightened grub so part of this process would be setting the >> subarch bit. > My reasoning was based on Roger's remark: > > "OTOH if Linux is capable of booting from the native entry point inside > of a PVH container, we would only have to port OVMF and grub in order > to work inside of a PVH container, leaving the rest of the logic > untouched." Right, and in my mind porting OVMF/grub includes creating proper zeropage. BTW, another option might be to "type_of_loader = (9 << 4) | 0", which is what init_pvh_bootparams() does. In fact, whatever is done in the firmware should probably match what that routine does. -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 13:17, Roger Pau Monné wrote: > On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: >> On 29/09/17 17:51, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: On 29/09/17 17:24, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > Then, I also wonder whether it would make sense for this grub to load > the kernel using the PVH entry point or the native entry point. Would > it be possible to boot a Linux kernel up to the point where cpuid can > be used inside of a PVH container? I don't think today's Linux allows that. This has been discussed very thoroughly at the time Boris added PVH V2 support to the kernel. >>> >>> OK, I'm not going to insist on that, but my plans for FreeBSD is to >>> make the native entry point capable of booting inside of a PVH >>> container up to the point where cpuid (or whatever method) can be used >>> to detect the environment. >> >> Looking more thoroughly into the Linux boot code I think this could >> work for Linux, too. But only if we can tell PVH from HVM in the guest. >> How would you do that in FreeBSD? Via flags in the boot params? This >> would the have to be done in the boot loader (e.g. grub or OVMF). > > My plan was not to differentiate between HVM and PVH, but rather to > make use of the ACPI information in order to decide which devices are > available and which are not inside of a PVH guest. > > For example in the FADT "IA-PC Boot Architecture Flags" field for PVH > we already set "VGA Not Present" and "CMOS RTC Not Present". There > might be other flags/fields that must be set, but I would like to > avoid having a CPUID bit or similar saying "PVH", because then Xen > will be tied to always providing the same set of devices in PVH > containers. I looked through the xen_pvh_domain() use cases in the Linux kernel again. Maybe we can really manage to not need differentiating PVH from HVM until ACPI table scan. We'd need another hook for Xen, but this should be easy as KVM already has a hook where we'd need one. So this can be made more general and we are fine. I even think we can drop some of the PVH tests, as the PVH-specific handling (e.g. for grant table initialization) should work for HVM, too. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 20:00, Boris Ostrovsky wrote: > On 11/03/2017 02:40 PM, Juergen Gross wrote: >> On 03/11/17 19:35, Boris Ostrovsky wrote: >>> On 11/03/2017 02:23 PM, Juergen Gross wrote: On 03/11/17 19:19, Boris Ostrovsky wrote: > On 11/03/2017 02:05 PM, Juergen Gross wrote: >> So again the question: how to tell whether we are PVH or HVM in >> init_hypervisor_platform()? ACPi tables are scanned way later... > Can we make grub/OVMF append a boot option? > > Or set setup_header.hardware_subarch to something? We already have > X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use > hardware_subarch_data (will need to get a buy-in from x86 maintainers, I > think). But wouldn't this break the idea to reuse the native boot paths in grub/OVMF without further modifications? >>> WDYM? We will have to have some sort of a plugin in either one to build >>> the zeropage anyway. So we'd set hardware_subarch there, in addition to >>> other things like setting memory and such. >> But isn't the zeropage already being built? I admit that setting subarch >> isn't a big deal, but using another entry with a passed-through pvh >> start struct isn't either... > > I don't follow, sorry. My understanding is that zeropage will be built > by PVH-enlightened grub so part of this process would be setting the > subarch bit. My reasoning was based on Roger's remark: "OTOH if Linux is capable of booting from the native entry point inside of a PVH container, we would only have to port OVMF and grub in order to work inside of a PVH container, leaving the rest of the logic untouched." Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/03/2017 02:40 PM, Juergen Gross wrote: > On 03/11/17 19:35, Boris Ostrovsky wrote: >> On 11/03/2017 02:23 PM, Juergen Gross wrote: >>> On 03/11/17 19:19, Boris Ostrovsky wrote: On 11/03/2017 02:05 PM, Juergen Gross wrote: > So again the question: how to tell whether we are PVH or HVM in > init_hypervisor_platform()? ACPi tables are scanned way later... Can we make grub/OVMF append a boot option? Or set setup_header.hardware_subarch to something? We already have X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use hardware_subarch_data (will need to get a buy-in from x86 maintainers, I think). >>> But wouldn't this break the idea to reuse the native boot paths in >>> grub/OVMF without further modifications? >> WDYM? We will have to have some sort of a plugin in either one to build >> the zeropage anyway. So we'd set hardware_subarch there, in addition to >> other things like setting memory and such. > But isn't the zeropage already being built? I admit that setting subarch > isn't a big deal, but using another entry with a passed-through pvh > start struct isn't either... I don't follow, sorry. My understanding is that zeropage will be built by PVH-enlightened grub so part of this process would be setting the subarch bit. -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 19:37, Roger Pau Monné wrote: > On Fri, Nov 03, 2017 at 07:23:50PM +0100, Juergen Gross wrote: >> On 03/11/17 19:19, Boris Ostrovsky wrote: >>> On 11/03/2017 02:05 PM, Juergen Gross wrote: So again the question: how to tell whether we are PVH or HVM in init_hypervisor_platform()? ACPi tables are scanned way later... >>> >>> Can we make grub/OVMF append a boot option? >>> >>> Or set setup_header.hardware_subarch to something? We already have >>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use >>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I >>> think). >> >> But wouldn't this break the idea to reuse the native boot paths in >> grub/OVMF without further modifications? >> >> I'd rather have a way to ask the hypervisor whether we are in PVH mode >> (e.g. via CPUID or a hypercall to test for a devicemodel having itself >> registered). > > Keep in mind that from Xen's PoV PVH is just a HVM guest. Xen > currently keeps a bitmap of the emulated devices that are available to > a guest, but that's so far an internal field. We could consider > exporting this on a cpuid leaf, but then we need to make it a fixed > ABI. > > Maybe we can make a list of platform devices that are not available on > PVH and that Linux assumes to be present? The main problem in Linux is that we have to know whether we are a HVM or a PVH guest. Okay, we could scan for the Xen PCI-device to make the distinction, but this is again rather late in the boot process. Using implicit assumptions is normally a bad way to make such a decision. I'd rather be either told by the boot loader I'm running as PVH guest (and the boot loader does know it), or I want to ask the hypervisor (which right now doesn't know), or I need another clear distinction like the existence of the Xen PCI-device (which might be a problem in the future for a PVH dom0 in a nested Xen environment). Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 19:35, Boris Ostrovsky wrote: > On 11/03/2017 02:23 PM, Juergen Gross wrote: >> On 03/11/17 19:19, Boris Ostrovsky wrote: >>> On 11/03/2017 02:05 PM, Juergen Gross wrote: So again the question: how to tell whether we are PVH or HVM in init_hypervisor_platform()? ACPi tables are scanned way later... >>> Can we make grub/OVMF append a boot option? >>> >>> Or set setup_header.hardware_subarch to something? We already have >>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use >>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I >>> think). >> But wouldn't this break the idea to reuse the native boot paths in >> grub/OVMF without further modifications? > > WDYM? We will have to have some sort of a plugin in either one to build > the zeropage anyway. So we'd set hardware_subarch there, in addition to > other things like setting memory and such. But isn't the zeropage already being built? I admit that setting subarch isn't a big deal, but using another entry with a passed-through pvh start struct isn't either... Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Nov 03, 2017 at 07:23:50PM +0100, Juergen Gross wrote: > On 03/11/17 19:19, Boris Ostrovsky wrote: > > On 11/03/2017 02:05 PM, Juergen Gross wrote: > >> > >> So again the question: how to tell whether we are PVH or HVM in > >> init_hypervisor_platform()? ACPi tables are scanned way later... > > > > Can we make grub/OVMF append a boot option? > > > > Or set setup_header.hardware_subarch to something? We already have > > X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use > > hardware_subarch_data (will need to get a buy-in from x86 maintainers, I > > think). > > But wouldn't this break the idea to reuse the native boot paths in > grub/OVMF without further modifications? > > I'd rather have a way to ask the hypervisor whether we are in PVH mode > (e.g. via CPUID or a hypercall to test for a devicemodel having itself > registered). Keep in mind that from Xen's PoV PVH is just a HVM guest. Xen currently keeps a bitmap of the emulated devices that are available to a guest, but that's so far an internal field. We could consider exporting this on a cpuid leaf, but then we need to make it a fixed ABI. Maybe we can make a list of platform devices that are not available on PVH and that Linux assumes to be present? Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/03/2017 02:23 PM, Juergen Gross wrote: > On 03/11/17 19:19, Boris Ostrovsky wrote: >> On 11/03/2017 02:05 PM, Juergen Gross wrote: >>> So again the question: how to tell whether we are PVH or HVM in >>> init_hypervisor_platform()? ACPi tables are scanned way later... >> Can we make grub/OVMF append a boot option? >> >> Or set setup_header.hardware_subarch to something? We already have >> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use >> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I >> think). > But wouldn't this break the idea to reuse the native boot paths in > grub/OVMF without further modifications? WDYM? We will have to have some sort of a plugin in either one to build the zeropage anyway. So we'd set hardware_subarch there, in addition to other things like setting memory and such. -boris > > I'd rather have a way to ask the hypervisor whether we are in PVH mode > (e.g. via CPUID or a hypercall to test for a devicemodel having itself > registered). > > In case we need to set an indicator in zeropage I'd opt for reusing > X86_SUBARCH_XEN. > > > Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 19:19, Boris Ostrovsky wrote: > On 11/03/2017 02:05 PM, Juergen Gross wrote: >> >> So again the question: how to tell whether we are PVH or HVM in >> init_hypervisor_platform()? ACPi tables are scanned way later... > > Can we make grub/OVMF append a boot option? > > Or set setup_header.hardware_subarch to something? We already have > X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use > hardware_subarch_data (will need to get a buy-in from x86 maintainers, I > think). But wouldn't this break the idea to reuse the native boot paths in grub/OVMF without further modifications? I'd rather have a way to ask the hypervisor whether we are in PVH mode (e.g. via CPUID or a hypercall to test for a devicemodel having itself registered). In case we need to set an indicator in zeropage I'd opt for reusing X86_SUBARCH_XEN. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/03/2017 02:05 PM, Juergen Gross wrote: > > So again the question: how to tell whether we are PVH or HVM in > init_hypervisor_platform()? ACPi tables are scanned way later... Can we make grub/OVMF append a boot option? Or set setup_header.hardware_subarch to something? We already have X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use hardware_subarch_data (will need to get a buy-in from x86 maintainers, I think). -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 16:27, Juergen Gross wrote: > On 03/11/17 16:10, Boris Ostrovsky wrote: >> On 11/03/2017 10:59 AM, Juergen Gross wrote: >>> On 03/11/17 15:36, Boris Ostrovsky wrote: On 11/03/2017 10:24 AM, Juergen Gross wrote: > On 03/11/17 15:07, Roger Pau Monné wrote: >> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: >>> On 03/11/17 13:17, Roger Pau Monné wrote: On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: > On 29/09/17 17:51, Roger Pau Monné wrote: >> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: >>> On 29/09/17 17:24, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: Then, I also wonder whether it would make sense for this grub to load the kernel using the PVH entry point or the native entry point. Would it be possible to boot a Linux kernel up to the point where cpuid can be used inside of a PVH container? >>> I don't think today's Linux allows that. This has been discussed >>> very thoroughly at the time Boris added PVH V2 support to the >>> kernel. >> OK, I'm not going to insist on that, but my plans for FreeBSD is to >> make the native entry point capable of booting inside of a PVH >> container up to the point where cpuid (or whatever method) can be >> used >> to detect the environment. > Looking more thoroughly into the Linux boot code I think this could > work for Linux, too. But only if we can tell PVH from HVM in the > guest. > How would you do that in FreeBSD? Via flags in the boot params? This > would the have to be done in the boot loader (e.g. grub or OVMF). My plan was not to differentiate between HVM and PVH, but rather to make use of the ACPI information in order to decide which devices are available and which are not inside of a PVH guest. For example in the FADT "IA-PC Boot Architecture Flags" field for PVH we already set "VGA Not Present" and "CMOS RTC Not Present". There might be other flags/fields that must be set, but I would like to avoid having a CPUID bit or similar saying "PVH", because then Xen will be tied to always providing the same set of devices in PVH containers. >>> Why? This would depend on the semantics tied to the flag. It could just >>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls). >>> >>> Linux would have a problem with the ACPI approach as it would try BIOS >>> calls way before it is initializing its ACPI handling. So in Linux I'd >>> need another way to tell I'm running in PVH mode, e.g. a "no legacy" >>> bit in the Xen HVM cpuid leaf. >> If you are booted from the PVH entry point, there's no BIOS or UEFI >> (ie: no firmware), if you are booted from the BIOS entry point there's >> a BIOS and the same applies to UEFI. How does Linux differentiate >> whether it's booted from BIOS or UEFI? > They use different entries. In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to see if we can use EFI entry point to also be able to boot PVH guest but found some issues with that approach, which is why we ended up with a dedicated PVH entry point. I am curious though, Juergen --- what do we need besides zeropage to allow us to boot PVH from startup_64? >>> Oh, you are right. I managed to get lost in the early boot paths. >>> >>> Only setting up the hyperpage seems to be missing, but this should be >>> doable. And setting xen_pvh, of course. >> >> That last part was actually my question --- do we need to have xen_pvh >> set before we get to xen-specific code for the first time (which I think >> is init_hypervisor_platform()) from startup_64? >> >> Because if we do --- who will set it? > > This should be easily testable: Just set an internal indicator in > xen_prepare_pvh() and copy that to xen_pvh in init_hypervisor_platform() > and see whether the kernel still comes up in PVH mode. > > The same can be done with setting up the hypercall page. > > Just building a kernel trying that... Okay, setting xen_pvh late is working. I can't omit setting up the hypercall_page right now, of course, as then I wouldn't get the memory map. But this shouldn't be a problem for the grub/OVMF case as those would setup the memory map for the guest in zeropage. So again the question: how to tell whether we are PVH or HVM in init_hypervisor_platform()? ACPi tables are scanned way later... Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 16:10, Boris Ostrovsky wrote: > On 11/03/2017 10:59 AM, Juergen Gross wrote: >> On 03/11/17 15:36, Boris Ostrovsky wrote: >>> On 11/03/2017 10:24 AM, Juergen Gross wrote: On 03/11/17 15:07, Roger Pau Monné wrote: > On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: >> On 03/11/17 13:17, Roger Pau Monné wrote: >>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: On 29/09/17 17:51, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: >> On 29/09/17 17:24, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >>> Then, I also wonder whether it would make sense for this grub to >>> load >>> the kernel using the PVH entry point or the native entry point. >>> Would >>> it be possible to boot a Linux kernel up to the point where cpuid >>> can >>> be used inside of a PVH container? >> I don't think today's Linux allows that. This has been discussed >> very thoroughly at the time Boris added PVH V2 support to the kernel. > OK, I'm not going to insist on that, but my plans for FreeBSD is to > make the native entry point capable of booting inside of a PVH > container up to the point where cpuid (or whatever method) can be used > to detect the environment. Looking more thoroughly into the Linux boot code I think this could work for Linux, too. But only if we can tell PVH from HVM in the guest. How would you do that in FreeBSD? Via flags in the boot params? This would the have to be done in the boot loader (e.g. grub or OVMF). >>> My plan was not to differentiate between HVM and PVH, but rather to >>> make use of the ACPI information in order to decide which devices are >>> available and which are not inside of a PVH guest. >>> >>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH >>> we already set "VGA Not Present" and "CMOS RTC Not Present". There >>> might be other flags/fields that must be set, but I would like to >>> avoid having a CPUID bit or similar saying "PVH", because then Xen >>> will be tied to always providing the same set of devices in PVH >>> containers. >> Why? This would depend on the semantics tied to the flag. It could just >> mean "don't assume availability of legacy stuff" (e.g. BIOS calls). >> >> Linux would have a problem with the ACPI approach as it would try BIOS >> calls way before it is initializing its ACPI handling. So in Linux I'd >> need another way to tell I'm running in PVH mode, e.g. a "no legacy" >> bit in the Xen HVM cpuid leaf. > If you are booted from the PVH entry point, there's no BIOS or UEFI > (ie: no firmware), if you are booted from the BIOS entry point there's > a BIOS and the same applies to UEFI. How does Linux differentiate > whether it's booted from BIOS or UEFI? They use different entries. >>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to >>> see if we can use EFI entry point to also be able to boot PVH guest but >>> found some issues with that approach, which is why we ended up with a >>> dedicated PVH entry point. >>> >>> I am curious though, Juergen --- what do we need besides zeropage to >>> allow us to boot PVH from startup_64? >> Oh, you are right. I managed to get lost in the early boot paths. >> >> Only setting up the hyperpage seems to be missing, but this should be >> doable. And setting xen_pvh, of course. > > That last part was actually my question --- do we need to have xen_pvh > set before we get to xen-specific code for the first time (which I think > is init_hypervisor_platform()) from startup_64? > > Because if we do --- who will set it? This should be easily testable: Just set an internal indicator in xen_prepare_pvh() and copy that to xen_pvh in init_hypervisor_platform() and see whether the kernel still comes up in PVH mode. The same can be done with setting up the hypercall page. Just building a kernel trying that... Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/03/2017 10:59 AM, Juergen Gross wrote: > On 03/11/17 15:36, Boris Ostrovsky wrote: >> On 11/03/2017 10:24 AM, Juergen Gross wrote: >>> On 03/11/17 15:07, Roger Pau Monné wrote: On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: > On 03/11/17 13:17, Roger Pau Monné wrote: >> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: >>> On 29/09/17 17:51, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > On 29/09/17 17:24, Roger Pau Monné wrote: >> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >> Then, I also wonder whether it would make sense for this grub to load >> the kernel using the PVH entry point or the native entry point. Would >> it be possible to boot a Linux kernel up to the point where cpuid can >> be used inside of a PVH container? > I don't think today's Linux allows that. This has been discussed > very thoroughly at the time Boris added PVH V2 support to the kernel. OK, I'm not going to insist on that, but my plans for FreeBSD is to make the native entry point capable of booting inside of a PVH container up to the point where cpuid (or whatever method) can be used to detect the environment. >>> Looking more thoroughly into the Linux boot code I think this could >>> work for Linux, too. But only if we can tell PVH from HVM in the guest. >>> How would you do that in FreeBSD? Via flags in the boot params? This >>> would the have to be done in the boot loader (e.g. grub or OVMF). >> My plan was not to differentiate between HVM and PVH, but rather to >> make use of the ACPI information in order to decide which devices are >> available and which are not inside of a PVH guest. >> >> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH >> we already set "VGA Not Present" and "CMOS RTC Not Present". There >> might be other flags/fields that must be set, but I would like to >> avoid having a CPUID bit or similar saying "PVH", because then Xen >> will be tied to always providing the same set of devices in PVH >> containers. > Why? This would depend on the semantics tied to the flag. It could just > mean "don't assume availability of legacy stuff" (e.g. BIOS calls). > > Linux would have a problem with the ACPI approach as it would try BIOS > calls way before it is initializing its ACPI handling. So in Linux I'd > need another way to tell I'm running in PVH mode, e.g. a "no legacy" > bit in the Xen HVM cpuid leaf. If you are booted from the PVH entry point, there's no BIOS or UEFI (ie: no firmware), if you are booted from the BIOS entry point there's a BIOS and the same applies to UEFI. How does Linux differentiate whether it's booted from BIOS or UEFI? >>> They use different entries. >> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to >> see if we can use EFI entry point to also be able to boot PVH guest but >> found some issues with that approach, which is why we ended up with a >> dedicated PVH entry point. >> >> I am curious though, Juergen --- what do we need besides zeropage to >> allow us to boot PVH from startup_64? > Oh, you are right. I managed to get lost in the early boot paths. > > Only setting up the hyperpage seems to be missing, but this should be > doable. And setting xen_pvh, of course. That last part was actually my question --- do we need to have xen_pvh set before we get to xen-specific code for the first time (which I think is init_hypervisor_platform()) from startup_64? Because if we do --- who will set it? -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 15:36, Boris Ostrovsky wrote: > On 11/03/2017 10:24 AM, Juergen Gross wrote: >> On 03/11/17 15:07, Roger Pau Monné wrote: >>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: On 03/11/17 13:17, Roger Pau Monné wrote: > On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: >> On 29/09/17 17:51, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: On 29/09/17 17:24, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > Then, I also wonder whether it would make sense for this grub to load > the kernel using the PVH entry point or the native entry point. Would > it be possible to boot a Linux kernel up to the point where cpuid can > be used inside of a PVH container? I don't think today's Linux allows that. This has been discussed very thoroughly at the time Boris added PVH V2 support to the kernel. >>> OK, I'm not going to insist on that, but my plans for FreeBSD is to >>> make the native entry point capable of booting inside of a PVH >>> container up to the point where cpuid (or whatever method) can be used >>> to detect the environment. >> Looking more thoroughly into the Linux boot code I think this could >> work for Linux, too. But only if we can tell PVH from HVM in the guest. >> How would you do that in FreeBSD? Via flags in the boot params? This >> would the have to be done in the boot loader (e.g. grub or OVMF). > My plan was not to differentiate between HVM and PVH, but rather to > make use of the ACPI information in order to decide which devices are > available and which are not inside of a PVH guest. > > For example in the FADT "IA-PC Boot Architecture Flags" field for PVH > we already set "VGA Not Present" and "CMOS RTC Not Present". There > might be other flags/fields that must be set, but I would like to > avoid having a CPUID bit or similar saying "PVH", because then Xen > will be tied to always providing the same set of devices in PVH > containers. Why? This would depend on the semantics tied to the flag. It could just mean "don't assume availability of legacy stuff" (e.g. BIOS calls). Linux would have a problem with the ACPI approach as it would try BIOS calls way before it is initializing its ACPI handling. So in Linux I'd need another way to tell I'm running in PVH mode, e.g. a "no legacy" bit in the Xen HVM cpuid leaf. >>> If you are booted from the PVH entry point, there's no BIOS or UEFI >>> (ie: no firmware), if you are booted from the BIOS entry point there's >>> a BIOS and the same applies to UEFI. How does Linux differentiate >>> whether it's booted from BIOS or UEFI? >> They use different entries. > > In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to > see if we can use EFI entry point to also be able to boot PVH guest but > found some issues with that approach, which is why we ended up with a > dedicated PVH entry point. > > I am curious though, Juergen --- what do we need besides zeropage to > allow us to boot PVH from startup_64? Oh, you are right. I managed to get lost in the early boot paths. Only setting up the hyperpage seems to be missing, but this should be doable. And setting xen_pvh, of course. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 11/03/2017 10:24 AM, Juergen Gross wrote: > On 03/11/17 15:07, Roger Pau Monné wrote: >> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: >>> On 03/11/17 13:17, Roger Pau Monné wrote: On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: > On 29/09/17 17:51, Roger Pau Monné wrote: >> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: >>> On 29/09/17 17:24, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: Then, I also wonder whether it would make sense for this grub to load the kernel using the PVH entry point or the native entry point. Would it be possible to boot a Linux kernel up to the point where cpuid can be used inside of a PVH container? >>> I don't think today's Linux allows that. This has been discussed >>> very thoroughly at the time Boris added PVH V2 support to the kernel. >> OK, I'm not going to insist on that, but my plans for FreeBSD is to >> make the native entry point capable of booting inside of a PVH >> container up to the point where cpuid (or whatever method) can be used >> to detect the environment. > Looking more thoroughly into the Linux boot code I think this could > work for Linux, too. But only if we can tell PVH from HVM in the guest. > How would you do that in FreeBSD? Via flags in the boot params? This > would the have to be done in the boot loader (e.g. grub or OVMF). My plan was not to differentiate between HVM and PVH, but rather to make use of the ACPI information in order to decide which devices are available and which are not inside of a PVH guest. For example in the FADT "IA-PC Boot Architecture Flags" field for PVH we already set "VGA Not Present" and "CMOS RTC Not Present". There might be other flags/fields that must be set, but I would like to avoid having a CPUID bit or similar saying "PVH", because then Xen will be tied to always providing the same set of devices in PVH containers. >>> Why? This would depend on the semantics tied to the flag. It could just >>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls). >>> >>> Linux would have a problem with the ACPI approach as it would try BIOS >>> calls way before it is initializing its ACPI handling. So in Linux I'd >>> need another way to tell I'm running in PVH mode, e.g. a "no legacy" >>> bit in the Xen HVM cpuid leaf. >> If you are booted from the PVH entry point, there's no BIOS or UEFI >> (ie: no firmware), if you are booted from the BIOS entry point there's >> a BIOS and the same applies to UEFI. How does Linux differentiate >> whether it's booted from BIOS or UEFI? > They use different entries. In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to see if we can use EFI entry point to also be able to boot PVH guest but found some issues with that approach, which is why we ended up with a dedicated PVH entry point. I am curious though, Juergen --- what do we need besides zeropage to allow us to boot PVH from startup_64? -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 15:07, Roger Pau Monné wrote: > On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: >> On 03/11/17 13:17, Roger Pau Monné wrote: >>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: On 29/09/17 17:51, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: >> On 29/09/17 17:24, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >>> Then, I also wonder whether it would make sense for this grub to load >>> the kernel using the PVH entry point or the native entry point. Would >>> it be possible to boot a Linux kernel up to the point where cpuid can >>> be used inside of a PVH container? >> >> I don't think today's Linux allows that. This has been discussed >> very thoroughly at the time Boris added PVH V2 support to the kernel. > > OK, I'm not going to insist on that, but my plans for FreeBSD is to > make the native entry point capable of booting inside of a PVH > container up to the point where cpuid (or whatever method) can be used > to detect the environment. Looking more thoroughly into the Linux boot code I think this could work for Linux, too. But only if we can tell PVH from HVM in the guest. How would you do that in FreeBSD? Via flags in the boot params? This would the have to be done in the boot loader (e.g. grub or OVMF). >>> >>> My plan was not to differentiate between HVM and PVH, but rather to >>> make use of the ACPI information in order to decide which devices are >>> available and which are not inside of a PVH guest. >>> >>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH >>> we already set "VGA Not Present" and "CMOS RTC Not Present". There >>> might be other flags/fields that must be set, but I would like to >>> avoid having a CPUID bit or similar saying "PVH", because then Xen >>> will be tied to always providing the same set of devices in PVH >>> containers. >> >> Why? This would depend on the semantics tied to the flag. It could just >> mean "don't assume availability of legacy stuff" (e.g. BIOS calls). >> >> Linux would have a problem with the ACPI approach as it would try BIOS >> calls way before it is initializing its ACPI handling. So in Linux I'd >> need another way to tell I'm running in PVH mode, e.g. a "no legacy" >> bit in the Xen HVM cpuid leaf. > > If you are booted from the PVH entry point, there's no BIOS or UEFI > (ie: no firmware), if you are booted from the BIOS entry point there's > a BIOS and the same applies to UEFI. How does Linux differentiate > whether it's booted from BIOS or UEFI? They use different entries. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote: > On 03/11/17 13:17, Roger Pau Monné wrote: > > On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: > >> On 29/09/17 17:51, Roger Pau Monné wrote: > >>> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > On 29/09/17 17:24, Roger Pau Monné wrote: > > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > > Then, I also wonder whether it would make sense for this grub to load > > the kernel using the PVH entry point or the native entry point. Would > > it be possible to boot a Linux kernel up to the point where cpuid can > > be used inside of a PVH container? > > I don't think today's Linux allows that. This has been discussed > very thoroughly at the time Boris added PVH V2 support to the kernel. > >>> > >>> OK, I'm not going to insist on that, but my plans for FreeBSD is to > >>> make the native entry point capable of booting inside of a PVH > >>> container up to the point where cpuid (or whatever method) can be used > >>> to detect the environment. > >> > >> Looking more thoroughly into the Linux boot code I think this could > >> work for Linux, too. But only if we can tell PVH from HVM in the guest. > >> How would you do that in FreeBSD? Via flags in the boot params? This > >> would the have to be done in the boot loader (e.g. grub or OVMF). > > > > My plan was not to differentiate between HVM and PVH, but rather to > > make use of the ACPI information in order to decide which devices are > > available and which are not inside of a PVH guest. > > > > For example in the FADT "IA-PC Boot Architecture Flags" field for PVH > > we already set "VGA Not Present" and "CMOS RTC Not Present". There > > might be other flags/fields that must be set, but I would like to > > avoid having a CPUID bit or similar saying "PVH", because then Xen > > will be tied to always providing the same set of devices in PVH > > containers. > > Why? This would depend on the semantics tied to the flag. It could just > mean "don't assume availability of legacy stuff" (e.g. BIOS calls). > > Linux would have a problem with the ACPI approach as it would try BIOS > calls way before it is initializing its ACPI handling. So in Linux I'd > need another way to tell I'm running in PVH mode, e.g. a "no legacy" > bit in the Xen HVM cpuid leaf. If you are booted from the PVH entry point, there's no BIOS or UEFI (ie: no firmware), if you are booted from the BIOS entry point there's a BIOS and the same applies to UEFI. How does Linux differentiate whether it's booted from BIOS or UEFI? Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 03/11/17 13:17, Roger Pau Monné wrote: > On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: >> On 29/09/17 17:51, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: On 29/09/17 17:24, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > Then, I also wonder whether it would make sense for this grub to load > the kernel using the PVH entry point or the native entry point. Would > it be possible to boot a Linux kernel up to the point where cpuid can > be used inside of a PVH container? I don't think today's Linux allows that. This has been discussed very thoroughly at the time Boris added PVH V2 support to the kernel. >>> >>> OK, I'm not going to insist on that, but my plans for FreeBSD is to >>> make the native entry point capable of booting inside of a PVH >>> container up to the point where cpuid (or whatever method) can be used >>> to detect the environment. >> >> Looking more thoroughly into the Linux boot code I think this could >> work for Linux, too. But only if we can tell PVH from HVM in the guest. >> How would you do that in FreeBSD? Via flags in the boot params? This >> would the have to be done in the boot loader (e.g. grub or OVMF). > > My plan was not to differentiate between HVM and PVH, but rather to > make use of the ACPI information in order to decide which devices are > available and which are not inside of a PVH guest. > > For example in the FADT "IA-PC Boot Architecture Flags" field for PVH > we already set "VGA Not Present" and "CMOS RTC Not Present". There > might be other flags/fields that must be set, but I would like to > avoid having a CPUID bit or similar saying "PVH", because then Xen > will be tied to always providing the same set of devices in PVH > containers. Why? This would depend on the semantics tied to the flag. It could just mean "don't assume availability of legacy stuff" (e.g. BIOS calls). Linux would have a problem with the ACPI approach as it would try BIOS calls way before it is initializing its ACPI handling. So in Linux I'd need another way to tell I'm running in PVH mode, e.g. a "no legacy" bit in the Xen HVM cpuid leaf. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote: > On 29/09/17 17:51, Roger Pau Monné wrote: > > On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > >> On 29/09/17 17:24, Roger Pau Monné wrote: > >>> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > >>> Then, I also wonder whether it would make sense for this grub to load > >>> the kernel using the PVH entry point or the native entry point. Would > >>> it be possible to boot a Linux kernel up to the point where cpuid can > >>> be used inside of a PVH container? > >> > >> I don't think today's Linux allows that. This has been discussed > >> very thoroughly at the time Boris added PVH V2 support to the kernel. > > > > OK, I'm not going to insist on that, but my plans for FreeBSD is to > > make the native entry point capable of booting inside of a PVH > > container up to the point where cpuid (or whatever method) can be used > > to detect the environment. > > Looking more thoroughly into the Linux boot code I think this could > work for Linux, too. But only if we can tell PVH from HVM in the guest. > How would you do that in FreeBSD? Via flags in the boot params? This > would the have to be done in the boot loader (e.g. grub or OVMF). My plan was not to differentiate between HVM and PVH, but rather to make use of the ACPI information in order to decide which devices are available and which are not inside of a PVH guest. For example in the FADT "IA-PC Boot Architecture Flags" field for PVH we already set "VGA Not Present" and "CMOS RTC Not Present". There might be other flags/fields that must be set, but I would like to avoid having a CPUID bit or similar saying "PVH", because then Xen will be tied to always providing the same set of devices in PVH containers. Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 29/09/17 17:51, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: >> On 29/09/17 17:24, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >>> Then, I also wonder whether it would make sense for this grub to load >>> the kernel using the PVH entry point or the native entry point. Would >>> it be possible to boot a Linux kernel up to the point where cpuid can >>> be used inside of a PVH container? >> >> I don't think today's Linux allows that. This has been discussed >> very thoroughly at the time Boris added PVH V2 support to the kernel. > > OK, I'm not going to insist on that, but my plans for FreeBSD is to > make the native entry point capable of booting inside of a PVH > container up to the point where cpuid (or whatever method) can be used > to detect the environment. Looking more thoroughly into the Linux boot code I think this could work for Linux, too. But only if we can tell PVH from HVM in the guest. How would you do that in FreeBSD? Via flags in the boot params? This would the have to be done in the boot loader (e.g. grub or OVMF). Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 10/03/2017 04:56 AM, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 05:22:25PM +, Boris Ostrovsky wrote: On 09/29/2017 01:07 PM, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 05:02:48PM +, Boris Ostrovsky wrote: On 09/29/2017 11:51 AM, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: On 29/09/17 17:24, Roger Pau Monné wrote: On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: Then, I also wonder whether it would make sense for this grub to load the kernel using the PVH entry point or the native entry point. Would it be possible to boot a Linux kernel up to the point where cpuid can be used inside of a PVH container? I don't think today's Linux allows that. This has been discussed very thoroughly at the time Boris added PVH V2 support to the kernel. OK, I'm not going to insist on that, but my plans for FreeBSD is to make the native entry point capable of booting inside of a PVH container up to the point where cpuid (or whatever method) can be used to detect the environment. Do you recall what's preventing the native entry point from booting inside of a PVH container? If certain emulated devices not present are needed at early boot we could look into either replacing them with other options available inside of a PVH container, or as a last resort making them available on a PVH container. Very much IIRC one of the reasons was the fact that zeropage (bootparams) needed to be properly formatted. And the hypercall page needs to be set up. But in this case bootparams is going to be setup by grub, so it should be fine (just like it's done on bare metal). Yes, I think so. Couldn't the hypercall page be setup at some point during early boot? Not sure if setting it up at the same point HVM does would be fine for PVH? Probably --- I think the only reason we set it early is because we need to call XENMEM_memory_map to set bootparams. One other thing I noticed is that we need to set acpi_irq_model before hypervisor is discovered (can't remember why) but I suppose this can be worked around. Having said that --- since for direct boot we still need to go via PVH-specific entry point I am not sure how much we will gain by having grub avoid it. Being able to boot inside of a PVH container using the native entry point would prevent us from having to add PVH loader specific code to each firmware we plan to support in PVH mode. If Linux must be started using the PVH entry point in order to run inside of a PVH container, it means we would have to add a PVH loader to OVFM and grub at least. OTOH if Linux is capable of booting from the native entry point inside of a PVH container, we would only have to port OVMF and grub in order to work inside of a PVH container, leaving the rest of the logic untouched. Right, I understand that. I was simply trying to say that PVH-specific entry point is likely to stay for us to continue booting PVH guests directly. -boris -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Sep 29, 2017 at 05:22:25PM +, Boris Ostrovsky wrote: > On 09/29/2017 01:07 PM, Roger Pau Monné wrote: > > On Fri, Sep 29, 2017 at 05:02:48PM +, Boris Ostrovsky wrote: > >> On 09/29/2017 11:51 AM, Roger Pau Monné wrote: > >>> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > On 29/09/17 17:24, Roger Pau Monné wrote: > > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > > Then, I also wonder whether it would make sense for this grub to load > > the kernel using the PVH entry point or the native entry point. Would > > it be possible to boot a Linux kernel up to the point where cpuid can > > be used inside of a PVH container? > I don't think today's Linux allows that. This has been discussed > very thoroughly at the time Boris added PVH V2 support to the kernel. > >>> OK, I'm not going to insist on that, but my plans for FreeBSD is to > >>> make the native entry point capable of booting inside of a PVH > >>> container up to the point where cpuid (or whatever method) can be used > >>> to detect the environment. > >>> > >>> Do you recall what's preventing the native entry point from booting > >>> inside of a PVH container? If certain emulated devices not present are > >>> needed at early boot we could look into either replacing them with > >>> other options available inside of a PVH container, or as a last resort > >>> making them available on a PVH container. > >> Very much IIRC one of the reasons was the fact that zeropage > >> (bootparams) needed to be properly formatted. And the hypercall page > >> needs to be set up. > > But in this case bootparams is going to be setup by grub, so it should > > be fine (just like it's done on bare metal). > > Yes, I think so. > > > > > Couldn't the hypercall page be setup at some point during early boot? > > Not sure if setting it up at the same point HVM does would be fine for > > PVH? > > Probably --- I think the only reason we set it early is because we need > to call XENMEM_memory_map to set bootparams. One other thing I noticed > is that we need to set acpi_irq_model before hypervisor is discovered > (can't remember why) but I suppose this can be worked around. > > Having said that --- since for direct boot we still need to go via > PVH-specific entry point I am not sure how much we will gain by having > grub avoid it. Being able to boot inside of a PVH container using the native entry point would prevent us from having to add PVH loader specific code to each firmware we plan to support in PVH mode. If Linux must be started using the PVH entry point in order to run inside of a PVH container, it means we would have to add a PVH loader to OVFM and grub at least. OTOH if Linux is capable of booting from the native entry point inside of a PVH container, we would only have to port OVMF and grub in order to work inside of a PVH container, leaving the rest of the logic untouched. Thanks, Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 09/29/2017 01:07 PM, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 05:02:48PM +, Boris Ostrovsky wrote: >> On 09/29/2017 11:51 AM, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: On 29/09/17 17:24, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > Then, I also wonder whether it would make sense for this grub to load > the kernel using the PVH entry point or the native entry point. Would > it be possible to boot a Linux kernel up to the point where cpuid can > be used inside of a PVH container? I don't think today's Linux allows that. This has been discussed very thoroughly at the time Boris added PVH V2 support to the kernel. >>> OK, I'm not going to insist on that, but my plans for FreeBSD is to >>> make the native entry point capable of booting inside of a PVH >>> container up to the point where cpuid (or whatever method) can be used >>> to detect the environment. >>> >>> Do you recall what's preventing the native entry point from booting >>> inside of a PVH container? If certain emulated devices not present are >>> needed at early boot we could look into either replacing them with >>> other options available inside of a PVH container, or as a last resort >>> making them available on a PVH container. >> Very much IIRC one of the reasons was the fact that zeropage >> (bootparams) needed to be properly formatted. And the hypercall page >> needs to be set up. > But in this case bootparams is going to be setup by grub, so it should > be fine (just like it's done on bare metal). Yes, I think so. > > Couldn't the hypercall page be setup at some point during early boot? > Not sure if setting it up at the same point HVM does would be fine for > PVH? Probably --- I think the only reason we set it early is because we need to call XENMEM_memory_map to set bootparams. One other thing I noticed is that we need to set acpi_irq_model before hypervisor is discovered (can't remember why) but I suppose this can be worked around. Having said that --- since for direct boot we still need to go via PVH-specific entry point I am not sure how much we will gain by having grub avoid it. -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Sep 29, 2017 at 05:02:48PM +, Boris Ostrovsky wrote: > On 09/29/2017 11:51 AM, Roger Pau Monné wrote: > > On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > >> On 29/09/17 17:24, Roger Pau Monné wrote: > >>> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > >>> Then, I also wonder whether it would make sense for this grub to load > >>> the kernel using the PVH entry point or the native entry point. Would > >>> it be possible to boot a Linux kernel up to the point where cpuid can > >>> be used inside of a PVH container? > >> I don't think today's Linux allows that. This has been discussed > >> very thoroughly at the time Boris added PVH V2 support to the kernel. > > OK, I'm not going to insist on that, but my plans for FreeBSD is to > > make the native entry point capable of booting inside of a PVH > > container up to the point where cpuid (or whatever method) can be used > > to detect the environment. > > > > Do you recall what's preventing the native entry point from booting > > inside of a PVH container? If certain emulated devices not present are > > needed at early boot we could look into either replacing them with > > other options available inside of a PVH container, or as a last resort > > making them available on a PVH container. > > Very much IIRC one of the reasons was the fact that zeropage > (bootparams) needed to be properly formatted. And the hypercall page > needs to be set up. But in this case bootparams is going to be setup by grub, so it should be fine (just like it's done on bare metal). Couldn't the hypercall page be setup at some point during early boot? Not sure if setting it up at the same point HVM does would be fine for PVH? Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 09/29/2017 11:51 AM, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: >> On 29/09/17 17:24, Roger Pau Monné wrote: >>> On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >>> Then, I also wonder whether it would make sense for this grub to load >>> the kernel using the PVH entry point or the native entry point. Would >>> it be possible to boot a Linux kernel up to the point where cpuid can >>> be used inside of a PVH container? >> I don't think today's Linux allows that. This has been discussed >> very thoroughly at the time Boris added PVH V2 support to the kernel. > OK, I'm not going to insist on that, but my plans for FreeBSD is to > make the native entry point capable of booting inside of a PVH > container up to the point where cpuid (or whatever method) can be used > to detect the environment. > > Do you recall what's preventing the native entry point from booting > inside of a PVH container? If certain emulated devices not present are > needed at early boot we could look into either replacing them with > other options available inside of a PVH container, or as a last resort > making them available on a PVH container. Very much IIRC one of the reasons was the fact that zeropage (bootparams) needed to be properly formatted. And the hypercall page needs to be set up. -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Sep 29, 2017 at 03:33:58PM +, Juergen Gross wrote: > On 29/09/17 17:24, Roger Pau Monné wrote: > > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > > Then, I also wonder whether it would make sense for this grub to load > > the kernel using the PVH entry point or the native entry point. Would > > it be possible to boot a Linux kernel up to the point where cpuid can > > be used inside of a PVH container? > > I don't think today's Linux allows that. This has been discussed > very thoroughly at the time Boris added PVH V2 support to the kernel. OK, I'm not going to insist on that, but my plans for FreeBSD is to make the native entry point capable of booting inside of a PVH container up to the point where cpuid (or whatever method) can be used to detect the environment. Do you recall what's preventing the native entry point from booting inside of a PVH container? If certain emulated devices not present are needed at early boot we could look into either replacing them with other options available inside of a PVH container, or as a last resort making them available on a PVH container. Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Sep 29, 2017 at 4:24 PM, Roger Pau Monnéwrote: > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >> I'm thinking about to add support for Xen PVH guests to grub2. >> >> Basically I see two options how to do it: >> >> a) add PVH support to current grub.xen (both, 32- and 64-bit), in >>order to use the same grub binary for either pv-domains or >>pvh-domains >> b) create a new variant grub.xenpvh capable only to boot pvh-domains >> >> Are there any preferences which way to go? > > IMHO, it should be possible to create a grub binary that can be loaded > in PVH mode but that's not tied to a specific bitness. Let's avoid > having a grub32 and grub64, that's not needed at all for PVH. Oh, right -- yeah, a single binary that can do both is definitely worth having a separate binary from the PV one. -george ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On 29/09/17 17:24, Roger Pau Monné wrote: > On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: >> I'm thinking about to add support for Xen PVH guests to grub2. >> >> Basically I see two options how to do it: >> >> a) add PVH support to current grub.xen (both, 32- and 64-bit), in >>order to use the same grub binary for either pv-domains or >>pvh-domains >> b) create a new variant grub.xenpvh capable only to boot pvh-domains >> >> Are there any preferences which way to go? > > IMHO, it should be possible to create a grub binary that can be loaded > in PVH mode but that's not tied to a specific bitness. Let's avoid > having a grub32 and grub64, that's not needed at all for PVH. Right. OTOH we have those two variants already for PV. So the question is whether we want grub32 and grub64 or do we want grub32, grub64 and grub.pvh. > Then, I also wonder whether it would make sense for this grub to load > the kernel using the PVH entry point or the native entry point. Would > it be possible to boot a Linux kernel up to the point where cpuid can > be used inside of a PVH container? I don't think today's Linux allows that. This has been discussed very thoroughly at the time Boris added PVH V2 support to the kernel. So my plan was to use the PVH entry point of the kernel. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Sep 29, 2017 at 02:46:53PM +, Juergen Gross wrote: > I'm thinking about to add support for Xen PVH guests to grub2. > > Basically I see two options how to do it: > > a) add PVH support to current grub.xen (both, 32- and 64-bit), in >order to use the same grub binary for either pv-domains or >pvh-domains > b) create a new variant grub.xenpvh capable only to boot pvh-domains > > Are there any preferences which way to go? IMHO, it should be possible to create a grub binary that can be loaded in PVH mode but that's not tied to a specific bitness. Let's avoid having a grub32 and grub64, that's not needed at all for PVH. Then, I also wonder whether it would make sense for this grub to load the kernel using the PVH entry point or the native entry point. Would it be possible to boot a Linux kernel up to the point where cpuid can be used inside of a PVH container? Thanks, Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Xen PVH support in grub2
On Fri, Sep 29, 2017 at 3:46 PM, Juergen Grosswrote: > I'm thinking about to add support for Xen PVH guests to grub2. > > Basically I see two options how to do it: > > a) add PVH support to current grub.xen (both, 32- and 64-bit), in >order to use the same grub binary for either pv-domains or >pvh-domains > b) create a new variant grub.xenpvh capable only to boot pvh-domains > > Are there any preferences which way to go? Well from a user perspective, it's better to have fewer binaries, and from a builder / distro perspective it's better to have fewer builds. So I would only do 'b' if 'a' is significantly more complicated. -George ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel