On Tue, Mar 23, 2021 at 10:39:53AM -0700, Christopher Clark wrote: > On Thu, Mar 18, 2021 at 9:43 AM Roger Pau Monné <roger....@citrix.com> wrote: > > > > Just took a quick look at it. > > > > On Mon, Mar 15, 2021 at 11:18:13PM -0400, Daniel P. Smith wrote: > > > + > > > +---------------+-----------+------------+-----------+-------------+---------------------+ > > > + | **Xen Dom0** | **Linux** | **Late** | **Jail** | **Xen** | > > > **Xen Hyperlaunch** | > > > + | **(Classic)** | **KVM** | **HW Dom** | **house** | > > > **dom0less**+---------+-----------+ > > > + | | | | | | > > > Static | Dynamic | > > > + > > > +===============+===========+============+===========+=============+=========+===========+ > > > + | Hypervisor able to launch multiple VMs during host boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | Y | Y | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Hypervisor supports Static Partitioning > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | Y | Y | > > > Y | | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Able to launch VMs dynamically after host boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Y | Y | Y* | Y | Y* | > > > | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Supports strong isolation between all VMs started at host boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | Y | Y | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Enables flexible sequencing of VM start during host boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Prevent all-powerful static root domain being launched at boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | Y* | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Operates without a Highly-privileged management VM (eg. Dom0) > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | Y* | | Y* | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Operates without a privileged toolstack VM (Control Domain) > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | Y* | > > > Y | | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Extensible VM configuration applied before launch of VMs at host boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Flexible granular assignment of permissions and functions to VMs > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | Supports extensible VM measurement architecture for DRTM and > > > attestation | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | PCI passthrough configured at host boot > > > | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > + | | | | | | > > > Y | Y | > > > + > > > +---------------+-----------+------------+-----------+-------------+---------+-----------+ > > > > I'm curious about this, I assume this is done using vPCI so that > > there's no hardware domain (and user-space device model) involved for > > PCI passthrough? > > That would be an incorrect assumption. See below for why. > > > I'm also not sure how you are going to handle things like SR-IOV > > devices. Right now SR-IOV capability is setup and initialized by the > > hardware domain, and the new virtual devices are notified to Xen once > > setup is done. Do you plan to move those bits into Xen, so that it can > > setup and initialize the SR-IOV capability? > > While you could do it with the vPCI, as you point out this will not work > for SR-IOV. With hyperlaunch, these cases will require the use of a boot > domain, which is for all intents and purposes, a lightweight/restricted > toolstack domain. > > The boot domain will have to do the necessary operations to ensure that > when startup is finished, PCI passthrough will be successfully setup. > Note, this may have to include the boot domain unpausing the hardware > domain to help complete the setup before the boot domain can exit and > allow the remaining domains to come online.
OK, I was expecting hyperlaunch to do all domain creation in the hypervisor. If you offload domain creation of guests with pci-passthrough devices to a control domain and/or hardware domain, I'm not sure I see the difference from normal domain creation, ie: it's no longer something specific to hyperlaunch, as I could achieve the same by using the existing xendomains init script. Also you need a way to pass the configuration from the hypervisor into a control domain that would then wait for the hardware domain to come up and afterwards launch a guest with a pci-passthorugh device. The passing of this information from the hypervisor to the control domain would need to be done in an OS agnostic way if possible. Don't get me wrong, I don't think such approach is bad, I'm just unsure whether such functionality is really part of hyperlaunch, or instead something that you can achieve outside of hyperlaunch already. Thanks, Roger.
