On 7/8/23 14:08, Stefano Stabellini wrote:
> On Sat, 8 Jul 2023, Rich Persaud wrote:
>> On Jul 8, 2023, at 03:29, Luca Fancellu <luca.fance...@arm.com> wrote:
>>>
>>>>>>
>>>>>> Instead, the use case configurations should themselves be
describable.
>>>>>
>>>>> Thanks Christopher, Daniel and all!
>>>>>
>>>>> So if I understand correctly, you are in favor if renaming
Dom0less to
>>>>> Hyperlaunch throughout the Xen codebase? And we need a
clarification of
>>>>> the docs/, especially docs/features/dom0less.pandoc?
>>>>
>>>> Christopher wrote:
>>>>>> = Community resourcing
>>>>
>>>> Note the pre-requisite work items for upstream Xen, listed under
"Community Resourcing", to merge code for Hyperlaunch common interfaces
and test cases, with docs on configuration of Hyperlaunch to deliver
functionality for dom0less use cases.
>>>
>>> Are you saying that before renaming the “dom0less” feature, we
should wait for it to be ported to the common code?
>>
>> Why "wait"? In what timeframe do you expect dom0less to use
Hyperlaunch code?
>>
>> Can kernel component foo adopt the name of kernel component bar
without code change?
>>
>> Can dom0less stakeholders derive Hyperlaunch benefits without using
Hyperlaunch code?
>
>
> I think Rich is saying that before using the same name we should make
> sure that the interfaces and features are actually comparable and maybe
> even "compatible". I think that is very reasonable. Rich, did I
> understand correctly?
Essentially, yes this is what is being sought here. This does not mean
that the full capability has to be present for the adoption of the
common name, but that it can be accomplished through a path of integration.
> The Hyperlaunch (x86) code is not yet upstream, but the design document
> that describes the device tree interface shows an interface that is very
> similar, almost compatible, with today's dom0less (ARM) device tree
> interface.
I would caution the use of the current in-tree document as it is today,
it was posted under the design folder as it was only the design and not
burdened with the realities of implementation. Along the path of v1
implementation, recent PVH expansion, and roles update, each have
required updates to the design which are not yet included in the in-tree
docs.
To address, this the plan below starts with the documentation patch
posted in v1 of the hyperlaunch series:
https://lists.xenproject.org/archives/html/xen-devel/2022-07/msg00353.html
and will contain updates for community feedback received:
https://lists.xenproject.org/archives/html/xen-devel/2022-07/msg01015.html
and development work that has been done for PVH and roles.
> The structure of the device tree information is the same. Going through
> it I could only spot only tiny differences:
> - top level node is "hypervisor" instead of "chosen"
> - "module-addr" instead of "reg"
> - "module,kernel" instead of "multiboot,kernel"
> - "module,ramdisk" instead of "multiboot,ramdisk"
>
> The rest is the same. If we sort out these small differences one way or
> the other then the resulting interface should actually be fully
> compatible and we could reuse the existing Dom0less (ARM) code to parse
> an HyperLaunch (x86) configuration.
>
> The top level node is not a problem. We could easily deal with both
> "hypervisor" and also "chosen". Or we could pick a third different name
> for both: "domains" which is the one used by System Device Tree.
>
> I think we should rename "module-addr" to "reg" in the hyperlaunch
> design document. I don't think it would have any effect on the existing
> hyperlaunch (x86) code and usage because direct addresses are typically
> not used on x86.
>
> "module,kernel" and "module,ramdisk": we could either get rid of them in
> favor of "multiboot,kernel" and "multiboot,ramdisk", or we could add
> "module,kernel" and "module,ramdisk" as alternative aliases in the
> existing dom0less (ARM) code. We already have "xen,linux-zimage" and
> "xen,linux-initrd" as aliases so it is not a problem.
>
>
> Also, I do think that Dom0less stakeholders would benefit from
> Hyperlaunch code such as Dom0's reduction of privilege. Things like
> "permissions" and "functions" of the Hyperlauch device tree interface
> design document.
The roles work takes that to its final conclusion, from which everyone
will benefit.
> So, my opinion is that we should go ahead with dom0less->hyperlaunch
> rename but we should also try to make the two device tree interfaces
> compatible, sorting out the small differences above. That would help a
> lot in terms of documentation and tooling. It would be ideal if things
> like ImageBuilder worked equally well for Hyperlaunch (x86) and Dom0less
> (ARM).
Let me build on the plan laid out by Christopher, that detailed arriving
at the completion for hyperlaunch, and provide a set of steps to arrive
at a new short-term milestone to a larger roadmap. The intent being to
arrive at the immediate desire of the community to see dom0less renamed.
For the longer term, as Christopher was alluding, there is still a
significant amount of work to be done to deliver one of the biggest
market differentiating capabilities for Xen in some time.
As Stefano has acknowledged, hyperlaunch is a concept that is beyond
domain construction and that is, or will be, embodied by a set of
interfaces and capabilities. Considering dom0less as it is implemented
today, does not meet the former and in spirit meets the latter.
Compounding this is that dom0less is a supported feature today, with its
own defined interface, and a user base that is using that interface, at
least to some degree AIUI.
We have a responsibility to the dom0less user base and future
hyperlaunch users with requirements based on hyperlaunch design docs and
presentations. As such, any action taken should be done so under a
larger roadmap of adding the complete hyperlaunch capability to Xen.
With the initial funding by AMD, the first milestone was to be moving
Xen on to a common representation of boot material provided to the
hypervisor. As a result of this discussion, I would like to put forth a
new nearer term milestone of incorporating dom0less under hyperlaunch.
The naming suggestions by the community are greatly appreciated, and I
do not want to seem dismissive of clear offers of help and assistance.
This area is something Christopher and I discussed at length during the
drafting of the hyperlaunch design. Something we arrived at is that
there is only a single top level feature, which is hyperlaunch. The
hyperlaunch feature itself will provide multiple means to configure how
a launch will occur, one could even consider them modes of operation.
Reviewing the design doc, you will see that an initial attempt to
categorize them was done, splitting them into a dynamic mode and static
mode. Under these modes were multiple use cases. With that, we did not
mean to limit the hyperlaunch modes to strictly those two, and thus
other modes are more than reasonable to add. As such, my suggestion
would be the introduction of the dom0less compatibility mode for
hyperlaunch. Its very definition is a mode of hyperlaunch that supports
the "legacy" dom0less configuration interface. This approach allows
dom0less to effectively become hyperlaunch, deconflicts the fact that
hyperlaunch proper has a different interface than dom0less, and provides
a clean roadmap for migration to existing dom0less users.
To provide a concrete, measurable set of steps to achieve the dom0less
merging milestone, I will lay out the approach as three patch series
that will be a collaboration by the community. Each will need to be
submitted, reviewed and merged into the tree, with the end being the
existing dom0less becoming hyperlaunch's dom0less compatibility mode.
== Patch Series 1 (Resourced by Apertus)
The goal of this series is to properly introduce the hyperlaunch
"feature" in to Xen. This series would be submitted by myself and will
consist of two patches derived from the original hyperlaunch v1 series.
The first patch is the docs patch that updates the hyperlaunch device
tree design to reflect review feedback and updates for PVH and roles,
mentioned above.
The second patch will start with the v1 series patch that moved the fdt
parsing helpers into common:
https://lists.xenproject.org/archives/html/xen-devel/2022-07/msg00352.html
The patch currently moves common FDT parsing to common/fdt.c, it will be
updated to add this path to DEVICE TREE in MAINTAINERS. As part of
updating MAINTAINERS, there will be the addition of HYPERLAUNCH, which
will own common/domain-builder/ and doc/design/launch paths. As such,
this will effectively be a declaration of the top level hyperlaunch
feature, with this as an interface, and establish the maintainers of the
feature.
== Patch Series 2 (Requesting resourcing by Arm)
The goal of this series would be to move the dom0less device tree
parsing under hyperlaunch. We would respectfully request a member of the
Arm community to volunteer to take ownership and steward the series
through submission and review process.
The implementation of this series would see dom0less device tree parsing
to use, and expand if necessary, the common/fdt parsing helpers. I
personally would see this logic move under common/domain-builder using a
file name that would not collide with the files from the hyperlaunch v1
series, e.g. a suggestion would be fdt-dom0less.c.
As for ownership of that file, I would suggest the addition of a
HYPERLAUNCH DOM0LESS COMPATIBILITY be added to MAINTAINERS with the
appropriate Arm community members, but understand and willing to
consider the position that it falls under HYPERLAUNCH. The purpose of
using HYPERLAUNCH DOM0LESS COMPATIBILITY will be to provide a means to
signal the retirement of dom0less compatibility mode at some future point.
== Patch Series 3 (Resourced by Apertus)
The goal of this series will be to formalize hyperlaunch dom0less
compatibility mode. The series would be submitted by myself and/or
Christopher. It will consist of documentation patches that will add a
doc/features/hyperlaunch.rst and a
doc/features/hyperlaunch/dom0less-compatiblilty.rst. The hyperlaunch.rst
path will fall under HYPERLAUNCH, while dom0less-compatiblilty.rst will
fall under HYPERLAUNCH DOM0LESS COMPATIBILITY. This will also see
DOM0LESS in SUPPORTED.md be renamed to HYPERLAUNCH.
In summary, I would like to reiterate points made by Rich and
Christopher. The motivation and concept of the hypervisor
differentiating capability that is hyperlaunch goes back to the 2012
domain builder work that was a companion to the hardware domain work.
Since then, there have been a few of us in the OpenXT community that
desired to make this an integral part of Xen. Internal OpenXT
discussions in 2017-2018 along with the announcement of dom0less
inspired confidence there were in fact other uses cases and interest in
Xen gaining such an integrated capability.
A fact that should not go overlooked or undervalued is that hyperlauch
would not exist today without the extremely generous support StarLab
provided to sponsor Christopher and I to do the R&D, proof of viability,
and a full working prototype that provided the basis for the v1 series.
This was not a minor investment on their part, and taking the design to
completion requires further substantial investment. Due to an
acquisition imposed shift in market focus, StarLab has since stepped away.
Fortunately, and for which we are immensely appreciative, AMD has
recently stepped up to fund some hyperlaunch work items. With a growing
number of hyperlaunch use cases, I would be happy to help those with
budgetary influence to communicate the business benefits of investment
in open work items, targeting specific launch configurations, safety and
security properties.
The amount of work to be done goes beyond just parallel domain
construction, there are tangential capabilities that need updating and
incorporation. For instance, a topic discussed during the summit, VPCI
and device assignment at/during hypervisor startup. Anyone looking or
willing to provide purely financial support, feel free to reach out
directly to Christopher and me, as we have a few avenues to enable the
flow of funds.
Attached are candidate work items for funding and resourcing of
hyperlaunch integration, building upon the proposed Apertus and Arm
patch series for dom0less rename and migration to hyperlaunch common
interfaces.
Lastly, thank you to everyone who has taken the time to engage and
collaborate on how to resolve the task at hand!
V/r,
Daniel P. Smith
===
Here are outlines for the next development items appropriate for funding
for progress on hyperlaunch integration, following on from the three
series described above.
== Work Item: initial x86 Hyperlaunch SUPPORT and launch of dom0
(Resourcing TBD)
The goal is to enable initial hyperlaunch SUPPORT on x86, building upon
from the work in the posted v3 series of hyperlaunch.
Development proceeds from patches 9-12 of the v1 hyperlaunch series, to
add boot with minimal construction of a classic dom0 from a device tree
configuration. The Hyperlaunch SUPPORT statement would add x86 and Xen
nightly CI and release-gating test configurations would be extended to
include hyperlaunch on x86.
== Work Item: Hyperlaunch XSM policy for guests (Resourcing TBD)
The goal is to ensure that security policies of guests started via
hyperlaunch are accurately aligned with the configured functions of each
guest, which would allow for reduction of privilege of the dom0 in
dom0less systems.
A new XSM policy with granular permissions for domain functions, aligned
with the roles work, would be integrated into the hypervisor. It would
define new domain security labels appropriate for assigning to domUs,
matching privileges for the domain functions that can be assigned with
hyperlaunch, and allowing for more expressive policy control than is
expressible with dom0less.
== Work Item: Hyperlaunch of Arm guests SUPPORT (Resourcing TBD)
The goal of this effort is to enable hyperlaunch of Arm guests.
It would enable use of the architecture-common Hyperlaunch Device Tree
format on Arm, providing the forward migration path from the Dom0less
format, to the new format with additional flexibility for guest
construction, including control over console privilege assignment and
XSM label control.
It would achieve the objective of using common cross-architecture boot
structures and community-maintained common code.
Development would proceed from patches 9-10 of the v1 hyperlaunch
series, to add boot with construction of dom0 and domU guests from a
hyperlaunch device tree configuration.
Xen nightly CI and release-gating test configurations would be extended
to include hyperlaunch with guests on Arm.
== Work Item: Hyperlaunch of x86 PV and PVH guests (Resourcing TBD)
The goal of this effort is to enable Hyperlaunch of x86 PV and PVH guests.
This builds upon the initial x86 Hyperlaunch support to add domain
construction of PV and PVH guests.
Xen nightly CI and release-gating test configurations would be extended
to include hyperlaunch with guests on x86.
== Work Item: XSM-enforced FuSA (Resourcing TBD)
Hyperlaunch enables disaggregation, and XSM enables granular policy.
The goal of this work is to design and implement new modular,
fine-grained policy to integrate into Xen for control for
safety-critical VMs and isolation from non-safety-critical VMs.
== Work Item: Design for Hyperlaunch of x86 HVM guests (Resourcing TBD)
The goal of this effort is to research, prototype and produce design
documentation for enabling hyperlaunch of x86 HVM guests, ie. with
support for a device emulator.
This would build upon the x86 Hyperlaunch support for PVH guests.
Investigation to consider:
- launch alongside a classic dom0 domain
- launch on a static partitioned system without a dom0
- stub domains for device emulator isolation
- boot domain integration
- device assignment
-- Thanks for your consideration!
