Re: Dynamic & heterogeneous machines, initial configuration: problems
On Wed, Jan 31, 2024 at 09:14:21PM +0100, Markus Armbruster wrote:
> == What users want for initial configuration ==
>
> 1. QMP only
>
>Management applications need to use QMP for monitoring anyway. They
>may want to use it for initial configuration, too. Libvirt does.
>
>They still need to bootstrap a QMP monitor, and for that, CLI is fine
>as long as it's simple and stable.
>
> 2. CLI and configuration files
>
>Human users want a CLI and configuration files.
>
>CLI is good for quick tweaks, and to explore.
>
>For more permanent, non-trivial configuration, configuration files
>are more suitable, because they are easier to read, edit, and
>document than long command lines.
>
>
> == What we have for initial configuration ==
>
> Half of 1. and half of 2., satisfying nobody's needs.
>
> Management applications need to do a lot of non-trivial initial
> configuration with the CLI.
>
> Human users struggle with inconsistent syntax, insufficiently expressive
> configuration files, and huge command lines.
Our two sets of users (humans & machines) have pretty different
desires in many respects. To suit machines, we've made our config
more and more expressive & detailed. This has worked well for
machines. Humans have largely ignored most of it though, and
stuck with the massively simpler ("legacy") config approaches.
Every now & then though, humans are forced to use the modern low
level config to access some edge case feature not exposed in the
legacy syntax. Pain and suffering ensues.
I feel like we have become somewhat incapable of innovating on
features that are in the interests of humans, because our thought
processes get derailed by a desire to keep thing fully expressive
for machines, and the human areas of code are often the most
crufty with highest risk of breakage.
My wish is that when we switch to a new binary, we exclusively
focus on machines, and build a human focused frontend above
that, so we have clean separation, and we can do whatever we
thing is right for humans without being distracted by whether
machines can consume it or not, and vica-verca.
> = Problem 2: Defining machines =
>
> This is how I understand the problem. Please correct me where I'm off.
>
>
> == How we'd like to build machines ==
>
> We want to build machines declaratively, by configuring devices and
> their connections.
>
> We want to build composite devices the same way.
>
> The non-composite devices are provided by the QEMU binary.
>
> Users want to build machines as variations of canned machine types
> shipped with QEMU. Some users may want to build their own machines from
> scratch.
>
> To enable all this, machine configuration needs to be composable and
> dynamic.
>
> Composable means configuration can be assembled from components,
> recursively.
>
> Dynamic means it can be done during qemu-system-FOO initial
> configuration.
>
>
> == What we have for defining machines ==
>
> A QEMU binary provides a fixed set of device types, some of them
> composite, and a fixed set of machine types.
>
> Machines are QOM objects: instance of a concrete subtype of "machine".
>
> Devices are usually QOM objects: instance of a concrete subtype of
> "device". Exceptions remain in old code nobody can be bothered to
> update.
>
> Both machine types and composite devices are built from devices
> by code, i.e. imperatively, not declaratively.
>
> The code can be parameterized. For QOM objects, parameters should be
> QOM properties, but machine type code additionally uses global old-style
> configuration such as -drive and -serial.
>
> Code may create default backends for convenience. Machine type code may
> also create backends to honor global old-style configuration. Only some
> backends are QOM objects.
The default devices/backends in machines are an artifact of use trying
to do something which suits both humans and machines, while at the same
time hardcoding machine definitions.
Our '-nodefaults' hack is a gross solution to this problem.
An ability to dynamically define machines could give us a far more
attractive solution to this problem.
eg we could have a 'q35-minimal.cfg' configuration that defined
the minimal 'q35' machine type, along with a 'q35-recommended.cfg'
that added the typical extra devices, and possibly even a further
'q35-simple.cfg' that added the typical extra devices, alongw ith
typical extra backend connectivity.
Using a 'q35-minimal.cfg' the mgmt app would have to setup all
backends and extra devices explicitly as it saw fit.
Using a 'q35-recommended.cfg' the human would merely have to
provide backend configuration
Using a 'q35-simple.cfg' the human would merely need to provide
a disk image path.
this would also solve our forever problem of "sensible defaults"
for RAM size, CPU model, etc being an undecidable problem. We
could have 'q35-tiny.cfg' and 'q35-huge.cfg', or any number of
other profile variants. Or we could have
Re: Dynamic & heterogeneous machines, initial configuration: problems
Hi Markus, On Wed, Jan 31, 2024 at 09:14:21PM +0100, Markus Armbruster wrote: > Date: Wed, 31 Jan 2024 21:14:21 +0100 > From: Markus Armbruster > Subject: Dynamic & heterogeneous machines, initial configuration: problems > > This memo is the fruit of discussions with Philippe Mathieu-Daudé. > Its errors are mine. > > QEMU defines machines statically in C code. We've long wished we could > define them dynamically in some suitable DSL. This is what we call > "dynamic machines". > > There's a need for machines that contain more than one target's CPUs. > This is what we call "heterogeneous machines". They require a single > binary capable of any of the targets involved. > > There's substantial overlap with a seemingly unrelated problem: > machine-friendly initial configuration. > > To keep the memo's length in check (sort of), it focuses on (known) > problems. > > > = Problem 1: Initial configuration = > > Previously discussed in > > Subject: Redesign of QEMU startup & initial configuration > Date: Thu, 02 Dec 2021 07:57:38 +0100 > Message-ID: <87lf13cx3x@dusky.pond.sub.org> > > > == What users want for initial configuration == > > 1. QMP only > >Management applications need to use QMP for monitoring anyway. They >may want to use it for initial configuration, too. Libvirt does. > >They still need to bootstrap a QMP monitor, and for that, CLI is fine >as long as it's simple and stable. > > 2. CLI and configuration files > >Human users want a CLI and configuration files. > >CLI is good for quick tweaks, and to explore. > >For more permanent, non-trivial configuration, configuration files >are more suitable, because they are easier to read, edit, and >document than long command lines. > > > == What we have for initial configuration == > > Half of 1. and half of 2., satisfying nobody's needs. > > Management applications need to do a lot of non-trivial initial > configuration with the CLI. > > Human users struggle with inconsistent syntax, insufficiently expressive > configuration files, and huge command lines. > > > = Problem 2: Defining machines = > > This is how I understand the problem. Please correct me where I'm off. > > > == How we'd like to build machines == > > We want to build machines declaratively, by configuring devices and > their connections. > > We want to build composite devices the same way. > > The non-composite devices are provided by the QEMU binary. > > Users want to build machines as variations of canned machine types > shipped with QEMU. Some users may want to build their own machines from > scratch. > > To enable all this, machine configuration needs to be composable and > dynamic. > > Composable means configuration can be assembled from components, > recursively. > > Dynamic means it can be done during qemu-system-FOO initial > configuration. > > > == What we have for defining machines == > > A QEMU binary provides a fixed set of device types, some of them > composite, and a fixed set of machine types. > > Machines are QOM objects: instance of a concrete subtype of "machine". > > Devices are usually QOM objects: instance of a concrete subtype of > "device". Exceptions remain in old code nobody can be bothered to > update. > > Both machine types and composite devices are built from devices > by code, i.e. imperatively, not declaratively. > > The code can be parameterized. For QOM objects, parameters should be > QOM properties, but machine type code additionally uses global old-style > configuration such as -drive and -serial. > > Code may create default backends for convenience. Machine type code may > also create backends to honor global old-style configuration. Only some > backends are QOM objects. > > Machine types split their code between object creation (QOM methods > .instance_init() and .instance_post_init()) and machine initialization > (MachineClass method .init()). However, basically everything is done in > the latter. > > QOM device types split their code between object creation and device > realization (qdev method .realize()). The actual split varies widely > between devices. Developers are commonly unsure what to put where. > > After machine type code is done, the resulting machine can still be > adjusted with device cold plug and unplug: -device, device_add, > device_del. Only works for a subset of the devices. > > Related, but out of scope here: hot plug and unplug. > > > = Common sub-problem: qemu-system initial startup
Dynamic & heterogeneous machines, initial configuration: problems
This memo is the fruit of discussions with Philippe Mathieu-Daudé. Its errors are mine. QEMU defines machines statically in C code. We've long wished we could define them dynamically in some suitable DSL. This is what we call "dynamic machines". There's a need for machines that contain more than one target's CPUs. This is what we call "heterogeneous machines". They require a single binary capable of any of the targets involved. There's substantial overlap with a seemingly unrelated problem: machine-friendly initial configuration. To keep the memo's length in check (sort of), it focuses on (known) problems. = Problem 1: Initial configuration = Previously discussed in Subject: Redesign of QEMU startup & initial configuration Date: Thu, 02 Dec 2021 07:57:38 +0100 Message-ID: <87lf13cx3x@dusky.pond.sub.org> == What users want for initial configuration == 1. QMP only Management applications need to use QMP for monitoring anyway. They may want to use it for initial configuration, too. Libvirt does. They still need to bootstrap a QMP monitor, and for that, CLI is fine as long as it's simple and stable. 2. CLI and configuration files Human users want a CLI and configuration files. CLI is good for quick tweaks, and to explore. For more permanent, non-trivial configuration, configuration files are more suitable, because they are easier to read, edit, and document than long command lines. == What we have for initial configuration == Half of 1. and half of 2., satisfying nobody's needs. Management applications need to do a lot of non-trivial initial configuration with the CLI. Human users struggle with inconsistent syntax, insufficiently expressive configuration files, and huge command lines. = Problem 2: Defining machines = This is how I understand the problem. Please correct me where I'm off. == How we'd like to build machines == We want to build machines declaratively, by configuring devices and their connections. We want to build composite devices the same way. The non-composite devices are provided by the QEMU binary. Users want to build machines as variations of canned machine types shipped with QEMU. Some users may want to build their own machines from scratch. To enable all this, machine configuration needs to be composable and dynamic. Composable means configuration can be assembled from components, recursively. Dynamic means it can be done during qemu-system-FOO initial configuration. == What we have for defining machines == A QEMU binary provides a fixed set of device types, some of them composite, and a fixed set of machine types. Machines are QOM objects: instance of a concrete subtype of "machine". Devices are usually QOM objects: instance of a concrete subtype of "device". Exceptions remain in old code nobody can be bothered to update. Both machine types and composite devices are built from devices by code, i.e. imperatively, not declaratively. The code can be parameterized. For QOM objects, parameters should be QOM properties, but machine type code additionally uses global old-style configuration such as -drive and -serial. Code may create default backends for convenience. Machine type code may also create backends to honor global old-style configuration. Only some backends are QOM objects. Machine types split their code between object creation (QOM methods .instance_init() and .instance_post_init()) and machine initialization (MachineClass method .init()). However, basically everything is done in the latter. QOM device types split their code between object creation and device realization (qdev method .realize()). The actual split varies widely between devices. Developers are commonly unsure what to put where. After machine type code is done, the resulting machine can still be adjusted with device cold plug and unplug: -device, device_add, device_del. Only works for a subset of the devices. Related, but out of scope here: hot plug and unplug. = Common sub-problem: qemu-system initial startup = QAPI/QMP is our most capable, flexible, and mature configuration interface. We need to offer machine-friendly initial configuration via QMP, and we'd very much like to have a QAPI-based CLI and configuration files (see "What users want for initial configuration" above). Dynamic machine configuration happens during initial startup. This makes it part of the larger initial configuration problem. We want an integrated solution for the larger configuration problem that includes machine configuration. Traditionally, QMP becomes available quite late, long after machine initialization. This precludes use of QMP for most parts of initial configuration, including dynamic machine configuration. To enable a bit of machine configuration via QMP, experimental CLI option -preconfig delays part of initial startup including machine initialization by moving it into QMP command x-exit-preconfig. Only selected
