Hi Jan,
Am 14.08.2020 um 17:29 schrieb Jan Kiszka:
Hi Christoph,
On 14.08.20 10:58, Christoph Gerum wrote:
Over the last few months, our team has been working on an automatic
configuration utility for jailhouse. As our effort now nears a usable
state, we would like to present it to the jailhouse community.
Our tooling is available on https://github.com/ekut-es/autojail
The documentation is available on
https://atreus.informatik.uni-tuebingen.de/~gerum/autojail/
The tooling works by extracting information about the target hardware
from the Linux runtime system (currently mainly
/sys/firmware/devicetree/),
As you are processing a device tree already, can you also consume an
offline dtb?
We currently, are only processing the online device tree, as we want to
also know information from device tree overlays and dynamic device tree
nodes generated by kernel modules.
Did you also already have a look at system-dt
(https://elinux.org/Device_tree_future#System_Device_Tree_2019.2C_2020)?
That could become the next logical step for dt-based systems.
I have not been aware of system-dt. It might be an interesting
possibility to completely avoid autojail specific configurations.
and then generating jailhouse cell configs
for root- and guest cells from a minimal configuration file.
Important "detail": It comes with a new human-processable configuration
format, you YAML schema, right? Is that format as powerful (or
low-level) as the binary format, i.e. a full replacement of the current
C files?
The yml files currently are more or less a superset of the low-level
format. e.g. a memory region might look like:
memory_regions:
"System RAM":
physical_start_addr: 0x0
virtual_start_addr: 0x0
size: 1 GB
flags: [MEM_READ, MEM_WRITE, MEM_EXECUTE]
With a bit of syntactic sugar e.g. sizes can use human readable units,
no counting of memory_regions needed.
Or does it provide abstractions like
Abstractions are currently implemented by making most entries optional:
"give me 1GB, I don't care where"
would be:
memory_regions:
"System RAM":
size: 1 GB
flags: [MEM_READ, MEM_WRITE, MEM_EXECUTE]
"give me device B, with whatever I/O resources it comes"?
This is currently expressed by the device tree path:
memory_regions:
gpio: /soc/gpiomem
How is the (planned) workflow then? Currently on x86, you call "config
create" and then perform manual tuning on the resulting C config for the
root cell to make it work. I suspect that is what "autojail extract" is
about. In your case, tuning would be done on to the yml file? How would
will the workflow be for non-root cell configs?
Hint: Add a user story to your documents would help getting a big
picture quicker.
We have a different workflow and at least one user story per milestone:
Assuming that we are able to connect to a target eval board via ssh.
Milestone 1: Simplified config.
autojail init
autojail extract
manually create cells.yml to specifie the configuration. Guest cells are
created in the same cells.yml. e.g.:
cells:
root:
name: "The Root Cell"
type: root
...
guest1:
name: "The first guest celll"
type: linux
autojail config
manually test generated *.c configuration, and change cells.yml until it
works.
Milestone 2: "jailhouse config create" on steroids
At this milestone we wan't to allow the following use case:
autojail init
autojail extract
autojail config --num-linux-guest 2 --memory-per-guest 128 MB
autojail deploy
Then you should be able to ssh into the root cell and directly into each
guest and the root cell. Ideally with minimal, manual interventions.
E.g we will configure ivshmem_net from root to each guest, and configure
a bridge device or port forwarding in the root cell.
Milestone 3+: Cell config management
These have not fully finalized yet, but from Milestone 3 we intend to
use an extended cells.yml that supports fine tuning capabilities on par
with the current configuration format, while allowing tool support for
the fine tuning as well.
It differs from the current jailhouse config in the following ways:
I suspect you meant the current config generator
Yes, sorry.
- It targets aarch64 instead of x86_64.
A final architecture will have to address all archs, even if the input
for x86 will remain different.
We don't have any plans to fully support X86 in autojail at the moment.
But I am considering integrating our device tree work into pyjailhouse
to allow jailhouse config create for ARM and AARCH64. At least as a
first step. But if our configuration file format would prove interesting
as well, we might reconsider those plans.
- It supports configuration of guest and root cells.
- It allows a simplified configuration of IVSHMEM_NET.
The current release has the following limitations, which we would like
to address in the coming weeks:
- No generation of configuration for the inmates (networking,
device-tree)
- Dead Simple Memory allocator that will probably give up at a
relatively low memory pressure.
Where does the memory allocator come into play?
The static memory allocator currently allows us to leave out physical
and virtual addresses for the memory regions mapped to RAM. While
keeping some additional constraints: e.g. MEM_LOADABLE regions are added
to the root cell as well if their physical address range does not
overlap with any memory region already specified in the root cell.
- Configuration of inter-cell communication is supported for
IVSHMEM_NET devices only
- Only tested on Raspberry PI 4B
- No configuration of SMMU or other IOMMUs
The current release has so far only been tested on Raspberry PI 4B, and
this announcement mainly is here as a request for comments, and to
evaluate how our work might fit into the general jailhouse ecosystem.
- Would there be interest to somehow integrate it more closely into
the jailhouse ecosystem?
- We would be very interested, if we could use it as a configuration
generator for the current work on memory coloring in jailhouse.
Specifically coloring is a scenario where more tooling support for
config generation and validation is needed, indeed. You may have seen
that new "jailhouse config check" command which performs the latter (and
should work with your approach as well as it processes binary configs),
but we also need more of the former.
One of the next points on my bucket list will be to directly integrate
jailhouse config check into our workflow. Would you be willing to accept
patches to move the actual checks to pyjailhouse to allow an easier use
in external tooling.
So, yes, there is definitely interest in new ideas and concrete
solutions, specifically for arm64 (though not only). After having to
refactor configs/ recently, more than once, I would appreciate a lot if
we could reduce the manual maintenance.
What needs careful thoughts are the possible use cases and workflows. We
need a solution that automates what can be safely automated, ideally
warns when user input or validation is needed and does not stand in the
way when manual tweaking must be applied. That is due to the low
abstraction level of the binary config format the hypervisor consumes.
One of our main goals is to be an experimentation field for finding the
right abstractions to allow an easy configuration, while still enabling
the fine tuning needed for more complex use cases.
Christoph
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/2b64a2ef-970b-bc7c-6e7d-8dd3f9fcc5e1%40uni-tuebingen.de.
