On 15.08.20 00:37, Christoph Gerum wrote:
Hi Jan,
Am 14.08.2020 um 17:29 schrieb Jan Kiszka:
Hi Christoph,
On 14.08.20 10:58, Christoph Gerum wrote:
Over the last few months, our team has been working on an automatic
configuration utility for jailhouse. As our effort now nears a usable
state, we would like to present it to the jailhouse community.
Our tooling is available on https://github.com/ekut-es/autojail
The documentation is available on
https://atreus.informatik.uni-tuebingen.de/~gerum/autojail/
The tooling works by extracting information about the target hardware
from the Linux runtime system (currently mainly
/sys/firmware/devicetree/),
As you are processing a device tree already, can you also consume an
offline dtb?
We currently, are only processing the online device tree, as we want to
also know information from device tree overlays and dynamic device tree
nodes generated by kernel modules.
Merging an overlay into a dtb while parsing it anyway is no black magic.
Offline support would be a very valuable feature, like we've seen with
the current config collector.
But nodes generated by modules are unlikely to be relevant for Jailhouse
- how should a module know more about the hardware than the device tree
which is supposed to describe it? Can you give an example?
Did you also already have a look at system-dt
(https://elinux.org/Device_tree_future#System_Device_Tree_2019.2C_2020)?
That could become the next logical step for dt-based systems.
I have not been aware of system-dt. It might be an interesting
possibility to completely avoid autojail specific configurations.
and then generating jailhouse cell configs
for root- and guest cells from a minimal configuration file.
Important "detail": It comes with a new human-processable
configuration format, you YAML schema, right? Is that format as
powerful (or low-level) as the binary format, i.e. a full replacement
of the current C files?
The yml files currently are more or less a superset of the low-level
format. e.g. a memory region might look like:
memory_regions:
"System RAM":
physical_start_addr: 0x0
virtual_start_addr: 0x0
size: 1 GB
flags: [MEM_READ, MEM_WRITE, MEM_EXECUTE]
With a bit of syntactic sugar e.g. sizes can use human readable units,
no counting of memory_regions needed.
Or does it provide abstractions like
Abstractions are currently implemented by making most entries optional:
"give me 1GB, I don't care where"
would be:
memory_regions:
"System RAM":
size: 1 GB
flags: [MEM_READ, MEM_WRITE, MEM_EXECUTE]
"give me device B, with whatever I/O resources it comes"?
This is currently expressed by the device tree path:
memory_regions:
gpio: /soc/gpiomem
These are interesting features!
How is the (planned) workflow then? Currently on x86, you call "config
create" and then perform manual tuning on the resulting C config for
the root cell to make it work. I suspect that is what "autojail
extract" is about. In your case, tuning would be done on to the yml
file? How would will the workflow be for non-root cell configs?
Hint: Add a user story to your documents would help getting a big
picture quicker.
We have a different workflow and at least one user story per milestone:
Assuming that we are able to connect to a target eval board via ssh.
Milestone 1: Simplified config.
autojail init
I don't get the init step. From a user perspective, it looks useless and
could be done internally when needed.
autojail extract
manually create cells.yml to specifie the configuration. Guest cells are
created in the same cells.yml. e.g.:
cells:
root:
name: "The Root Cell"
type: root
...
guest1:
name: "The first guest celll"
type: linux
autojail config
manually test generated *.c configuration, and change cells.yml until it
works.
Skip the c-file generation and emit binary configs directly. Way more
user-friendly.
That reminds me of Andrej's pending patches in my inbox which add that
feature to pyjailhouse - you could simply use that once merged.
Milestone 2: "jailhouse config create" on steroids
At this milestone we wan't to allow the following use case:
autojail init
autojail extract
autojail config --num-linux-guest 2 --memory-per-guest 128 MB
autojail deploy
Then you should be able to ssh into the root cell and directly into each
guest and the root cell. Ideally with minimal, manual interventions.
E.g we will configure ivshmem_net from root to each guest, and configure
a bridge device or port forwarding in the root cell.
Milestone 3+: Cell config management
These have not fully finalized yet, but from Milestone 3 we intend to
use an extended cells.yml that supports fine tuning capabilities on par
with the current configuration format, while allowing tool support for
the fine tuning as well.
If your config format is able express that, that would be a great
progress. Say, you can request a complete device tree node, but then you
can additionally define that a specific MMIO range of that device shall
not be accessible.
Stabilizing the config format for all these use cases will not be
trivial, but it can be the way forward for Jailhouse.
It differs from the current jailhouse config in the following ways:
I suspect you meant the current config generator
Yes, sorry.
- It targets aarch64 instead of x86_64.
A final architecture will have to address all archs, even if the input
for x86 will remain different.
We don't have any plans to fully support X86 in autojail at the moment.
But I am considering integrating our device tree work into pyjailhouse
to allow jailhouse config create for ARM and AARCH64. At least as a
first step. But if our configuration file format would prove interesting
as well, we might reconsider those plans.
Definitely. Keep also in mind that our binary format is a moving target.
So, even if you build consequently upon pyjailhouse for parsing and
generating, you may have to rebase frequently when keeping things
out-of-tree.
- It supports configuration of guest and root cells.
- It allows a simplified configuration of IVSHMEM_NET.
The current release has the following limitations, which we would like
to address in the coming weeks:
- No generation of configuration for the inmates (networking,
device-tree)
- Dead Simple Memory allocator that will probably give up at a
relatively low memory pressure.
Where does the memory allocator come into play?
The static memory allocator currently allows us to leave out physical
and virtual addresses for the memory regions mapped to RAM. While
keeping some additional constraints: e.g. MEM_LOADABLE regions are added
to the root cell as well if their physical address range does not
overlap with any memory region already specified in the root cell.
Got it - a valuable feature as well! Even more when coloring will come
into play.
- Configuration of inter-cell communication is supported for
IVSHMEM_NET devices only
- Only tested on Raspberry PI 4B
- No configuration of SMMU or other IOMMUs
The current release has so far only been tested on Raspberry PI 4B, and
this announcement mainly is here as a request for comments, and to
evaluate how our work might fit into the general jailhouse ecosystem.
- Would there be interest to somehow integrate it more closely into
the jailhouse ecosystem?
- We would be very interested, if we could use it as a configuration
generator for the current work on memory coloring in jailhouse.
Specifically coloring is a scenario where more tooling support for
config generation and validation is needed, indeed. You may have seen
that new "jailhouse config check" command which performs the latter
(and should work with your approach as well as it processes binary
configs), but we also need more of the former.
One of the next points on my bucket list will be to directly integrate
jailhouse config check into our workflow. Would you be willing to accept
patches to move the actual checks to pyjailhouse to allow an easier use
in external tooling.
That is the purpose of pyjailhouse: Make functionality available for
different frontends. So, yes.
So, yes, there is definitely interest in new ideas and concrete
solutions, specifically for arm64 (though not only). After having to
refactor configs/ recently, more than once, I would appreciate a lot
if we could reduce the manual maintenance.
What needs careful thoughts are the possible use cases and workflows.
We need a solution that automates what can be safely automated,
ideally warns when user input or validation is needed and does not
stand in the way when manual tweaking must be applied. That is due to
the low abstraction level of the binary config format the hypervisor
consumes.
One of our main goals is to be an experimentation field for finding the
right abstractions to allow an easy configuration, while still enabling
the fine tuning needed for more complex use cases.
Sounds reasonable, also given what I wrote above. We need to find the
right balance between downstream experimenting and upstream exposing (to
more use cases), though.
Jan
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/ef151dec-8123-b296-6a23-4a9827a68874%40web.de.
