On 2018-04-26 18:55, [email protected] wrote:
> I just see how much I need to understand beyond what I asked...
>
> How should it work from a 10,000ft view if we lived in a perfect world?
>
> PCI controllerS are terminated on the main Linux. Jaihouse hypervisor
> take over and reexposes virtual ones to root-cell who then re-probes the
> whole space?
If mmcfg interception is done, we mostly filter accesses, allowing only
those to configured devices of a cell. We also inject ivshmem devices
this way, if there are any configured on the controller. Do do not
probe, we just execute filter and injection rules.
On x86, we also configure the IOMMU to allow DMA requests from
cell-owned devices to hit cell-owned memory. And the same goes with
MSI/MSI-X interrupts.
> As IOMMUs are also attached to the PCI domains, I assume the same
> transfer must be done for IOMMUs?
My current ARM picture is, but that may be incomplete, that we have two
scenarios here:
One is about PCI host controllers that are behind an IOMMU that does
not allow us to hand out individual PCI devices. We treat such
controllers like platform devices and do binary access management (all
or nothing). Also, we do not intercept mmcfg of such controllers or even
try to inject our ivshmem devices into them. That's why we have the
virtual host controller.
And then there are controllers that do allow per-device assignment
because the IOMMU can tell devices apart. That would be a candidate for
mmcfg interception, to do fine-grained PCI device assignment and to
inject virtual devices at that chance. Currently, this is limited to the
generic PCI controller - and we do no IOMMU configuration yet anyway...
> Some PCI devices are "platform" devices (i.e. not really behind a PCI
> bridge but controlled through PCI) and pugable devices (those really are
> PCI devices) so the IOMMU config may not be exactly the same for those
> two types of devices.
I would assume that such devices - if they are on a partitionable host
controller - would remain under the ownership of the most trusted cell
and we could only hand out the other devices on that host. But maybe
things are more complicated. I didn't configure such a system in such
details yet (many of our ARM reference configs are still permissively
laid out).
>
> shouldn't we have a
>
> struct pci_domain {
> __u64 pci_mmconfig_base;
> __u32 pci_mmconfig_size;
> __u8 pci_mmconfig_start_bus;
> __u8 pci_mmconfig_end_bus;
> struct jailhouse_iommu* iommu;
> } __attribute__((packed));
>
> and
>
> struct jailhouse_system {
> char signature[6];
> __u16 revision;
> struct jailhouse_memory hypervisor_memory;
> struct jailhouse_debug_console debug_console;
> struct {
> struct pci_domain pci_domains[MAX_PCI_DOMAINS];
> __u8 pci_is_virtual;
>
As Jailhouse does not yet have IOMMU support on ARM, I also wasn't
forced to think about the specific relationships between IOMMUs and PCI
controllers yet, not to speak of multiple host controllers, thus
multiple domains. The latter is even not support on x86 yet. Also, the
relationships between IOMMU units and controlled devices is less regular
on x86 than it might be on ARM.
So, your ThunderX seems to challenge us with such a case. Does it
implement an ARM SMMU? v2, v3 or so?
Jan
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
