On 27/11/19 4:34 PM, Jan Kiszka wrote:
On 27.11.19 11:44, Nikhil Devshatwar wrote:
On 27/11/19 3:30 PM, Jan Kiszka wrote:
On 27.11.19 10:53, Nikhil Devshatwar wrote:
On 19/11/19 12:14 AM, Jan Kiszka wrote:
Hi Nikhil,
On 18.11.19 14:26, Devshatwar, Nikhil wrote:
HI Jan,
Current Jailhouse implementation has support for partitioning the
SPI interrupts on a GICv3.
However, the LPI interrupts managed by the GIC ITS cannot be
partitioned.
Is this something we would need a GICv4 for?
Yes. this applies to GICv3 and GICv4.
Too bad...
Typical usage of LPI are for connecting PCIe devices on arm64
platforms where the endpoint sends MSI interrupts.
Kernel drivers for GIC assumes full ownership of the ITS module.
Means, it also touches registers it does not own? Or are there
conceptually shared resources in the ITS that everyone needs to
access when programmed own parts of it?
Software needs to access the ID registers, setup base address of
the different tables and queues.
Most of this is done once for each PCI device
When emulating, all these accesses should get trapped and policed
by Jailhouse
To enable partitioning of PCI devices/functions (from same or
different PCI controller) across VMs,
One of the key important feature that will be needed is the ITS
virtualization.
Since the GIC ITS does not support per VM tables, one option
would be to use emulation of the ITS module.
Here is the proposal for such an approach.
·GIC kernel driver allocates separate ITS table in its VM address
space
·When Jailhouse is enabled, Hypervisor allocates a new ITS table
and copies the descriptors from root cell
·All accesses to the ITS registers are trapped and emulated in
the Jailhouse
·Any attempt from a VM trying to setup the descriptor for a
certain device needs to be policed against VM ownership
·Jailhouse validates and updates the physical ITS table
descriptors to match the VM maintained tables
This will be similar to the approach taken for SMMU stage1
emulation developed by Pratyush Yadav.
What so you think about adding this support in the Jailhouse?
Would you be willing to accept patches for the ITS emulation?
Well, we do want assignment of PCI devices to different cells when
this is feasible, and we do want not only DMA partitioning but
also safe & secure interrupt handling. OTW: Yes, we need something
that enables that.
And when the hardware does not (yet) support us in enforcing the
partitions during runtime, we need to emulate that in software.
It's just the question how complex that emulation needs to become,
for Linux or for "educated" guests.
To give an overview, following will be the changes:
* Initialize ITS as part of the jailhouse enable, setup tables and
command queues
* Most ITS MMR emulation will be allow/forbid read/write of a
bitfield based on the VM accessing it
* Emulation of base address register write would be to map guest
table, copy few entries from guest table to hypervisor maintained
table
* Emulate event and command queues similar to the SMMU emulation
Ugh, sounds not as simple as I was hoping for.
* Add support for LPI injection into vCPU (Since VM's CPU affinity
is fixed, this can be optimized and can be a blind injection)
But this is something going away with GICv4, thanks to direct
injection, no?
Yes, the GICv4 has virtual LPI injection support.
BTW, those accesses to the ITS will not be on the interrupt
handling path, will they? Only for the setup and maybe affinity
adjustment?
IRQ acknoledgement does not need to access the ITS device table or
interrupt translation table.
...but an injection trampoline is needed. No difference to current
SPI injections, though.
BTW, how does interrupt remapping play into this? I suspect there is
a similar concept on ARM as well, right?
ITS supports collection tables. ITS translates the LPI into a INTID
and collectionID
collectionID is used as index into collection table to find the
interrupt remapping to different CPUs.
When we are emulating, we may decide to support collection tables
optionally/incrementatlly.
More details at
https://static.docs.arm.com/dai0492/b/GICv3_Software_Overview_Official_Release_B.pdf
Reading... When we initialize the ITS for physical use by Jailhouse,
this means we will migrate the prior usage of the root cell into an
emulated one, right?
Yes, that's right.
Read out the current state, copy it over the Jailhouse resources while
validating its current content, enabling the new setup.
You only have to create a copy of the device table.
Individual device specific Interrupt translation tables are anyways
owned by the ITS kernel driver, Jailhouse should not touch it.
Reminds me a lot of VT-d interrupt remapping - and I thought that
would be complex...
I am not familiar with VT-d interrupt remapping, so can't comment.
But this will be similar to the SMMU takeover. Only the pointers to the
ITT tables are to be maintained by hypervisor. It should not worry about
each eventID and LPI as such
Nikhil D
Jan
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/55bc75bd-1cf3-f414-68c1-7ca22c0ab7a3%40ti.com.
