On 23.07.20 00:42, [email protected] wrote:
Jan schrieb am Dienstag, 21. Juli 2020 um 15:40:30 UTC+2:
On 21.07.20 13:30, Thorsten wrote:
> I have a an ivshmem-net defined between the root cell and the first
> non-root Linux, and another one between the first NR-Linux and a
second
> NR-Linux. The second one between the NR-Ls is working smoothly fine.
>
> However, I cannot get the first one up. Jailhouse-console-ivshmem
says
> the cells are connected, In Linux the device comes up fine with
mem and
> alike, but the stats stay at zero, no carrier detected. I have
checked
> the configs to my best knowledge, mem-regions + devices look right.
Sounds a lot like an IRQ delivery issue. You can check that via
/proc/interrupts. At the point both peers are up, they should also see
config IRQs (first vector). If not, check if you have assigned the
right
IOMMU ID to the virtual PCI device in the root cell (use ID 1 if
none or
0 was used so far). See also
jailhouse-images/recipes-jailhouse/jailhouse/files/nuc6cay.c.
Thanks, you're spot-on right, but the problem persists. Partially.
I tried quite few variations of the configuration, but there are no MSIs
for virtual PCI devices in the root cell (/proc/interrupts as the
indicator). One thing that I noticed, no matter how I set the the
iommu-value for the root cell [0,1], the int-signalling in the NR-cells
always worked.
That is because the root cell has interrupt remapping on. That is needed
in order to handover with x2APIC enabled, which is faster. The non-root
cells can use x2APIC without that, thus do not have interrupt remapping
enabled.
I finally resorted to a different box having a similar processor (Atom
E3930 2-core vs E3950 4-core in the problem-box) and it worked there as
expected by setting iommu to 1. So I am still lost about the cause. The
two systems are sufficiently different boards to spot an obvious difference.
At this point, this problem is not a show-stopper for my trial
application, as the "beef" runs in the NR-cells. -- I can go on looking
further into the issue, if it is of interest.
If you want to debug this further, instrument along the injection path:
- ivshmem_register_mmio, IVSHMEM_REG_DOORBELL
- ivshmem_trigger_interrupt
- arch_ivshmem_trigger_interrupt - if you enter apic_send_irq, it's
likely fine
Another area to check is upfront caching in irq_cache.msg. On the
receiving side, that is triggered on programming of the MSI-X registers.
arch_ivshmem_update_msix is eventually called, and that does
x86_pci_translate_msi. You come out of the latter without irq_msg.valid
if the interrupt mapping translation goes south, e.g. due to a wrong
IOMMU unit.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/84037e45-31c6-6011-9491-e10c17d775bb%40siemens.com.
