Hi,
I have a Sun Oracle Netra T5220. I'm running Solaris 11.3 as the
control domain. I want my OpenBSD to be an IO domain, i.e., a domain
that has direct access to IO: in my case, the four RJ45 network
interfaces on the back of the server (plus an NVMe card). In T5220,
the four interfaces connect to two Intel 82571EB chips, which in turn
connect to a single PCIe bus.
So in Solaris I first enable IO virtualization for the PCIe bus:
root@solaris:~# ldm set-io iov=on pci
And then assign these two chips to my IO domain:
root@solaris:~# ldm add-io MB/NET0 router
root@solaris:~# ldm add-io MB/NET2 router
Everything looks good:
root@solaris:~# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------
pci BUS pci primary IOV
niu NIU niu primary
MB/RISER0/PCIE0 PCIE pci router OCC
MB/RISER1/PCIE1 PCIE pci primary EMP
MB/RISER2/PCIE2 PCIE pci primary EMP
MB/PCI_MEZZ/PCIE5 PCIE pci primary EMP
MB/NET0 PCIE pci router OCC
MB/NET2 PCIE pci router OCC
MB/SASHBA PCIE pci primary OCC
However, OpenBSD is unable to initialize the interfaces:
em0 at pci4 dev 0 function 0 "Intel 82571EB" rev 0x06: couldn't map
interrupt
em1 at pci4 dev 0 function 1 "Intel 82571EB" rev 0x06: couldn't map
interrupt
em2 at pci5 dev 0 function 0 "Intel 82571EB" rev 0x06: couldn't map
interrupt
em3 at pci5 dev 0 function 1 "Intel 82571EB" rev 0x06: couldn't map
interrupt
I think the OpenBSD em driver is to blame (and not the
virtualization), because I also assigned an NVMe card to the very same
OpenBSD system and it works great.
Before I ran OpenBSD 7.5 as the control domain without the IO
virtualization enabled, and the interfaces worked fine. I found a
similar issue described here, where they ran OpenBSD as a guest
system:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205445
There the conclusion was:
> The OpenBSD em/iwm drivers may either not use MSI, or require legacy
> interrupts to be valid before using MSI.
I don't know what to do with this. I would appreciate it someone
could give me some clue.
Thanks & best,
Irek
