I have a Mellanox 10G adapter in my Qubes box. When I spin up a domU with the device passed-thru, I am unable to load the device driver due to what appears to be a lack of support for MSI-x interrupt mapping.
There was some chatter a while back regarding MSI support: https://www.google.com/url?q=https%3A%2F%2Fwww.qubes-os.org%2Fnews%2F2017%2F10%2F18%2Fmsi-support%2F&sa=D&sntz=1&usg=AFQjCNGXTUp6QSX9Fb0v5Q6hyVc0i6NwfQ One might hope that this would enable MSI-x support as well, but apparently, not. My guess is that someone added a kluge to permit writing to the MSI enable regions in PCI config space for a device, neglecting to add the kluges for MSI-x as well. I find that I need to flag the mellanox as 'permissive' or else I get errors flagged on the dom0 console log regarding attempts to write to PCI config space. Once I do so, I am still unable to load the device driver for the ConnectX 4-LX device (mlx5_core). It fails when it attempts to allocate the set of MSIX vectors sized based on the # of CPUs online. The driver assumes MSIx are available. No fall-back to MSI. No fall-back to INT A/ INT B. Are there some other magic knobs I need to tweak? MSIx and Xen does raise some interesting issues. I would like to have to option of spinning up a domU with, say, 20 VCPU and, knowing that the Mellanox will assign queues to MSIx and I can assign MSIx to CPUs, I would like to have dom0 bind the vCPU to real CPU so that the interrupt mapping works correctly. This would be for some network performance work I have to do occasionally. I am also keen to enable the VF devices in the adapter (using some domU instance to enable) so that these VF instances can be passed to other domU instances. Also want to see if I can get hardware offload working and OVS working in qubes. Just for fun. Q: if a domU kernel enables VF devices in a mapped PF device instance, will the dom0 kernel discover the VF devices? IE: what is the mechanism whereby a kernel discovers the need for a bus-walk? This has to work correctly for Xen, no? -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/cdfd36f3-2087-4f34-98fb-732f77fb21ed%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
