On 04.07.2022 18:31, G.R. wrote: > On Tue, Jul 5, 2022 at 12:21 AM Roger Pau Monné <roger....@citrix.com> wrote: >> >> On Mon, Jul 04, 2022 at 11:37:13PM +0800, G.R. wrote: >>> On Mon, Jul 4, 2022 at 11:15 PM G.R. <firemet...@users.sourceforge.net> >>> wrote: >>>> >>>> On Mon, Jul 4, 2022 at 10:51 PM G.R. <firemet...@users.sourceforge.net> >>>> wrote: >>>>> >>>>> On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné <roger....@citrix.com> >>>>> wrote: >>>>>> Can you paste the lspci -vvv output for any other device you are also >>>>>> passing through to this guest? >>>>>> >>>> >>>> As reminded by this request, I tried to assign this nvme device to >>>> another FreeBSD12 domU. >>> Just to clarify, this time this NVME SSD is the only device I passed to >>> this VM. >>> >>>> This time it does not fail at the VM setup stage, but the device is >>>> still not usable at the domU. >>>> The nvmecontrol command is not able to talk to the device at all: >>>> nvme0: IDENTIFY (06) sqid:0 cid:0 nsid:0 cdw10:00000001 cdw11:00000000 >>>> nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:0 cdw0:0 >>>> nvme0: IDENTIFY (06) sqid:0 cid:0 nsid:0 cdw10:00000001 cdw11:00000000 >>>> nvme0: ABORTED - BY REQUEST (00/07) sqid:0 cid:0 cdw0:0 >>>> >>>> The QEMU log says the following: >>>> 00:05.0] Write-back to unknown field 0x09 (partially) inhibited (0x00) >>>> [00:05.0] If the device doesn't work, try enabling permissive mode >>>> [00:05.0] (unsafe) and if it helps report the problem to xen-devel >>>> [00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30, >>>> entry 0) >>> >>> I retried with the following: >>> pci=['05:00.0,permissive=1,msitranslate=1'] >>> Those extra options suppressed some error logging, but still didn't >>> make the device usable to the domU. >>> The nvmecontrol command still get ABORTED result from the kernel... >>> >>> The only thing remained in the QEMU file is this one: >>> [00:05.0] msi_msix_setup: Error: Mapping of MSI-X (err: 61, vec: 0x30, >>> entry 0) >> >> Hm it seems like Xen doesn't find the position of the MSI-X table >> correctly, given there's only one error path from msi.c returning >> -ENODATA (61). >> >> Are there errors from pciback when this happens? I would expect the >> call to pci_prepare_msix() from pciback to fail and thus also report >> some error? >> >> I think it's likely I will have to provide an additional debug patch >> to Xen, maybe Jan has an idea of what could be going on. >> > pciback reports the same MSI-x related error. > But even with DEBUG enabled, I didn't see more context reported. > Please find details from the attachment.
And nothing pertinent in "xl dmesg"? Looking back through the thread I couldn't spot a complete hypervisor log (i.e. from boot to assignment attempt). An issue with MSI-X table determination, as Roger suspects, would typically be associated with a prominent warning emitted to the log. But there are also further possible sources of -ENXIO, which would go silently. Jan
