Hi Peter,

Thanks for getting back to me. Here's the info you requested:

[    0.163085] acpi PNP0A03:00: host bridge window
[0xd000000000-0xd0100fffff] (ignored, not CPU addressable)

 That one is most likely a bug in bhyve, where the space used for 64-bit
BAR placement isn't tested against the max physaddr width of the host

 To confirm, would you be able to report on this value on your system ?

# sudo pkg install cpuid
# cpuid | grep ^80000008
On my Intel i7-4790K CPU:

# cpuid | grep ^80000008
80000008 00003027 00000000 00000000 00000000

The device has an MSI capability, but the nvidia driver may not use it.
bhyve PCI passthrough requires the use of MSI/MSI-x interrupts, and
doesn't support using legacy interrupts.

This could be confirmed from the output of /proc/interrupts when
booting Linux on the system.
Output of /proc/interrupts:

   0:        137   IO-APIC-edge      timer
   1:          9   IO-APIC-edge      i8042
   4:        965   IO-APIC-edge      serial
   8:          0   IO-APIC-edge      rtc0
   9:          0   IO-APIC-fasteoi   acpi
  12:        138   IO-APIC-edge      i8042
  17:          0   IO-APIC-fasteoi   snd_hda_intel
  24:          0   PCI-MSI-edge      virtio0-config
  25:       8535   PCI-MSI-edge      virtio0-req.0
  26:          0   PCI-MSI-edge      virtio1-config
  27:        123   PCI-MSI-edge      virtio1-input.0
  28:          1   PCI-MSI-edge      virtio1-output.0
NMI:          0   Non-maskable interrupts
LOC:       6050   Local timer interrupts
SPU:          0   Spurious interrupts
PMI:          0   Performance monitoring interrupts
IWI:       2484   IRQ work interrupts
RTR:          0   APIC ICR read retries
RES:          0   Rescheduling interrupts
CAL:          0   Function call interrupts
TLB:          0   TLB shootdowns
TRM:          0   Thermal event interrupts
THR:          0   Threshold APIC interrupts
MCE:          0   Machine check exceptions
MCP:          1   Machine check polls
ERR:          0
MIS:          0

I guess the lack of a line containing PCI-MSI-* here indicates the nvidia driver isn't using an MSI/MSI-x interrupt? However, searching the web suggests the Linux nvidia driver does use MSI interrupts. This taken from a working non-VM Linux dmesg:

[ 4.330536] nvidia 0000:05:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    4.330542] nvidia 0000:05:00.0: setting latency timer to 64

Source: https://bugzilla.kernel.org/show_bug.cgi?id=20432#c2
(Thread also mentions disabling MSI)

I'll try some Linux boot options and reordering the devices when calling bhyve to see if that changes anything.



freebsd-virtualization@freebsd.org mailing list
To unsubscribe, send any mail to 

Reply via email to