https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=290920
--- Comment #3 from [email protected] --- Yeah it is elusive bug for sure, what I can confirm is it must have been a PR into EFI kernel code itself, as booting FreeBSD 14 kernel with same /boot environment of FreeBSD 15 fixes it. Then again Dell r720 firmware is so old, won't get fixes for any new UEFI code, so problematic for people still running it in home lab. The only fix I could do for this was to create a FreeBSD 15-BETA5 memstick, set its bios to boot in BIOS mode, boot the USB stick and modify /boot/loader.conf with vfs.root.mountfrom="zfs:zroot/ROOT/default", to boot from nvme drives. Essentially just getting rid of refind completely and coping my own /boot environment to USB stick. After more testing on FreeBSD 15-BETA5, I found at least 2 more issues. Modifying /etc/pkg/FreeBSD.conf to pull latest ports, I found programs like dhcpcd to hang, killall dhcpcd does nothing, a firm "kill -9 <of each pid>" was needed. Last one is quite a serious one with bhyve, same environment for vm-bhyve I had on FreeBSD 14, I can confirm passthrough is broken on guests. In /boot/loader.conf: pptdevs="9/0/0 10/0/0 66/0/0" works fine, I can confirm the 2 USB ports and nvidia GPU are available for passthrough, however when passing it through on guest it is not working for any of them. 2 USB(zigbee and zwave) sticks are missing on home assistant, and ubuntu VM with nvidia GPU, mind you I had to use FreeBSD 14 stable for nvidia patches. Checking USB stick passthrough on home assistant: # dmesg | grep -i xhci [ 0.745789] xhci_hcd 0000:00:06.0: xHCI Host Controller [ 0.748133] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 1 [ 0.756495] xhci_hcd 0000:00:06.0: hcc params 0x014051c7 hci version 0x100 quirks 0x0000000100000090 [ 0.763687] xhci_hcd 0000:00:06.0: xHCI Host Controller [ 0.766148] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 2 [ 0.769165] xhci_hcd 0000:00:06.0: Host supports USB 3.0 SuperSpeed [ 1.500133] xhci_hcd 0000:00:06.0: Host took too long to start, waited 32000 microseconds. [ 1.656982] xhci_hcd 0000:00:06.0: startup error -19 [ 1.659106] xhci_hcd 0000:00:06.0: USB bus 2 deregistered [ 1.662286] xhci_hcd 0000:00:06.0: remove, state 1 [ 1.670295] xhci_hcd 0000:00:06.0: USB bus 1 deregistered [ 1.671337] xhci_hcd 0000:00:07.0: xHCI Host Controller [ 1.672184] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1 [ 1.678468] xhci_hcd 0000:00:07.0: hcc params 0x014051c7 hci version 0x100 quirks 0x0000000100000090 [ 1.680689] xhci_hcd 0000:00:07.0: xHCI Host Controller [ 1.681523] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 2 [ 1.682613] xhci_hcd 0000:00:07.0: Host supports USB 3.0 SuperSpeed [ 1.685692] xhci_hcd 0000:00:07.0: Host took too long to start, waited 32000 microseconds. [ 2.412808] xhci_hcd 0000:00:07.0: startup error -19 [ 2.414863] xhci_hcd 0000:00:07.0: USB bus 2 deregistered [ 2.417269] xhci_hcd 0000:00:07.0: remove, state 1 [ 2.425292] xhci_hcd 0000:00:07.0: USB bus 1 deregistered [ 2.428056] xhci_hcd 0000:00:09.0: xHCI Host Controller [ 2.430338] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 1 [ 2.435170] xhci_hcd 0000:00:09.0: hcc params 0x03401281 hci version 0x100 quirks 0x000000000000b930 [ 2.440101] xhci_hcd 0000:00:09.0: xHCI Host Controller [ 2.442350] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 2 [ 2.445446] xhci_hcd 0000:00:09.0: Host supports USB 3.0 SuperSpeed [ 2.454254] usb usb1: Product: xHCI Host Controller [ 2.456179] usb usb1: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.474092] usb usb2: Product: xHCI Host Controller [ 2.476031] usb usb2: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.704998] usb 2-1: new SuperSpeed USB device number 2 using xhci_hcd # dmesg | grep -i usb [ 0.310422] ACPI: bus type USB registered [ 0.311371] usbcore: registered new interface driver usbfs [ 0.312082] usbcore: registered new interface driver hub [ 0.312358] usbcore: registered new device driver usb [ 0.748133] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 1 [ 0.766148] xhci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 2 [ 0.769165] xhci_hcd 0000:00:06.0: Host supports USB 3.0 SuperSpeed [ 1.659106] xhci_hcd 0000:00:06.0: USB bus 2 deregistered [ 1.670295] xhci_hcd 0000:00:06.0: USB bus 1 deregistered [ 1.672184] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1 [ 1.681523] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 2 [ 1.682613] xhci_hcd 0000:00:07.0: Host supports USB 3.0 SuperSpeed [ 2.414863] xhci_hcd 0000:00:07.0: USB bus 2 deregistered [ 2.425292] xhci_hcd 0000:00:07.0: USB bus 1 deregistered [ 2.430338] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 1 [ 2.442350] xhci_hcd 0000:00:09.0: new USB bus registered, assigned bus number 2 [ 2.445446] xhci_hcd 0000:00:09.0: Host supports USB 3.0 SuperSpeed [ 2.448159] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.12 [ 2.451450] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.454254] usb usb1: Product: xHCI Host Controller [ 2.456179] usb usb1: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.458520] usb usb1: SerialNumber: 0000:00:09.0 [ 2.460786] hub 1-0:1.0: USB hub found [ 2.464830] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM. [ 2.468061] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.12 [ 2.471267] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.474092] usb usb2: Product: xHCI Host Controller [ 2.476031] usb usb2: Manufacturer: Linux 6.12.51-haos xhci-hcd [ 2.478394] usb usb2: SerialNumber: 0000:00:09.0 [ 2.480588] hub 2-0:1.0: USB hub found [ 2.484681] usbcore: registered new interface driver usblp [ 2.487332] usbcore: registered new interface driver usb-storage [ 2.489765] usbcore: registered new interface driver usbserial_generic [ 2.492948] usbserial: USB Serial support registered for generic [ 2.557619] usbcore: registered new interface driver usbhid [ 2.559706] usbhid: USB HID core driver [ 2.704998] usb 2-1: new SuperSpeed USB device number 2 using xhci_hcd [ 2.721813] usb 2-1: New USB device found, idVendor=fb5d, idProduct=0001, bcdDevice= 0.00 [ 2.725380] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 2.728498] usb 2-1: Product: HID Tablet [ 2.730828] usb 2-1: Manufacturer: BHYVE [ 2.733740] usb 2-1: SerialNumber: 01 [ 2.739082] input: BHYVE HID Tablet as /devices/pci0000:00/0000:00:09.0/usb2/2-1/2-1:1.0/0003:FB5D:0001.0001/input/input4 [ 2.745058] hid-generic 0003:FB5D:0001.0001: input,hidraw0: USB HID v10.01 Mouse [BHYVE HID Tablet] on usb-0000:00: So its like ppt driver not being initialized correctly for passing through 2 USB sticks that works on FreeBSD 14. I have confirmed same problem with FreeBSD 15 stable. My config on FreeBSD 15 host is: router:/usr/local/etc # grep ppt /boot/loader.conf pptdevs="9/0/0 10/0/0 66/0/0" router:/usr/local/etc # 9 and 10 are USB sticks for home assistant, 66 is nvidia GPU. router:/usr/local/etc # grep passthru /vm/ha/ha.conf passthru0="9/0/0" passthru1="10/0/0" router:/usr/local/etc # router:/usr/local/etc # tail -5 /vm/ubuntu/ubuntu.conf # Passthru nvidia card passthru0="66/0/0" bhyve_options="-A -H -P" router:/usr/local/etc # router:/usr/local/etc # tail -10 /vm/ha/vm-bhyve.log Nov 10 14:14:52: [primary disk dev: file] Nov 10 14:14:52: initialising network device tap1 Nov 10 14:14:52: setting mtu of tap1 to 9000 Nov 10 14:14:52: adding tap1 -> br0 (services addm) Nov 10 14:14:52: bring up tap1 -> br0 (services addm) Nov 10 14:14:52: booting Nov 10 14:14:52: [bhyve options: -c 10,sockets=1,cores=10 -m 16G -AHPw -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U a5811c38-567e-11ed-bdeb-e2335bf65f00 -u -S] Nov 10 14:14:52: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,nvme,/vm/ha/disk0.img -s 5:0,virtio-net,tap1,mac=58:9c:11:0c:5a:a9 -s 6:0,passthru,9/0/0 -s 7:0,passthru,10/0/0 -s 8:0,fbuf,tcp=192.168.0.1:5915 -s 9:0,xhci,tablet] Nov 10 14:14:52: [bhyve console: -l com1,stdio] Nov 10 14:14:52: starting bhyve (run 1) router:/usr/local/etc # router:/usr/local/etc # pciconf -lv|grep ppt ppt0@pci0:9:0:0: class=0x0c0330 rev=0x02 hdr=0x00 vendor=0x1912 device=0x0015 subvendor=0x1912 subdevice=0x0015 ppt1@pci0:10:0:0: class=0x0c0330 rev=0x02 hdr=0x00 vendor=0x1912 device=0x0015 subvendor=0x1912 subdevice=0x0015 ppt2@pci0:66:0:0: class=0x030200 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1db4 subvendor=0x10de subdevice=0x1214 router:/usr/local/etc # -- You are receiving this mail because: You are the assignee for the bug.
