December 22, 2019 10:33 AM, "Vít Šesták"
<> wrote:

> As far as I know/understand:
> * At start, all the PCI devices are assigned to dom0.
> * When a qube with an attached PCI device starts, dom0 assigns the PCI device 
> to the qube, so it is
> no longer attached to dom0. It never gets actually attached to dom0 
> automatically (until reboot).
> If it was, a malicious qube could for example flash a malicious firmware to a 
> USB device and shut
> down in order to connect the malicious device to dom0.
> * In order to have some additional protection, rd.qubes.hide_all_usb hides 
> all USB devices. That
> is, the USB PCI device is attached to dom0, but Linux ignores it, maybe it 
> blacklists related
> kernel modules.
> The behavior you have observed suggests that both detached USB controller and 
> ignored USB
> controller cause an issue. So, maybe the problem is not in the process of 
> detaching a controller
> that is being used, but rather in not having the controller available.
> Intel includes some USB controller in CPU and quick Googling suggests that so 
> does AMD. Maybe there
> is some AMD-specific code in Linux kernel that expects the USB controller to 
> be available for
> whatever weird reason. Yes, it sounds strange, but it is the least 
> implausible explanation I was
> able to find.
> Regards,
> Vít Šesták 'v6ak'

Thanks for the info. Yes, that sounds correct from what I could tell, too. More 
specifically, what rd.qubes.hide_all_usb actually does is looks for USB 
controllers device files in /sys/bus/pci, and then calls their driver/unbind, 
driver/new_slot, and driver/bind functions. So basically, it forces the Linux 
USB driver (xhci_pci, in my case) to detach from the USB controller, and then 
attaches it to Xen's pciback driver so that it can be used by sys-usb later 
(although I don't know if this step is actually necessary, since starting 
sys-usb without hide_all_usb works just fine). I'm assuming this step happens 
before udev trigger, which probes devices including USB devices. Maybe that's 
why the hook assigns pciback instead of just unbinding the USB driver - so udev 
doesn't see that the device has no driver and attempt to bind the USB driver to 
it again. Or maybe the act of binding pciback is what actually places the USB 
controller under IOMMU isolation by Xen (otherwise the USB controller could 
still perform a DMA attack even with no driver bound to it). 

I don't know which step -- unbinding xhci_pci, or binding pciback -- actually 
causes the crash in this case. I can say, however, that one of them does cause 
an immediate crash, before sys-usb ever starts or has a chance to take over the 
USB controllers.

The same hook does the same thing for networking devices as well, so that those 
are never exposed. In my case this doesn't cause a problem because both network 
cards are their own devices on their own busses and have their own IOMMU 
groups, unlike my USB controllers.

Here's the actual code from 


type getarg >/dev/null 2>&1 || . /lib/

# Find all networking devices currenly installed...
HIDE_PCI="`lspci -mm -n | grep '^[^ ]* "02'|awk '{print $1}'`"

# ... and optionally all USB controllers...
if getargbool 0 rd.qubes.hide_all_usb; then
    HIDE_PCI="$HIDE_PCI `lspci -mm -n | grep '^[^ ]* "0c03'|awk '{print $1}'`"

HIDE_PCI="$HIDE_PCI `getarg rd.qubes.hide_pci | tr ',' ' '`"

modprobe xen-pciback 2>/dev/null || :

# ... and hide them so that Dom0 doesn't load drivers for them
for dev in $HIDE_PCI; do
    if [ -e /sys/bus/pci/devices/$BDF/driver ]; then
        echo -n $BDF > /sys/bus/pci/devices/$BDF/driver/unbind
    echo -n $BDF > /sys/bus/pci/drivers/pciback/new_slot
    echo -n $BDF > /sys/bus/pci/drivers/pciback/bind

As for USB controllers being on the CPU, yes that's what I found as well; all 
current CPUs bundle their integrated USB controllers right on the chip. I don't 
think code in the Linux kernel expects the USB controllers to be available. 
Rather I think it has to do with IOMMU grouping, which tends to be structured 
differently for AMD than Intel. I'm going to start a new thread about that here 

You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
To view this discussion on the web visit

Reply via email to