Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
Hello kvm-list! I sent the following text to Don with some additional detailed log files. If somebody else need them too, I can provide them with pn. Kind regards, Andreas Hartmann Andreas Hartmann schrieb: > Hello Don, > > thank you for your reply! > > I just want to describe in short the two problems I encounter (my > description was a bit chaotic :-) I hope it's better now :-)). I > attached details in two files (dmesg.bz2 and lspci.bz2. It's the > untouched raw output - no unbind or anything else had been done). > > General hardware setup for a quick overview: > > -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external > gfx0 port B) >+-00.2 ATI Technologies Inc Device 5a23 >+-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 >|\-00.1 ATI Technologies Inc Device aa90 >+-04.0-[02]00.0 Device 1b6f:7023 >+-05.0-[03]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B > PCI Express Gigabit Ethernet controller >+-09.0-[04]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B > PCI Express Gigabit Ethernet controller >+-0a.0-[05]00.0 Device 1b6f:7023 >+-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI > mode] >+-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller >+-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller >+-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller >+-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller >+-14.0 ATI Technologies Inc SBx00 SMBus Controller >+-14.1 ATI Technologies Inc SB700/SB800 IDE Controller >+-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) >+-14.3 ATI Technologies Inc SB700/SB800 LPC host controller >+-14.4-[06]07.0 RaLink RT2800 802.11n PCI >+-14.5 ATI Technologies Inc SB700/SB800 USB OHCI2 Controller >+-15.0-[07]-- >+-16.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller >+-16.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller >+-18.0 Advanced Micro Devices [AMD] Device 1600 >+-18.1 Advanced Micro Devices [AMD] Device 1601 >+-18.2 Advanced Micro Devices [AMD] Device 1602 >+-18.3 Advanced Micro Devices [AMD] Device 1603 >+-18.4 Advanced Micro Devices [AMD] Device 1604 >\-18.5 Advanced Micro Devices [AMD] Device 1605 > > The relevant devices are (for problem 1): > > 06:07.0 Network controller: RaLink RT2800 802.11n PCI > Subsystem: Linksys Device 0067 > Flags: bus master, slow devsel, latency 32, IRQ 21 > Memory at fd8e (32-bit, non-prefetchable) [size=64K] > Capabilities: [40] Power Management version 3 > Kernel driver in use: rt2800pci > > with the PCI-PCI bridge above: > > 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) > (prog-if 01 [Subtractive decode]) > Flags: bus master, VGA palette snoop, 66MHz, medium devsel, latency 64 > Bus: primary=00, secondary=06, subordinate=06, sec-latency=64 > I/O behind bridge: 9000-9fff > Memory behind bridge: fd80-fd8f > Prefetchable memory behind bridge: fd70-fd7f > > > > and (for problem 2): > > 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B > PCI Express Gigabit Ethernet controller (rev 01) > Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express > Gigabit Ethernet controller > Flags: bus master, fast devsel, latency 0, IRQ 10 > I/O ports at de00 [size=256] > Memory at fdbff000 (64-bit, non-prefetchable) [size=4K] > [virtual] Expansion ROM at fda0 [disabled] [size=128K] > Capabilities: [40] Power Management version 2 > Capabilities: [48] Vital Product Data > Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+ > Capabilities: [60] Express Endpoint, MSI 00 > Capabilities: [84] Vendor Specific Information: Len=4c > Capabilities: [100] Advanced Error Reporting > Capabilities: [12c] Virtual Channel > Capabilities: [148] Device Serial Number f1-11-00-00-68-4c-e0-00 > Capabilities: [154] Power Budgeting > > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B > PCI Express Gigabit Ethernet controller (rev 06) > Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard > Flags: bus master, fast devsel, latency 0, IRQ 10 > I/O ports at ce00 [size=256] > Memory at fd6ff000 (64-bit, prefetchable) [size=4K] > Memory at fd6f8000 (64-bit, prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ > Capabilities: [70] Express E
Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
On 12/13/2011 03:40 AM, Andreas Hartmann wrote: Hello Don! Andreas, sorry for the delay, I don't follow this email list frequently (read: !daily). Am Tue, 13 Dec 2011 01:21:41 +0100 schrieb Andreas Hartmann: Am Mon, 12 Dec 2011 13:36:36 -0500 schrieb Don Dutile: On 12/12/2011 06:15 AM, Andreas Hartmann wrote: Hello! I've got a few questions to a problem, which already was analyzed here sometime ago: http://markmail.org/message/dspovwvzp3wtdrf6#query:+page:1+mid:i2oph4xwfmiknt3y+state:results My situation is a bit different. I do have two PCI cards (a Linksys wlan card and an intel e100 card). Each of these cards should be managed in an own VM. I do have no problems with IRQ sharing (each device does have its own IRQ). I'm using linux 3.0.6, kvm 0.15 and libvirt 0.9.7. Mainboard is: GA-990XA-UD3. The problem is: both cards are behind a PCI-PCI bridge: 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) +-00.2 ATI Technologies Inc Device 5a23 +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 |\-00.1 ATI Technologies Inc Device aa90 +-04.0-[02]00.0 Device 1b6f:7023 +-09.0-[03]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller +-0a.0-[04]00.0 Device 1b6f:7023 +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-14.0 ATI Technologies Inc SBx00 SMBus Controller +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller +-14.4-[05]--+-06.0 Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 |\-07.0 RaLink RT2800 802.11n PCI You cannot assign two devices behind a (legacy) PCI (not PCIe) bridge to two different VMs because PCI devices dont provide a B:D.F in their transaction headers; In this case, the PCIe-PCI bridge/switch prepends the bridge's B:D.F in front of a transaction that is sourced by 05:06.0 or 05:07.0 . So, the devices cannot be isolated from each other's (mem) mapping domains in the IOMMU, thus, libvirt prevents this (security) violation from being enabled. Ok. If I remove the intel card and put in instead a 32 bit PCIe card like TP-Link TG-3468, I could assign each of these two cards to different VMs. KISS: Don't try to assign legacy PCI devices; just PCIe. ;-) Where is "TP-Link TG-3468" in lspci output below? the AMD devices in 18.0->18.5 ??? Is this correct? [...] 1. Is there any way to get the devices into different VMs? (I can't put them to another PCI slot as there are just 2 PCI slots on the board.) Only if the two slots are behind different PCIe-PCI bridges . 2. Is there any fix to prevent the host crash - maybe in a newer version of kvm or kernel? don't assign the PCI bridge; libvirt does all the proper bind/unbinding of devices for assigned devices, so the above echo steps are unnecessary if you use libvirt. If I just remove the intel card, I should be able to assign the wlan card to one of my VMs. I'll try that. Meanwhile I checked this scenario: I removed the intel card and rebooted. I got the following pci tree: -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) +-00.2 ATI Technologies Inc Device 5a23 +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 |\-00.1 ATI Technologies Inc Device aa90 +-04.0-[02]00.0 Device 1b6f:7023 +-09.0-[03]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller +-0a.0-[04]00.0 Device 1b6f:7023 +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-14.0 ATI Technologies Inc SBx00 SMBus Controller +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller +-14.4-[05]07.0 RaLink RT2800 802.11n PCI +-14.5 ATI Technologies Inc SB700/SB800 USB OHCI2 Control
Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
Hello Don! Am Tue, 13 Dec 2011 01:21:41 +0100 schrieb Andreas Hartmann : [...] > Ok. If I remove the intel card and put in instead a 32 bit PCIe card > like TP-Link TG-3468, I could assign each of these two cards to > different VMs. > > Is this correct? No - it isn't correct - but don't now why! 1. The legacy pci card, even if it's the only legacy one, can't be used, because it crashes the machine after removing the VM (see other post). 2. The TP-Link TG-3468 card crashes the host at the moment of "virsh start VM". 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01) Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller Flags: bus master, fast devsel, latency 0, IRQ 43 I/O ports at de00 [size=256] Memory at fdbff000 (64-bit, non-prefetchable) [size=4K] [virtual] Expansion ROM at fda0 [disabled] [size=128K] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+ Capabilities: [60] Express Endpoint, MSI 00 Capabilities: [84] Vendor Specific Information: Len=4c Capabilities: [100] Advanced Error Reporting Capabilities: [12c] Virtual Channel Capabilities: [148] Device Serial Number f1-11-00-00-68-4c-e0-00 Capabilities: [154] Power Budgeting Kernel driver in use: r8169 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06) Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard Flags: bus master, fast devsel, latency 0, IRQ 44 I/O ports at ce00 [size=256] Memory at fd6ff000 (64-bit, prefetchable) [size=4K] Memory at fd6f8000 (64-bit, prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Capabilities: [70] Express Endpoint, MSI 01 Capabilities: [b0] MSI-X: Enable- Count=4 Masked- Capabilities: [d0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [140] Virtual Channel Capabilities: [160] Device Serial Number 12-34-56-78-12-34-56-78 Kernel driver in use: r8169 First problem: I don't know, how to distinguish these two cards. They use both the same module and they have equal PCI-ID's (but different bus ID's). If the desired one (03:00.0) is manually unbound with these lines echo "10ec 8168" > /sys/bus/pci/drivers/pci-stub/new_id echo ":03:00.0" > /sys/bus/pci/devices/\:03\:00.0/driver/unbind echo ":03:00.0" > /sys/bus/pci/drivers/pci-stub/bind both devices are unbound (this is definitely wrong) and both devices use the same IRQ (as they were managed by r8169, they had different IRQs): 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01) Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller Flags: fast devsel, IRQ 17 I/O ports at de00 [size=256] Memory at fdbff000 (64-bit, non-prefetchable) [size=4K] [virtual] Expansion ROM at fda0 [disabled] [size=128K] Capabilities: [40] Power Management version 2 Capabilities: [48] Vital Product Data Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+ Capabilities: [60] Express Endpoint, MSI 00 Capabilities: [84] Vendor Specific Information: Len=4c Capabilities: [100] Advanced Error Reporting Capabilities: [12c] Virtual Channel Capabilities: [148] Device Serial Number f1-11-00-00-68-4c-e0-00 Capabilities: [154] Power Budgeting Kernel driver in use: pci-stub 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06) Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard Flags: fast devsel, IRQ 17 I/O ports at ce00 [size=256] Memory at fd6ff000 (64-bit, prefetchable) [size=4K] Memory at fd6f8000 (64-bit, prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [70] Express Endpoint, MSI 01 Capabilities: [b0] MSI-X: Enable- Count=4 Masked- Capabilities: [d0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [140] Virtual Channel Capabilities: [160] Device Serial Number 12-34-56-78-12-34-56-78 Kernel driver in use: pci-stub Second problem: If I do a "modprobe r8169", both devices are "adressed". How can I address just one of both? Sorry for all of these questions :-). I really would be glad, to get an answer! Kind
Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
Hello Don! Some additional information about the bridge device 00:14.4 itself (both legacy PCI cards are behind this bridge device - it's an output with both legacy devices plugged in): 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) (prog-if 01 [Subtractive decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop+ ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- If the bridge is put to the VM too, libvirt exits with the following error: error: internal error Unable to reset PCI device :00:14.4: no FLR, PM reset or bus reset available The wlan device looks like this: 05:07.0 Network controller: RaLink RT2800 802.11n PCI Subsystem: Linksys Device 0067 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- SERR- http://vger.kernel.org/majordomo-info.html
Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
Hello Don! Am Tue, 13 Dec 2011 01:21:41 +0100 schrieb Andreas Hartmann : > Am Mon, 12 Dec 2011 13:36:36 -0500 > schrieb Don Dutile : > > > On 12/12/2011 06:15 AM, Andreas Hartmann wrote: > > > Hello! > > > > > > I've got a few questions to a problem, which already was analyzed here > > > sometime ago: > > > http://markmail.org/message/dspovwvzp3wtdrf6#query:+page:1+mid:i2oph4xwfmiknt3y+state:results > > > > > > My situation is a bit different. I do have two PCI cards (a Linksys wlan > > > card and an intel e100 card). Each of these cards should be managed in > > > an own VM. I do have no problems with IRQ sharing (each device does have > > > its own IRQ). > > > > > > I'm using linux 3.0.6, kvm 0.15 and libvirt 0.9.7. Mainboard is: > > > GA-990XA-UD3. > > > > > > > > > The problem is: both cards are behind a PCI-PCI bridge: > > > > > > 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) > > > > > > > > > -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external > > > gfx0 port B) > > > +-00.2 ATI Technologies Inc Device 5a23 > > > +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 > > > |\-00.1 ATI Technologies Inc Device aa90 > > > +-04.0-[02]00.0 Device 1b6f:7023 > > > +-09.0-[03]00.0 Realtek Semiconductor Co., Ltd. > > > RTL8111/8168B PCI Express Gigabit Ethernet controller > > > +-0a.0-[04]00.0 Device 1b6f:7023 > > > +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller > > > [AHCI mode] > > > +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > > > +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > > > +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > > > +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > > > +-14.0 ATI Technologies Inc SBx00 SMBus Controller > > > +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller > > > +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) > > > +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller > > > > > > +-14.4-[05]--+-06.0 Intel Corporation 82557/8/9/0/1 Ethernet > > > Pro 100 > > > |\-07.0 RaLink RT2800 802.11n PCI > > > > > You cannot assign two devices behind a (legacy) PCI (not PCIe) bridge to two > > different VMs because PCI devices dont provide a B:D.F in their transaction > > headers; > > In this case, the PCIe-PCI bridge/switch prepends the bridge's B:D.F in > > front of > > a transaction that is sourced by 05:06.0 or 05:07.0 . > > > > So, the devices cannot be isolated from each other's (mem) mapping domains > > in the > > IOMMU, thus, libvirt prevents this (security) violation from being enabled. > > Ok. If I remove the intel card and put in instead a 32 bit PCIe card > like TP-Link TG-3468, I could assign each of these two cards to > different VMs. > > Is this correct? > > [...] > > > > 1. Is there any way to get the devices into different VMs? (I can't put > > > them to another PCI slot as there are just 2 PCI slots on the board.) > > Only if the two slots are behind different PCIe-PCI bridges . > > > > > 2. Is there any fix to prevent the host crash - maybe in a newer version > > > of kvm or kernel? > > don't assign the PCI bridge; libvirt does all the proper bind/unbinding of > > devices > > for assigned devices, so the above echo steps are unnecessary if you use > > libvirt. > > If I just remove the intel card, I should be able to assign the wlan > card to one of my VMs. I'll try that. Meanwhile I checked this scenario: I removed the intel card and rebooted. I got the following pci tree: -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) +-00.2 ATI Technologies Inc Device 5a23 +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 |\-00.1 ATI Technologies Inc Device aa90 +-04.0-[02]00.0 Device 1b6f:7023 +-09.0-[03]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller +-0a.0-[04]00.0 Device 1b6f:7023 +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-14.0 ATI Technologies Inc SBx00 SMBus Controller +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller +-14.4-[05]07.0 RaLink RT2800 802.11n PCI +-14.5 ATI Tech
Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
Am Mon, 12 Dec 2011 13:36:36 -0500 schrieb Don Dutile : > On 12/12/2011 06:15 AM, Andreas Hartmann wrote: > > Hello! > > > > I've got a few questions to a problem, which already was analyzed here > > sometime ago: > > http://markmail.org/message/dspovwvzp3wtdrf6#query:+page:1+mid:i2oph4xwfmiknt3y+state:results > > > > My situation is a bit different. I do have two PCI cards (a Linksys wlan > > card and an intel e100 card). Each of these cards should be managed in > > an own VM. I do have no problems with IRQ sharing (each device does have > > its own IRQ). > > > > I'm using linux 3.0.6, kvm 0.15 and libvirt 0.9.7. Mainboard is: > > GA-990XA-UD3. > > > > > > The problem is: both cards are behind a PCI-PCI bridge: > > > > 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) > > > > > > -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external > > gfx0 port B) > > +-00.2 ATI Technologies Inc Device 5a23 > > +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 > > |\-00.1 ATI Technologies Inc Device aa90 > > +-04.0-[02]00.0 Device 1b6f:7023 > > +-09.0-[03]00.0 Realtek Semiconductor Co., Ltd. > > RTL8111/8168B PCI Express Gigabit Ethernet controller > > +-0a.0-[04]00.0 Device 1b6f:7023 > > +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI > > mode] > > +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > > +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > > +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > > +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > > +-14.0 ATI Technologies Inc SBx00 SMBus Controller > > +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller > > +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) > > +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller > > > > +-14.4-[05]--+-06.0 Intel Corporation 82557/8/9/0/1 Ethernet > > Pro 100 > > |\-07.0 RaLink RT2800 802.11n PCI > > > You cannot assign two devices behind a (legacy) PCI (not PCIe) bridge to two > different VMs because PCI devices dont provide a B:D.F in their transaction > headers; > In this case, the PCIe-PCI bridge/switch prepends the bridge's B:D.F in front > of > a transaction that is sourced by 05:06.0 or 05:07.0 . > > So, the devices cannot be isolated from each other's (mem) mapping domains in > the > IOMMU, thus, libvirt prevents this (security) violation from being enabled. Ok. If I remove the intel card and put in instead a 32 bit PCIe card like TP-Link TG-3468, I could assign each of these two cards to different VMs. Is this correct? [...] > > 1. Is there any way to get the devices into different VMs? (I can't put > > them to another PCI slot as there are just 2 PCI slots on the board.) > Only if the two slots are behind different PCIe-PCI bridges . > > > 2. Is there any fix to prevent the host crash - maybe in a newer version of > > kvm or kernel? > don't assign the PCI bridge; libvirt does all the proper bind/unbinding of > devices > for assigned devices, so the above echo steps are unnecessary if you use > libvirt. If I just remove the intel card, I should be able to assign the wlan card to one of my VMs. I'll try that. Thank you for your advice! Kind regards, Andreas -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [AMD iommu] pci Failed to assign device "hostdev0" : Device or resource busy
On 12/12/2011 06:15 AM, Andreas Hartmann wrote: Hello! I've got a few questions to a problem, which already was analyzed here sometime ago: http://markmail.org/message/dspovwvzp3wtdrf6#query:+page:1+mid:i2oph4xwfmiknt3y+state:results My situation is a bit different. I do have two PCI cards (a Linksys wlan card and an intel e100 card). Each of these cards should be managed in an own VM. I do have no problems with IRQ sharing (each device does have its own IRQ). I'm using linux 3.0.6, kvm 0.15 and libvirt 0.9.7. Mainboard is: GA-990XA-UD3. The problem is: both cards are behind a PCI-PCI bridge: 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) -[:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) +-00.2 ATI Technologies Inc Device 5a23 +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 |\-00.1 ATI Technologies Inc Device aa90 +-04.0-[02]00.0 Device 1b6f:7023 +-09.0-[03]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller +-0a.0-[04]00.0 Device 1b6f:7023 +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-14.0 ATI Technologies Inc SBx00 SMBus Controller +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller +-14.4-[05]--+-06.0 Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 |\-07.0 RaLink RT2800 802.11n PCI You cannot assign two devices behind a (legacy) PCI (not PCIe) bridge to two different VMs because PCI devices dont provide a B:D.F in their transaction headers; In this case, the PCIe-PCI bridge/switch prepends the bridge's B:D.F in front of a transaction that is sourced by 05:06.0 or 05:07.0 . So, the devices cannot be isolated from each other's (mem) mapping domains in the IOMMU, thus, libvirt prevents this (security) violation from being enabled. +-14.5 ATI Technologies Inc SB700/SB800 USB OHCI2 Controller +-15.0-[06]-- +-16.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-16.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-18.0 Advanced Micro Devices [AMD] Device 1600 +-18.1 Advanced Micro Devices [AMD] Device 1601 +-18.2 Advanced Micro Devices [AMD] Device 1602 +-18.3 Advanced Micro Devices [AMD] Device 1603 +-18.4 Advanced Micro Devices [AMD] Device 1604 \-18.5 Advanced Micro Devices [AMD] Device 1605 That's what can be seen during boot: [0.621901] AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300 [0.621906] AMD-Vi:mmio-addr: fec3 [0.622091] AMD-Vi: DEV_SELECT_RANGE_START devid: 00:00.0 flags: 00 [0.622095] AMD-Vi: DEV_RANGE_END devid: 00:00.2 [0.622097] AMD-Vi: DEV_SELECT devid: 00:02.0 flags: 00 [0.622100] AMD-Vi: DEV_SELECT_RANGE_START devid: 01:00.0 flags: 00 [0.622102] AMD-Vi: DEV_RANGE_END devid: 01:00.1 [0.622105] AMD-Vi: DEV_SELECT devid: 00:04.0 flags: 00 [0.622107] AMD-Vi: DEV_SELECT devid: 02:00.0 flags: 00 [0.622109] AMD-Vi: DEV_SELECT devid: 00:09.0 flags: 00 [0.622112] AMD-Vi: DEV_SELECT devid: 03:00.0 flags: 00 [0.622114] AMD-Vi: DEV_SELECT devid: 00:0a.0 flags: 00 [0.622117] AMD-Vi: DEV_SELECT devid: 04:00.0 flags: 00 [0.622119] AMD-Vi: DEV_SELECT devid: 00:11.0 flags: 00 [0.622122] AMD-Vi: DEV_SELECT_RANGE_START devid: 00:12.0 flags: 00 [0.622124] AMD-Vi: DEV_RANGE_END devid: 00:12.2 [0.622127] AMD-Vi: DEV_SELECT_RANGE_START devid: 00:13.0 flags: 00 [0.622129] AMD-Vi: DEV_RANGE_END devid: 00:13.2 [0.622132] AMD-Vi: DEV_SELECT devid: 00:14.0 flags: d7 [0.622134] AMD-Vi: DEV_SELECT devid: 00:14.1 flags: 00 [0.622137] AMD-Vi: DEV_SELECT devid: 00:14.2 flags: 00 [0.622139] AMD-Vi: DEV_SELECT devid: 00:14.3 flags: 00 [0.622141] AMD-Vi: DEV_SELECT devid: 00:14.4 flags: 00 [0.622144] AMD-Vi: DEV_ALIAS_RANGE devid: 05:00.0 flags: 00 devid_to: 00:14.4 [0.622147] AMD-Vi: DEV_RANGE_END dev
