PCI device assignment to Guest

2009-04-03 Thread Eric Liu


Anyone has experience to assign PCI-E based InfiniBand card to
guest OS(RHEL5U2 with kernel 2.6.18-92) on latest AMD with IOMMU
support. Host OS has kernel 2.6.29.

Steps I used:

 $ echo -n 8086 10de  /sys/bus/pci/drivers/pci-stub/new_id
 $ echo -n :00:19.0 /sys/bus/pci/drivers/e1000e/unbind
 $ echo -n :00:19.0 /sys/bus/pci/drivers/pci-stub/bind

Then
I started guest with -pcidevice host=id. After guest is started, it
successfully detected pci device with lspci command, however kernel
can't bring up the device. dmesg shows infiniband kernel module can't
detect infiniband card firmware properly then aborted. I think it is
KVM issue rather than infiniband kernel module issue. Can anyone
suggest?

Thanks
Eric
_
Rediscover HotmailĀ®: Get quick friend updates right in your inbox. 
http://windowslive.com/RediscoverHotmail?ocid=TXT_TAGLM_WL_HM_Rediscover_Updates1_042009--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PCI device assignment to Guest

2009-04-03 Thread Chris Wright
* Eric Liu (ericliu2...@hotmail.com) wrote:
 Anyone has experience to assign PCI-E based InfiniBand card to
 guest OS(RHEL5U2 with kernel 2.6.18-92) on latest AMD with IOMMU
 support. Host OS has kernel 2.6.29.
 
 Steps I used:
 
  $ echo -n 8086 10de  /sys/bus/pci/drivers/pci-stub/new_id
  $ echo -n :00:19.0 /sys/bus/pci/drivers/e1000e/unbind
  $ echo -n :00:19.0 /sys/bus/pci/drivers/pci-stub/bind

The steps above are specific to an e1000e device.

 Then
 I started guest with -pcidevice host=id. After guest is started, it
 successfully detected pci device with lspci command, however kernel
 can't bring up the device. dmesg shows infiniband kernel module can't
 detect infiniband card firmware properly then aborted. I think it is
 KVM issue rather than infiniband kernel module issue. Can anyone
 suggest?

Sounds like you may have two drivers for this device.

Can you include (on the host) lspci -tv and lspci -vvv?

thanks,
-chris
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


RE: PCI device assignment to Guest

2009-04-03 Thread Eric Liu



Here are the exact steps I used:

1. lspci -n on host:
06:00.0 0c06: 15b3:634a (rev a0)


I want to assign this device to guest.

2. Uninstall driver for this device.

3. Unbind device with the following commands:

echo 15b3 634a /sys/bus/pci/drivers/pci-stub/new_id
echo :06:00.0 /sys/bus/pci/devices/:06:00.0/driver/unbind
echo :06:00.0 /sys/bus/pci/drivers/pci-stub/bind
4. start guest with... -pcidevice host=06:00.0
5. Guest os detects device with lspci command but failed to start.

lspci -tv on host:(last device here is what i want to assign)

-[:00]-+-01.0-[:03-04]--+-0d.0-[:04]--
   |\-0e.0  Broadcom BCM5785 [HT1000] SATA 
(PATA/IDE Mode)
   +-02.0  Broadcom BCM5785 [HT1000] Legacy South Bridge
   +-02.1  Broadcom BCM5785 [HT1000] IDE
   +-02.2  Broadcom BCM5785 [HT1000] LPC
   +-03.0  Broadcom BCM5785 [HT1000] USB
   +-03.1  Broadcom BCM5785 [HT1000] USB
   +-03.2  Broadcom BCM5785 [HT1000] USB
   +-04.0  ATI Technologies Inc ES1000
   +-07.0-[:05]--
   +-08.0-[:01]00.0  Broadcom Corporation NetXtreme BCM5721 
Gigabit Ethernet PCI Express
   +-09.0-[:02]00.0  Broadcom Corporation NetXtreme BCM5721 
Gigabit Ethernet PCI Express
   +-0a.0-[:06]00.0  Mellanox Technologies MT25418 [ConnectX IB 
DDR]



lspci -vvv: (only related portion)

06:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX IB DDR] (rev a0)
Subsystem: Mellanox Technologies Unknown device 0007
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fastTAbort- SERR-  
Date: Fri, 3 Apr 2009 10:13:22 -0700
 From: chr...@sous-sol.org
 To: ericliu2...@hotmail.com
 CC: kvm@vger.kernel.org
 Subject: Re: PCI device assignment to Guest

 * Eric Liu (ericliu2...@hotmail.com) wrote:
 Anyone has experience to assign PCI-E based InfiniBand card to
 guest OS(RHEL5U2 with kernel 2.6.18-92) on latest AMD with IOMMU
 support. Host OS has kernel 2.6.29.

 Steps I used:

 $ echo -n 8086 10de /sys/bus/pci/drivers/pci-stub/new_id
 $ echo -n :00:19.0 /sys/bus/pci/drivers/e1000e/unbind
 $ echo -n :00:19.0 /sys/bus/pci/drivers/pci-stub/bind

 The steps above are specific to an e1000e device.

 Then
 I started guest with -pcidevice host=id. After guest is started, it
 successfully detected pci device with lspci command, however kernel
 can't bring up the device. dmesg shows infiniband kernel module can't
 detect infiniband card firmware properly then aborted. I think it is
 KVM issue rather than infiniband kernel module issue. Can anyone
 suggest?

 Sounds like you may have two drivers for this device.

 Can you include (on the host) lspci -tv and lspci -vvv?

 thanks,
 -chris

_
Windows Liveā„¢: Keep your life in sync.
http://windowslive.com/explore?ocid=TXT_TAGLM_WL_allup_1a_explore_042009--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html