On Tuesday 29 January 2013 17:18, Mathis GAVILLON wrote: > Support for interrupt remapping should be now available (kernel has > been recompiled with but is there any way to check if it's available > ?). > I tried to do what you say in step 2 (num_vfs=1 and probe_vf=0) but it > doesn't change anything. I also tried num_vfs=2 and probe_vf=1 but > nothing is better. > Nevertheless, a strange information message appear in shell when I > boot my guest : > > Message from syslogd@localhost at Jan 29 15:51:49 ... > kernel:Disabling IRQ #16 > Or, Mathis,
This now looks like the problem that we had with a qemu change. qemu-kvm-0.12.1.2-2.348.el6 (or possibly earlier) was changed from the previous qemu version so that the pci header for the guest was not updated to indicate MSI_X enabled when the guest called pci_enable_msix() -- rather, a "lazy update" policy was introduced which did not update the flag in the Hypervisor until the guest called request_irq(). This caused problems for some vendor's network sriov drivers, and RH implemented instead a hybrid policy, compatible with their previous policy, which did not break any sriov drivers. The updated qemu module for RH is: qemu-kvm-0.12.1.2-2.349.el6 To see which version of qemu you have installed, just enter the following command: rpm -qvia | grep qemu I found a web site which has the rpm's of interest (for fedora core 18): http://rpmfind.net/linux/rpm2html/search.php?query=qemu-kvm You might try using the following rpm (built Jan 16): ftp://rpmfind.net/linux/fedora/linux/updates/18/x86_64/qemu-kvm-1.2.2-2.fc18.x86_64.rpm (which probably has the fix) If that does not work, you might try: ftp://rpmfind.net/linux/fedora/linux/development/rawhide/x86_64/os/Packages/q/qemu-kvm-1.3.0-5.fc19.x86_64.rpm -Jack > I put dmesg log from step 2 (host and guest) again : > > [ 1419.382452] mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011) > [ 1419.382454] mlx4_core: Initializing 0000:05:00.0 > [ 1419.382653] mlx4_core 0000:05:00.0: Enabling SR-IOV with 1 VFs > [ 1419.483687] pci 0000:05:00.1: [15b3:1002] type 00 class 0x0c0600 > [ 1419.484427] mlx4_core 0000:05:00.0: Running in master mode > [ 1420.486608] mlx4_core 0000:05:00.0: FW version 2.9.1200 (cmd intf > rev 3), max commands 16 > [ 1420.486610] mlx4_core 0000:05:00.0: Catastrophic error buffer at > 0x1f020, size 0x10, BAR 0 > [ 1420.486612] mlx4_core 0000:05:00.0: Communication vector bar:2 offset:0x800 > [ 1420.486613] mlx4_core 0000:05:00.0: FW size 385 KB > [ 1420.486615] mlx4_core 0000:05:00.0: Clear int @ f0058, BAR 0 > [ 1420.488474] mlx4_core 0000:05:00.0: Mapped 26 chunks/6168 KB for FW. > [ 1421.155871] mlx4_core 0000:05:00.0: BlueFlame available (reg size > 512, regs/page 8) > [ 1421.156035] mlx4_core 0000:05:00.0: Base MM extensions: flags > 00000cc0, rsvd L_Key 00000500 > [ 1421.156036] mlx4_core 0000:05:00.0: Max ICM size 4294967296 MB > [ 1421.156038] mlx4_core 0000:05:00.0: Max QPs: 16777216, reserved > QPs: 64, entry size: 256 > [ 1421.156040] mlx4_core 0000:05:00.0: Max SRQs: 16777216, reserved > SRQs: 64, entry size: 128 > [ 1421.156041] mlx4_core 0000:05:00.0: Max CQs: 16777216, reserved > CQs: 128, entry size: 128 > [ 1421.156043] mlx4_core 0000:05:00.0: Max EQs: 32, reserved EQs: 28, > entry size: 128 > [ 1421.156045] mlx4_core 0000:05:00.0: reserved MPTs: 16, reserved MTTs: 16 > [ 1421.156046] mlx4_core 0000:05:00.0: Max PDs: 131072, reserved PDs: > 4, reserved UARs: 7 > [ 1421.156048] mlx4_core 0000:05:00.0: Max QP/MCG: 131072, reserved MGMs: 0 > [ 1421.156049] mlx4_core 0000:05:00.0: Max CQEs: 4194304, max WQEs: > 16384, max SRQ WQEs: 16384 > [ 1421.156051] mlx4_core 0000:05:00.0: Local CA ACK delay: 15, max > MTU: 4096, port width cap: 3 > [ 1421.156052] mlx4_core 0000:05:00.0: Max SQ desc size: 1008, max SQ S/G: 62 > [ 1421.156054] mlx4_core 0000:05:00.0: Max RQ desc size: 512, max RQ S/G: 32 > [ 1421.156055] mlx4_core 0000:05:00.0: Max GSO size: 131072 > [ 1421.156056] mlx4_core 0000:05:00.0: Max counters: 256 > [ 1421.156058] mlx4_core 0000:05:00.0: Max RSS Table size: 256 > [ 1421.156059] mlx4_core 0000:05:00.0: DEV_CAP flags: > [ 1421.156060] mlx4_core 0000:05:00.0: RC transport > [ 1421.156062] mlx4_core 0000:05:00.0: UC transport > [ 1421.156063] mlx4_core 0000:05:00.0: UD transport > [ 1421.156064] mlx4_core 0000:05:00.0: XRC transport > [ 1421.156065] mlx4_core 0000:05:00.0: FCoIB support > [ 1421.156066] mlx4_core 0000:05:00.0: SRQ support > [ 1421.156068] mlx4_core 0000:05:00.0: IPoIB checksum offload > [ 1421.156069] mlx4_core 0000:05:00.0: P_Key violation counter > [ 1421.156070] mlx4_core 0000:05:00.0: Q_Key violation counter > [ 1421.156071] mlx4_core 0000:05:00.0: DPDP > [ 1421.156073] mlx4_core 0000:05:00.0: Big LSO headers > [ 1421.156074] mlx4_core 0000:05:00.0: APM support > [ 1421.156075] mlx4_core 0000:05:00.0: Atomic ops support > [ 1421.156077] mlx4_core 0000:05:00.0: Address vector port checking > support > [ 1421.156078] mlx4_core 0000:05:00.0: UD multicast support > [ 1421.156079] mlx4_core 0000:05:00.0: Router support > [ 1421.156080] mlx4_core 0000:05:00.0: IBoE support > [ 1421.156082] mlx4_core 0000:05:00.0: Unicast loopback support > [ 1421.156083] mlx4_core 0000:05:00.0: UDP RSS support > [ 1421.156084] mlx4_core 0000:05:00.0: Unicast VEP steering support > [ 1421.156085] mlx4_core 0000:05:00.0: Multicast VEP steering support > [ 1421.156087] mlx4_core 0000:05:00.0: Counters support > [ 1421.156088] mlx4_core 0000:05:00.0: RSS support > [ 1421.156090] mlx4_core 0000:05:00.0: RSS Toeplitz Hash Function support > [ 1421.156091] mlx4_core 0000:05:00.0: RSS XOR Hash Function support > [ 1421.156093] mlx4_core 0000:05:00.0: Steering mode is: B0 steering, > oper_log_mgm_entry_size = 10, modparam log_num_mgm_entry_size = 10 > [ 1421.156099] mlx4_core 0000:05:00.0: profile[ 0] ( CMPT): 2^26 > entries @ 0x 0, size 0x 100000000 > [ 1421.156101] mlx4_core 0000:05:00.0: profile[ 1] (RDMARC): 2^22 > entries @ 0x 100000000, size 0x 8000000 > [ 1421.156103] mlx4_core 0000:05:00.0: profile[ 2] ( MTT): 2^24 > entries @ 0x 108000000, size 0x 8000000 > [ 1421.156105] mlx4_core 0000:05:00.0: profile[ 3] ( QP): 2^18 > entries @ 0x 110000000, size 0x 4000000 > [ 1421.156107] mlx4_core 0000:05:00.0: profile[ 4] ( DMPT): 2^19 > entries @ 0x 114000000, size 0x 2000000 > [ 1421.156109] mlx4_core 0000:05:00.0: profile[ 5] ( ALTC): 2^18 > entries @ 0x 116000000, size 0x 1000000 > [ 1421.156111] mlx4_core 0000:05:00.0: profile[ 6] ( SRQ): 2^16 > entries @ 0x 117000000, size 0x 800000 > [ 1421.156113] mlx4_core 0000:05:00.0: profile[ 7] ( CQ): 2^16 > entries @ 0x 117800000, size 0x 800000 > [ 1421.156115] mlx4_core 0000:05:00.0: profile[ 8] ( MCG): 2^13 > entries @ 0x 118000000, size 0x 800000 > [ 1421.156117] mlx4_core 0000:05:00.0: profile[ 9] ( AUXC): 2^18 > entries @ 0x 118800000, size 0x 40000 > [ 1421.156119] mlx4_core 0000:05:00.0: profile[10] ( EQ): 2^10 > entries @ 0x 118840000, size 0x 20000 > [ 1421.156120] mlx4_core 0000:05:00.0: HCA context memory: reserving 4596096 > KB > [ 1421.156136] mlx4_core 0000:05:00.0: 4596096 KB of HCA context > requires 9068 KB aux memory. > [ 1421.185505] mlx4_core 0000:05:00.0: Mapped 39 chunks/9068 KB for ICM aux. > [ 1421.186824] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at 0 for ICM. > [ 1421.188122] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 40000000 for ICM. > [ 1421.189418] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 80000000 for ICM. > [ 1421.189754] mlx4_core 0000:05:00.0: Mapped 1 chunks/64 KB at > c0000000 for ICM. > [ 1421.190400] mlx4_core 0000:05:00.0: Mapped 1 chunks/128 KB at > 118840000 for ICM. > [ 1421.191677] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 108000000 for ICM. > [ 1421.193001] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 114000000 for ICM. > [ 1421.194279] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 110000000 for ICM. > [ 1421.195563] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118800000 for ICM. > [ 1421.196841] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 116000000 for ICM. > [ 1421.198127] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 100000000 for ICM. > [ 1421.199405] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 117800000 for ICM. > [ 1421.200684] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 117000000 for ICM. > [ 1421.201964] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118000000 for ICM. > [ 1421.203240] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118040000 for ICM. > [ 1421.204517] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118080000 for ICM. > [ 1421.205802] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1180c0000 for ICM. > [ 1421.207079] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118100000 for ICM. > [ 1421.208356] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118140000 for ICM. > [ 1421.209633] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118180000 for ICM. > [ 1421.210912] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1181c0000 for ICM. > [ 1421.212191] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118200000 for ICM. > [ 1421.213465] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118240000 for ICM. > [ 1421.214742] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118280000 for ICM. > [ 1421.216024] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1182c0000 for ICM. > [ 1421.217303] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118300000 for ICM. > [ 1421.218580] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118340000 for ICM. > [ 1421.219855] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118380000 for ICM. > [ 1421.221132] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1183c0000 for ICM. > [ 1421.222407] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118400000 for ICM. > [ 1421.223685] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118440000 for ICM. > [ 1421.224963] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118480000 for ICM. > [ 1421.226245] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1184c0000 for ICM. > [ 1421.227522] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118500000 for ICM. > [ 1421.228801] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118540000 for ICM. > [ 1421.230079] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118580000 for ICM. > [ 1421.231355] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1185c0000 for ICM. > [ 1421.232633] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118600000 for ICM. > [ 1421.233909] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118640000 for ICM. > [ 1421.235186] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118680000 for ICM. > [ 1421.236470] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1186c0000 for ICM. > [ 1421.237748] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118700000 for ICM. > [ 1421.239026] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118740000 for ICM. > [ 1421.240303] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 118780000 for ICM. > [ 1421.241581] mlx4_core 0000:05:00.0: Mapped 1 chunks/256 KB at > 1187c0000 for ICM. > [ 1421.744993] mlx4_core 0000:05:00.0: Started init_resource_tracker: 80 > slaves > [ 1421.745454] mlx4_core 0000:05:00.0: irq 176 for MSI/MSI-X > [ 1421.745466] mlx4_core 0000:05:00.0: irq 177 for MSI/MSI-X > [ 1421.750286] mlx4_core 0000:05:00.0: NOP command IRQ test passed > [ 1421.750911] mlx4_core: Initializing 0000:82:00.0 > [ 1421.751006] mlx4_core 0000:82:00.0: Enabling SR-IOV with 1 VFs > [ 1421.851346] pci 0000:82:00.1: [15b3:1004] type 00 class 0x028000 > [ 1421.852051] mlx4_core 0000:82:00.0: Running in master mode > [ 1422.853834] mlx4_core 0000:82:00.0: FW version 2.11.500 (cmd intf > rev 3), max commands 16 > [ 1422.853837] mlx4_core 0000:82:00.0: Catastrophic error buffer at > 0x1f020, size 0x10, BAR 0 > [ 1422.853839] mlx4_core 0000:82:00.0: Communication vector bar:2 offset:0x800 > [ 1422.853840] mlx4_core 0000:82:00.0: FW size 385 KB > [ 1422.853842] mlx4_core 0000:82:00.0: Clear int @ f0058, BAR 0 > [ 1422.855140] mlx4_core 0000:82:00.0: Mapped 26 chunks/6168 KB for FW. > [ 1425.518005] mlx4_core 0000:82:00.0: BlueFlame available (reg size > 512, regs/page 8) > [ 1425.518117] mlx4_core 0000:82:00.0: Base MM extensions: flags > 00104cc0, rsvd L_Key 00008000 > [ 1425.518119] mlx4_core 0000:82:00.0: Max ICM size 4294967296 MB > [ 1425.518120] mlx4_core 0000:82:00.0: Max QPs: 16777216, reserved > QPs: 64, entry size: 256 > [ 1425.518122] mlx4_core 0000:82:00.0: Max SRQs: 16777216, reserved > SRQs: 64, entry size: 128 > [ 1425.518124] mlx4_core 0000:82:00.0: Max CQs: 16777216, reserved > CQs: 128, entry size: 128 > [ 1425.518125] mlx4_core 0000:82:00.0: Max EQs: 32, reserved EQs: 28, > entry size: 128 > [ 1425.518127] mlx4_core 0000:82:00.0: reserved MPTs: 256, reserved MTTs: 32 > [ 1425.518128] mlx4_core 0000:82:00.0: Max PDs: 131072, reserved PDs: > 4, reserved UARs: 7 > [ 1425.518130] mlx4_core 0000:82:00.0: Max QP/MCG: 131072, reserved MGMs: 0 > [ 1425.518131] mlx4_core 0000:82:00.0: Max CQEs: 4194304, max WQEs: > 16384, max SRQ WQEs: 16384 > [ 1425.518133] mlx4_core 0000:82:00.0: Local CA ACK delay: 15, max > MTU: 4096, port width cap: 2 > [ 1425.518134] mlx4_core 0000:82:00.0: Max SQ desc size: 1008, max SQ S/G: 62 > [ 1425.518136] mlx4_core 0000:82:00.0: Max RQ desc size: 512, max RQ S/G: 32 > [ 1425.518137] mlx4_core 0000:82:00.0: Max GSO size: 131072 > [ 1425.518138] mlx4_core 0000:82:00.0: Max counters: 256 > [ 1425.518140] mlx4_core 0000:82:00.0: Max RSS Table size: 256 > [ 1425.518141] mlx4_core 0000:82:00.0: DEV_CAP flags: > [ 1425.518142] mlx4_core 0000:82:00.0: RC transport > [ 1425.518144] mlx4_core 0000:82:00.0: UC transport > [ 1425.518145] mlx4_core 0000:82:00.0: UD transport > [ 1425.518146] mlx4_core 0000:82:00.0: XRC transport > [ 1425.518147] mlx4_core 0000:82:00.0: FCoIB support > [ 1425.518148] mlx4_core 0000:82:00.0: SRQ support > [ 1425.518150] mlx4_core 0000:82:00.0: IPoIB checksum offload > [ 1425.518151] mlx4_core 0000:82:00.0: P_Key violation counter > [ 1425.518152] mlx4_core 0000:82:00.0: Q_Key violation counter > [ 1425.518154] mlx4_core 0000:82:00.0: DPDP > [ 1425.518155] mlx4_core 0000:82:00.0: Big LSO headers > [ 1425.518156] mlx4_core 0000:82:00.0: APM support > [ 1425.518158] mlx4_core 0000:82:00.0: Atomic ops support > [ 1425.518159] mlx4_core 0000:82:00.0: Address vector port checking > support > [ 1425.518160] mlx4_core 0000:82:00.0: UD multicast support > [ 1425.518161] mlx4_core 0000:82:00.0: Router support > [ 1425.518163] mlx4_core 0000:82:00.0: IBoE support > [ 1425.518164] mlx4_core 0000:82:00.0: Unicast loopback support > [ 1425.518165] mlx4_core 0000:82:00.0: FCS header control > [ 1425.518167] mlx4_core 0000:82:00.0: UDP RSS support > [ 1425.518168] mlx4_core 0000:82:00.0: Unicast VEP steering support > [ 1425.518169] mlx4_core 0000:82:00.0: Multicast VEP steering support > [ 1425.518170] mlx4_core 0000:82:00.0: Counters support > [ 1425.518172] mlx4_core 0000:82:00.0: Port management change event > support > [ 1425.518173] mlx4_core 0000:82:00.0: 64 byte EQE support > [ 1425.518175] mlx4_core 0000:82:00.0: 64 byte CQE support > [ 1425.518176] mlx4_core 0000:82:00.0: RSS support > [ 1425.518177] mlx4_core 0000:82:00.0: RSS Toeplitz Hash Function support > [ 1425.518178] mlx4_core 0000:82:00.0: RSS XOR Hash Function support > [ 1425.518180] mlx4_core 0000:82:00.0: Device manage flow steering support > [ 1425.518182] mlx4_core 0000:82:00.0: 64B EQEs/CQEs supported by the > device but not enabled > [ 1425.518184] mlx4_core 0000:82:00.0: Steering mode is: B0 steering, > oper_log_mgm_entry_size = 10, modparam log_num_mgm_entry_size = 10 > [ 1425.518190] mlx4_core 0000:82:00.0: profile[ 0] ( CMPT): 2^26 > entries @ 0x 0, size 0x 100000000 > [ 1425.518193] mlx4_core 0000:82:00.0: profile[ 1] (RDMARC): 2^22 > entries @ 0x 100000000, size 0x 8000000 > [ 1425.518195] mlx4_core 0000:82:00.0: profile[ 2] ( MTT): 2^24 > entries @ 0x 108000000, size 0x 8000000 > [ 1425.518197] mlx4_core 0000:82:00.0: profile[ 3] ( QP): 2^18 > entries @ 0x 110000000, size 0x 4000000 > [ 1425.518199] mlx4_core 0000:82:00.0: profile[ 4] ( DMPT): 2^19 > entries @ 0x 114000000, size 0x 2000000 > [ 1425.518200] mlx4_core 0000:82:00.0: profile[ 5] ( ALTC): 2^18 > entries @ 0x 116000000, size 0x 1000000 > [ 1425.518202] mlx4_core 0000:82:00.0: profile[ 6] ( SRQ): 2^16 > entries @ 0x 117000000, size 0x 800000 > [ 1425.518204] mlx4_core 0000:82:00.0: profile[ 7] ( CQ): 2^16 > entries @ 0x 117800000, size 0x 800000 > [ 1425.518206] mlx4_core 0000:82:00.0: profile[ 8] ( MCG): 2^13 > entries @ 0x 118000000, size 0x 800000 > [ 1425.518208] mlx4_core 0000:82:00.0: profile[ 9] ( AUXC): 2^18 > entries @ 0x 118800000, size 0x 40000 > [ 1425.518210] mlx4_core 0000:82:00.0: profile[10] ( EQ): 2^10 > entries @ 0x 118840000, size 0x 20000 > [ 1425.518212] mlx4_core 0000:82:00.0: HCA context memory: reserving 4596096 > KB > [ 1425.518228] mlx4_core 0000:82:00.0: 4596096 KB of HCA context > requires 9068 KB aux memory. > [ 1425.554670] mlx4_core 0000:82:00.0: Mapped 39 chunks/9068 KB for ICM aux. > [ 1425.556059] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at 0 for ICM. > [ 1425.557429] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 40000000 for ICM. > [ 1425.558795] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 80000000 for ICM. > [ 1425.559144] mlx4_core 0000:82:00.0: Mapped 1 chunks/64 KB at > c0000000 for ICM. > [ 1425.559827] mlx4_core 0000:82:00.0: Mapped 1 chunks/128 KB at > 118840000 for ICM. > [ 1425.561182] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 108000000 for ICM. > [ 1425.562580] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 114000000 for ICM. > [ 1425.563932] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 110000000 for ICM. > [ 1425.565306] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118800000 for ICM. > [ 1425.566666] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 116000000 for ICM. > [ 1425.568018] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 100000000 for ICM. > [ 1425.569390] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 117800000 for ICM. > [ 1425.570754] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 117000000 for ICM. > [ 1425.572109] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118000000 for ICM. > [ 1425.573454] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118040000 for ICM. > [ 1425.574822] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118080000 for ICM. > [ 1425.576175] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1180c0000 for ICM. > [ 1425.577533] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118100000 for ICM. > [ 1425.578899] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118140000 for ICM. > [ 1425.580246] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118180000 for ICM. > [ 1425.581593] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1181c0000 for ICM. > [ 1425.582929] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118200000 for ICM. > [ 1425.584289] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118240000 for ICM. > [ 1425.585645] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118280000 for ICM. > [ 1425.587011] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1182c0000 for ICM. > [ 1425.588376] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118300000 for ICM. > [ 1425.589740] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118340000 for ICM. > [ 1425.591098] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118380000 for ICM. > [ 1425.592460] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1183c0000 for ICM. > [ 1425.593820] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118400000 for ICM. > [ 1425.595170] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118440000 for ICM. > [ 1425.596532] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118480000 for ICM. > [ 1425.597912] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1184c0000 for ICM. > [ 1425.599257] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118500000 for ICM. > [ 1425.600610] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118540000 for ICM. > [ 1425.601967] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118580000 for ICM. > [ 1425.603314] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1185c0000 for ICM. > [ 1425.604674] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118600000 for ICM. > [ 1425.606018] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118640000 for ICM. > [ 1425.607374] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118680000 for ICM. > [ 1425.608741] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1186c0000 for ICM. > [ 1425.610095] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118700000 for ICM. > [ 1425.611456] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118740000 for ICM. > [ 1425.612815] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 118780000 for ICM. > [ 1425.614165] mlx4_core 0000:82:00.0: Mapped 1 chunks/256 KB at > 1187c0000 for ICM. > [ 1426.193222] mlx4_core 0000:82:00.0: Started init_resource_tracker: 80 > slaves > [ 1426.193692] mlx4_core 0000:82:00.0: irq 178 for MSI/MSI-X > [ 1426.193705] mlx4_core 0000:82:00.0: irq 179 for MSI/MSI-X > [ 1426.200029] mlx4_core 0000:82:00.0: NOP command IRQ test passed > [ 1426.200605] mlx4_core: Initializing 0000:05:00.1 > [ 1426.200618] mlx4_core 0000:05:00.1: enabling device (0000 -> 0002) > [ 1426.200654] mlx4_core 0000:05:00.1: Skipping virtual function:1 > [ 1426.200674] mlx4_core: Initializing 0000:82:00.1 > [ 1426.200685] mlx4_core 0000:82:00.1: enabling device (0000 -> 0002) > [ 1426.200711] mlx4_core 0000:82:00.1: Skipping virtual function:1 > [ 1438.961741] pci-stub 0000:82:00.1: claimed by stub > [ 1438.967250] pci-stub 0000:82:00.1: enabling device (0000 -> 0002) > [ 1438.969739] mlx4_core 0000:82:00.0: FLR event for slave: 1 > [ 1438.969775] mlx4_core 0000:82:00.0: mlx4_handle_slave_flr > [ 1438.969776] mlx4_core 0000:82:00.0: mlx4_handle_slave_flr: clean slave: 1 > [ 1439.070931] assign device 0:82:0.1 > [ 1439.073469] mlx4_core 0000:82:00.0: FLR event for slave: 1 > [ 1439.073526] mlx4_core 0000:82:00.0: mlx4_handle_slave_flr > [ 1439.073529] mlx4_core 0000:82:00.0: mlx4_handle_slave_flr: clean slave: 1 > [ 1439.226723] mlx4_core 0000:82:00.0: Received reset from slave:1 > [ 1439.239542] pci-stub 0000:82:00.1: irq 180 for MSI/MSI-X > [ 1439.272548] pci-stub 0000:82:00.1: irq 180 for MSI/MSI-X > [ 1439.272559] pci-stub 0000:82:00.1: irq 181 for MSI/MSI-X > [ 1449.381120] mlx4_core 0000:82:00.0: Received reset from slave:1 > > > mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011) > mlx4_core: Initializing 0000:00:0b.0 > mlx4_core 0000:00:0b.0: enabling device (0000 -> 0002) > mlx4_core 0000:00:0b.0: Detected virtual function - running in slave mode > mlx4_core 0000:00:0b.0: Sending reset > mlx4_core 0000:00:0b.0: Sending vhcr0 > mlx4_core 0000:00:0b.0: 64B EQEs/CQEs supported by the device but not enabled > mlx4_core 0000:00:0b.0: HCA minimum page size:512 > mlx4_core 0000:00:0b.0: irq 48 for MSI/MSI-X > mlx4_core 0000:00:0b.0: irq 49 for MSI/MSI-X > mlx4_core 0000:00:0b.0: failed execution of VHCR_POST commandopcode 0x31 > mlx4_core 0000:00:0b.0: NOP command failed to generate MSI-X interrupt IRQ > 49). > mlx4_core 0000:00:0b.0: Trying again without MSI-X. > mlx4_core: probe of 0000:00:0b.0 failed with error -16 > > Mathis > > 2013/1/29 Jack Morgenstein <[email protected]>: > > On Tuesday 29 January 2013 15:15, Mathis GAVILLON wrote: > >> In fact, one boxe wasn't check in IOMMU submenu (Support for Interrupt > >> Remapping (EXPERIMENTAL)). I've recompiled the kernel of the host and > >> guest (is it necessary for this last one ?). But the error message is > >> already present. Just below, new dmeg log from host and guest O/S > >> (mlx4_core loaded with options num_vfs=1 probe_vf=1 debug_level=1 on > >> host and debug_level=1 on guest) : > >> > > Please do the following: > > 1. Check the Support for Interrupt Remapping (EXPERIMENTAL) box and > > recompile/reinstall. > > 2. set probe_vf=0, so that no vfs will run on the Hypervisor. That way you > > will > > see if any VF comes up without the "noise" in /var/log/messages in the > > Hypervisor generated > > by having one of the VFs run on the Hypervisor. > > > > 3. I noticed that you are enabling only a single VF, and that you are > > having the > > VF come up first on the Hypervisor, then moving it to run on a guest. > > It is better not to do things that way. Best to have it come up cleanly > > once on > > the guest. (the FLR event in your log results from this move. The VF > > gets reset). > > > > If you must have a VF running on the Hypervisor, use "num_vfs=2 > > probe_vf=1". > > You will then see Devices ...xx.0, ...xx.1 and ...xx.2 in lspci. The > > first VF > > (...xx.1) will be initialized on the Hypervisor. Attach ...xx.2 to the > > VM. > > > > -Jack > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
