Hi,
Seeing failures when trying to do PCI passthrough of Intel XL710 40G
interface to KVM vm.
0a:00.1 Ethernet controller: Intel Corporation Ethernet Controller
XL710 for 40GbE QSFP+ (rev 01)
>From dmesg on host:
[80326.559674] kvm: zapping shadow pages for mmio generation wraparound
[80327.271191] kvm [175994]: vcpu0 unhandled rdmsr: 0x1c9
[80327.271689] kvm [175994]: vcpu0 unhandled rdmsr: 0x1a6
[80327.272201] kvm [175994]: vcpu0 unhandled rdmsr: 0x1a7
[80327.272681] kvm [175994]: vcpu0 unhandled rdmsr: 0x3f6
[80327.376186] kvm [175994]: vcpu0 unhandled rdmsr: 0x606
The pci device is still available in the VM but stat transfer fails.
With the i40e driver, the data transfer fails.
Relevant dmesg output:
[ 11.544088] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
[ 11.689178] i40e 0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
[ 16.704071] ------------[ cut here ]------------
[ 16.705053] WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:303
dev_watchdog+0x23e/0x250()
[ 16.705053] NETDEV WATCHDOG: eth1 (i40e): transmit queue 1 timed out
[ 16.705053] Modules linked in: cirrus ttm drm_kms_helper i40e drm
ppdev serio_raw i2c_piix4 virtio_net parport_pc ptp virtio_balloon
crct10dif_pclmul pps_core parport pvpanic crc32_pclmul
ghash_clmulni_intel virtio_blk crc32c_intel virtio_pci virtio_ring
virtio ata_generic pata_acpi
[ 16.705053] CPU: 1 PID: 0 Comm: swapper/1 Not tainted
3.18.7-200.fc21.x86_64 #1
[ 16.705053] Hardware name: Fedora Project OpenStack Nova, BIOS
1.7.5-20140709_153950- 04/01/2014
[ 16.705053] 0000000000000000 2e5932b294d0c473 ffff88043fc83d48
ffffffff8175e686
[ 16.705053] 0000000000000000 ffff88043fc83da0 ffff88043fc83d88
ffffffff810991d1
[ 16.705053] ffff88042958f5c0 0000000000000001 ffff88042865f000
0000000000000001
[ 16.705053] Call Trace:
[ 16.705053] <IRQ> [<ffffffff8175e686>] dump_stack+0x46/0x58
[ 16.705053] [<ffffffff810991d1>] warn_slowpath_common+0x81/0xa0
[ 16.705053] [<ffffffff81099245>] warn_slowpath_fmt+0x55/0x70
[ 16.705053] [<ffffffff8166e62e>] dev_watchdog+0x23e/0x250
[ 16.705053] [<ffffffff8166e3f0>] ? dev_graft_qdisc+0x80/0x80
[ 16.705053] [<ffffffff810fd52a>] call_timer_fn+0x3a/0x120
[ 16.705053] [<ffffffff8166e3f0>] ? dev_graft_qdisc+0x80/0x80
[ 16.705053] [<ffffffff810ff692>] run_timer_softirq+0x212/0x2f0
[ 16.705053] [<ffffffff8109d7a4>] __do_softirq+0x124/0x2d0
[ 16.705053] [<ffffffff8109db75>] irq_exit+0x125/0x130
[ 16.705053] [<ffffffff817681d8>] smp_apic_timer_interrupt+0x48/0x60
[ 16.705053] [<ffffffff817662bd>] apic_timer_interrupt+0x6d/0x80
[ 16.705053] <EOI> [<ffffffff811005c8>] ? hrtimer_start+0x18/0x20
[ 16.705053] [<ffffffff8105ca96>] ? native_safe_halt+0x6/0x10
[ 16.705053] [<ffffffff810f81d3>] ? rcu_eqs_enter+0xa3/0xb0
[ 16.705053] [<ffffffff8101ec7f>] default_idle+0x1f/0xc0
[ 16.705053] [<ffffffff8101f64f>] arch_cpu_idle+0xf/0x20
[ 16.705053] [<ffffffff810dad35>] cpu_startup_entry+0x3c5/0x410
[ 16.705053] [<ffffffff8104a2af>] start_secondary+0x1af/0x1f0
[ 16.705053] ---[ end trace 7bda53aeda558267 ]---
[ 16.705053] i40e 0000:00:05.0 eth1: tx_timeout recovery level 1
[ 16.705053] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 519 Tx
ring 0 disable timeout
[ 16.744198] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 520 Tx
ring 64 disable timeout
[ 16.779322] i40e 0000:00:05.0: i40e_ptp_init: added PHC on eth1
[ 16.791819] i40e 0000:00:05.0: PF 40 attempted to control timestamp
mode on port 1, which is owned by PF 1
[ 16.933869] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
[ 18.853624] SELinux: initialized (dev tmpfs, type tmpfs), uses
transition SIDs
[ 22.720083] i40e 0000:00:05.0 eth1: tx_timeout recovery level 2
[ 22.826993] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 519 Tx
ring 0 disable timeout
[ 22.935288] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 520 Tx
ring 64 disable timeout
[ 23.669555] i40e 0000:00:05.0: i40e_ptp_init: added PHC on eth1
[ 23.682067] i40e 0000:00:05.0: PF 40 attempted to control timestamp
mode on port 1, which is owned by PF 1
[ 23.722423] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
[ 23.800206] i40e 0000:00:06.0: i40e_ptp_init: added PHC on eth2
[ 23.813804] i40e 0000:00:06.0: PF 48 attempted to control timestamp
mode on port 0, which is owned by PF 0
[ 23.855275] i40e 0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
[ 38.720091] i40e 0000:00:05.0 eth1: tx_timeout recovery level 3
[ 38.725844] random: nonblocking pool is initialized
[ 38.729874] i40e 0000:00:06.0: HMC error interrupt
[ 38.733425] i40e 0000:00:06.0: i40e_vsi_control_tx: VSI seid 518 Tx
ring 0 disable timeout
[ 38.738886] i40e 0000:00:06.0: i40e_vsi_control_tx: VSI seid 521 Tx
ring 64 disable timeout
[ 39.689569] i40e 0000:00:06.0: i40e_ptp_init: added PHC on eth2
[ 39.704197] i40e 0000:00:06.0: PF 48 attempted to control timestamp
mode on port 0, which is owned by PF 0
[ 39.746879] i40e 0000:00:06.0 eth2: NIC Link is Down
[ 39.838356] i40e 0000:00:05.0: i40e_ptp_init: added PHC on eth1
[ 39.851788] i40e 0000:00:05.0: PF 40 attempted to control timestamp
mode on port 1, which is owned by PF 1
[ 39.892822] i40e 0000:00:05.0 eth1: NIC Link is Down
[ 43.011610] i40e 0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
[ 43.059976] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full
Duplex, Flow Control: None
Would appreciate any information on how to debug this issue further
and if the "unhandled rdmsr" logs from KVM indicate some issues with
the device passthrough.
Thanks
Jacob
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
