I would check with the netconsole people on this one. I'm not sure netconsole is well supported and some of the things it does to queue up data before the interface is ready can cause issues with our driver. We tried fixing our driver but the kernel community, IIRC, said netconsole should be fixed instead. At the time there wasn't anyone who wanted to fix netconsole.
Todd Fujinaka Software Application Engineer Networking Division (ND) Intel Corporation todd.fujin...@intel.com (503) 712-4565 -----Original Message----- From: Kevin Newman [mailto:knew...@peak6.com] Sent: Friday, December 26, 2014 2:00 PM To: 'e1000-devel@lists.sourceforge.net' Subject: [E1000-devel] ixgbe Driver Crashing with netconsole Enabled on Boot Hi, I'm seeing the following Call Trace when using netconsole with an Intel X540-AT2 and ixgbe. This issue seems to freeze/crash any interface using the ixgbe driver once the link becomes ready on that interface and netconsole enables during boot: [ 14.683494] init: udev-fallback-graphics main process (1477) terminated with status 1 [ 14.734565] IPv6: ADDRCONF(NETDEV_UP): em1: link is not ready [ 14.961137] init: samba-ad-dc main process (1712) terminated with status 1 [ 15.499345] init: plymouth-upstart-bridge main process ended, respawning [ 18.738471] netpoll: netconsole: timeout waiting for carrier [ 18.744837] netpoll: netconsole: local port 31337 [ 18.750098] netpoll: netconsole: local IPv4 address 192.168.1.110 [ 18.756612] netpoll: netconsole: interface 'em1' [ 18.761765] netpoll: netconsole: remote port 20514 [ 18.767115] netpoll: netconsole: remote IPv4 address 192.168.1.49 [ 18.773821] netpoll: netconsole: remote ethernet address 00:00:0c:07:ac:04 [ 18.781627] console [netcon0] enabled [ 18.785732] ------------[ cut here ]------------ [ 18.785743] WARNING: CPU: 70 PID: 1238 at /home/apw/COD/linux/kernel/softirq.c:156 local_bh_enable_ip+0x6a/0x90() [ 18.785779] Modules linked in: x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel dcdbas kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x8s [ 18.785784] CPU: 70 PID: 1238 Comm: modprobe Not tainted 3.13.11-03131111-generic #201411111336 [ 18.785785] Hardware name: Dell Inc. PowerEdge R630/0CNCJW, BIOS 1.1.4 11/03/2014 [ 18.785793] 000000000000009c ffff8820259b9948 ffffffff81748be5 0000000000000082 [ 18.785800] 0000000000000000 ffff8820259b9988 ffffffff81069acc ffff88107fffbf00 [ 18.785807] 0000000000000001 0000000000000000 ffff8810241b4000 ffff88102504c800 [ 18.785808] Call Trace: [ 18.785822] [<ffffffff81748be5>] dump_stack+0x46/0x58 [ 18.785832] [<ffffffff81069acc>] warn_slowpath_common+0x8c/0xc0 [ 18.785837] [<ffffffff81069b1a>] warn_slowpath_null+0x1a/0x20 [ 18.785840] [<ffffffff8106f17a>] local_bh_enable_ip+0x6a/0x90 [ 18.785847] [<ffffffff8175506b>] _raw_spin_unlock_bh+0x1b/0x40 [ 18.785862] [<ffffffffa00ee22c>] ixgbe_poll+0xcc/0x2e0 [ixgbe] [ 18.785869] [<ffffffff8165c268>] poll_one_napi.isra.24+0x68/0xe0 [ 18.785877] [<ffffffffa00ec139>] ? ixgbe_msix_clean_rings+0x29/0x40 [ixgbe] [ 18.785881] [<ffffffff8165cca0>] netpoll_poll_dev+0xc0/0x1a0 [ 18.785885] [<ffffffff8165d07c>] netpoll_send_skb_on_dev+0x2fc/0x400 [ 18.785890] [<ffffffff8165d4dd>] netpoll_send_udp+0x27d/0x380 [ 18.785898] [<ffffffffa006e823>] write_msg.part.2+0xb3/0x110 [netconsole] [ 18.785905] [<ffffffffa006e8b5>] write_msg+0x35/0x40 [netconsole] [ 18.785914] [<ffffffff810bfb25>] call_console_drivers.constprop.17+0xa5/0x110 [ 18.785918] [<ffffffff810c0f8c>] console_unlock.part.12+0x19c/0x260 [ 18.785921] [<ffffffff810c106a>] console_unlock+0x1a/0x30 [ 18.785925] [<ffffffff810c140e>] vprintk_emit+0x1be/0x520 [ 18.785934] [<ffffffffa00003e4>] ? configfs_create+0xe4/0x110 [configfs] [ 18.785939] [<ffffffff8173cde3>] printk+0x61/0x63 [ 18.785943] [<ffffffff810c230b>] register_console+0x2db/0x370 [ 18.785947] [<ffffffffa007a000>] ? 0xffffffffa0079fff [ 18.785953] [<ffffffffa007a0ef>] init_netconsole+0xef/0x1000 [netconsole] [ 18.785957] [<ffffffffa007a000>] ? 0xffffffffa0079fff [ 18.785964] [<ffffffff8100215a>] do_one_initcall+0xfa/0x1b0 [ 18.785973] [<ffffffff8105bcb3>] ? set_memory_nx+0x43/0x50 [ 18.785979] [<ffffffff8173db80>] do_init_module+0x80/0x1d1 [ 18.785987] [<ffffffff810e6d01>] load_module+0x4e1/0x600 [ 18.785991] [<ffffffff810e43c0>] ? show_initstate+0x50/0x50 [ 18.785995] [<ffffffff810e6fce>] SyS_finit_module+0xae/0xd0 [ 18.786002] [<ffffffff8175e2ad>] system_call_fastpath+0x1a/0x1f [ 18.786004] ---[ end trace 0caa592d9a235754 ]--- [ 21.085372] ixgbe 0000:01:00.0 em1: NIC Link is Up 10 Gbps, Flow Control: RX [ 21.085688] IPv6: ADDRCONF(NETDEV_CHANGE): em1: link becomes ready root@pslchi6ddevtest1:/sys/module/printk/parameters# lspci |grep X5 01:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) 01:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) I'm not 100% sure how this is related, but this issue only seems to occur when I enable "ignore_loglevel" in /sys/module/printk/parameters or via the kernel cmdline. I'm running the latest SF driver, I think I have LRO disabled, and I'm running the latest Ubuntu mainline kernel. Any idea what could be causing this crash? Please let me know if I can provide more information. Thanks! root@pslchi6ddevtest1:/sys/module/printk/parameters# modinfo ixgbe filename: /lib/modules/3.13.11-03131111-generic/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko version: 3.23.2 license: GPL description: Intel(R) 10 Gigabit PCI Express Network Driver author: Intel Corporation, <linux.n...@intel.com> srcversion: 2ADA5E537923E983FA9DAE2 alias: pci:v00008086d00001560sv*sd*bc*sc*i* alias: pci:v00008086d00001558sv*sd*bc*sc*i* alias: pci:v00008086d0000154Asv*sd*bc*sc*i* alias: pci:v00008086d00001557sv*sd*bc*sc*i* alias: pci:v00008086d0000154Fsv*sd*bc*sc*i* alias: pci:v00008086d0000154Dsv*sd*bc*sc*i* alias: pci:v00008086d00001528sv*sd*bc*sc*i* alias: pci:v00008086d000010F8sv*sd*bc*sc*i* alias: pci:v00008086d0000151Csv*sd*bc*sc*i* alias: pci:v00008086d00001529sv*sd*bc*sc*i* alias: pci:v00008086d0000152Asv*sd*bc*sc*i* alias: pci:v00008086d000010F9sv*sd*bc*sc*i* alias: pci:v00008086d00001514sv*sd*bc*sc*i* alias: pci:v00008086d00001507sv*sd*bc*sc*i* alias: pci:v00008086d000010FBsv*sd*bc*sc*i* alias: pci:v00008086d00001517sv*sd*bc*sc*i* alias: pci:v00008086d000010FCsv*sd*bc*sc*i* alias: pci:v00008086d000010F7sv*sd*bc*sc*i* alias: pci:v00008086d00001508sv*sd*bc*sc*i* alias: pci:v00008086d000010DBsv*sd*bc*sc*i* alias: pci:v00008086d000010F4sv*sd*bc*sc*i* alias: pci:v00008086d000010E1sv*sd*bc*sc*i* alias: pci:v00008086d000010F1sv*sd*bc*sc*i* alias: pci:v00008086d000010ECsv*sd*bc*sc*i* alias: pci:v00008086d000010DDsv*sd*bc*sc*i* alias: pci:v00008086d0000150Bsv*sd*bc*sc*i* alias: pci:v00008086d000010C8sv*sd*bc*sc*i* alias: pci:v00008086d000010C7sv*sd*bc*sc*i* alias: pci:v00008086d000010C6sv*sd*bc*sc*i* alias: pci:v00008086d000010B6sv*sd*bc*sc*i* depends: ptp,dca vermagic: 3.13.11-03131111-generic SMP mod_unload modversions parm: InterruptType:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default IntMode (deprecated) (array of int) parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int) parm: MQ:Disable or enable Multiple Queues, default 1 (array of int) parm: DCA:Disable or enable Direct Cache Access, 0=disabled, 1=descriptor only, 2=descriptor and data (array of int) parm: RSS:Number of Receive-Side Scaling Descriptor Queues, default 0=number of cpus (array of int) parm: VMDQ:Number of Virtual Machine Device Queues: 0/1 = disable, 2-16 enable (default=8) (array of int) parm: max_vfs:Number of Virtual Functions: 0 = disable (default), 1-63 = enable this many VFs (array of int) parm: VEPA:VEPA Bridge Mode: 0 = VEB (default), 1 = VEPA (array of int) parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (0,1,956-488281), default 1 (array of int) parm: LLIPort:Low Latency Interrupt TCP Port (0-65535) (array of int) parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1) (array of int) parm: LLISize:Low Latency Interrupt on Packet Size (0-1500) (array of int) parm: LLIEType:Low Latency Interrupt Ethernet Protocol Type (array of int) parm: LLIVLANP:Low Latency Interrupt on VLAN priority threshold (array of int) parm: FdirPballoc:Flow Director packet buffer allocation level: 1 = 8k hash filters or 2k perfect filters 2 = 16k hash filters or 4k perfect filters 3 = 32k hash filters or 8k perfect filters (array of int) parm: AtrSampleRate:Software ATR Tx packet sample rate (array of int) parm: FCoE:Disable or enable FCoE Offload, default 1 (array of int) parm: LRO:Large Receive Offload (0,1), default 1 = on (array of int) parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599 based adapters, default 0 = Disable (array of int) root@pslchi6ddevtest1:/sys/module/printk/parameters# ethtool -S em1 |grep lro root@pslchi6ddevtest1:/sys/module/printk/parameters# uname -a Linux pslchi6ddevtest1 3.13.11-03131111-generic #201411111336 SMP Tue Nov 11 18:37:52 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux root@pslchi6ddevtest1:/sys/module/printk/parameters# ______________________________________________ See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
