Thanks for driver info. Because you are running in-kernel driver, we can enable the debug message level via ethtool. That will print HW ring info when issue occurs.
Here is the ethtool command to enable debug messages. # ethtool -s ethx msglvl 0x3c00 This will enable tx_done, rx_status, pktdata and hw message levels. You can confirm it by typing ethtool ethx , this will show you 'Current message level' Next time when issue occurs, please send me the full dmesg log after the issue occurred along with the bus trace. Thanks. -Tushar -----Original Message----- From: Pratyush Anand [mailto:pratyush.an...@st.com] Sent: Monday, January 09, 2012 8:21 PM To: Dave, Tushar N Cc: Greg KH; Pratyush Anand; e1000-devel@lists.sourceforge.net; net...@vger.kernel.org; Shiraz HASHIM; Deepak SIKRI; Bhavna YADAV; linux-...@vger.kernel.org; Linux NICS Subject: Re: Detected Hardware Unit Hang on Intel Wired Ethernet On 1/7/2012 12:25 AM, Dave, Tushar N wrote: > Pratyush, > > Sorry I got your name reversed. > Are you using in-kernel driver or one from Sourceforge. I am using in-kernel driver from kernel 2.6.37. > Please send me output of ethtool -i ethx. root@192.168.1.10:~# ethtool -i eth0 driver: e1000e version: 1.2.7-k2 firmware-version: 5.11-8 bus-info: 0000:01:00.0 Regards Pratyush > > -Tushar > > -----Original Message----- > From: Pratyush Anand [mailto:pratyush.an...@st.com] > Sent: Thursday, January 05, 2012 8:25 PM > To: Dave, Tushar N > Cc: Greg KH; Pratyush Anand; e1000-devel@lists.sourceforge.net; > net...@vger.kernel.org; Shiraz HASHIM; Deepak SIKRI; Bhavna YADAV; > linux-...@vger.kernel.org; Linux NICS > Subject: Re: Detected Hardware Unit Hang on Intel Wired Ethernet > > Thanks Tushar, > > On 1/6/2012 5:24 AM, Dave, Tushar N wrote: >> Anand, >> >> Sorry to hear that you have this issue with card. And yeah, thanks for doing >> the debugging and providing the bus trace. >> I think we should run the debug driver that prints the HW ring details when >> hang occurs. I can provide you a debug driver. You can then install debug >> driver and also let the bus tracer running. Once the issue occurs, provide >> me the full dmesg output (that has HW ring details) and bus trace. >> >> Tell me which card you have, 1gig or 10gig? Which driver are you running >> e1000e or igb or ixgbe? >> Can you also provide ethtool -i ethx output. >> >> Once I know which driver, I send you debug driver. > > I am using Intel PRO/1000 PT Server Adapter. > http://www.intel.com/content/www/us/en/network-adapters/gigabit-network-adapters/pro-1000-pt.html > > I am using e1000e driver. > > I see the problem when I try to mount rootfilesystem using NFS and use > MSI interrupt. I see this issue even before I can have cell prompt. > Please see first mail in this thread. > > http://www.mail-archive.com/e1000-devel@lists.sourceforge.net/msg04894.html > > Here, you can also see tx ring details when issue occur. > Please let me know, if you need any more info. > > Regards > Pratyush > >> >> Thanks. >> >> -Tushar >> >> -----Original Message----- >> From: netdev-ow...@vger.kernel.org [mailto:netdev-ow...@vger.kernel.org] On >> Behalf Of Pratyush Anand >> Sent: Wednesday, January 04, 2012 8:31 PM >> To: Greg KH >> Cc: Pratyush Anand; e1000-devel@lists.sourceforge.net; >> net...@vger.kernel.org; Shiraz HASHIM; Deepak SIKRI; Bhavna YADAV; >> linux-...@vger.kernel.org; Linux NICS >> Subject: Re: Detected Hardware Unit Hang on Intel Wired Ethernet >> >> On 1/5/2012 12:52 AM, Greg KH wrote: >>> On Wed, Jan 04, 2012 at 04:31:36PM +0530, Pratyush Anand wrote: >>>> Adding PCI mailing list too, as problem is coming only when MSI is enabled. >>>> >>>> If I connect an PCIe analyzer, I see that at the time of issue >>>> MRd(64) for 32 words has been issued with a wrong 64 bit address >>>> from ethernet card to my RC. >>>> In the normal course it always issues MRd(32) only. >>> >>> Bug in your pcie firmware controller? >>> >>> . >>> >> >> when you say "Bug in your pcie firmware controller?", is it RC's >> software or EP's software? >> >> Here I am pasting a part of analyzer log converted into text. >> Packet(177940), is an upstream request for MSI. Whenever any device >> writes at address 0x58A8F8, my PCIe RC considers it as MSI and generates >> an interrupt. So I receive MSI interrupt correctly in my software. Also >> MSI controller is correctly able to point me that the interrupt is from >> ethernet card. >> >> Now in Packet(178010), ethernet controller sends another upstream >> request for MRd(64) of 32 dwords with Address(AFECEB87:A9D88B00).Since, >> this address does not exist in my RC's world so, an UR is returned and >> hence the problem occurs. >> >> Now, question is, why ethernet card is generating inbound request with >> such a wrong address. I have taken log of all the tx_desc->buffer_addr >> programmed by software in function e1000_tx_queue. None of them is 64 >> bit or any invalid address. >> >> _______|_______________________________________________________________________ >> Packet(177916) Upstream 2.5(x1) TLP(1475) Mem MWr(32)(10:00000) Length(4) >> _______| RequesterID(003:00:0) Tag(2) Address(0EB00200) 1st BE(1111) >> _______| Last BE(1111) Data(4 dwords) LCRC(0x44E0407C) >> _______| Time Stamp(0013 . 460 549 544 s) >> _______|_______________________________________________________________________ >> Packet(177918) Downstream 2.5(x1) DLLP ACK AckNak_Seq_Num(1475) >> _______| CRC 16(0x0EB7) Time Stamp(0013 . 460 551 144 s) >> _______|_______________________________________________________________________ >> Packet(177940) Upstream 2.5(x1) TLP(1476) Mem MWr(32)(10:00000) Length(1) >> _______| RequesterID(003:00:0) Tag(30) Address(0058A8F8) 1st BE(0011) >> _______| Last BE(0000) Data(1 dword) LCRC(0xC21F32B6) >> _______| Time Stamp(0013 . 460 588 544 s) >> _______|_______________________________________________________________________ >> Packet(177942) Downstream 2.5(x1) DLLP ACK AckNak_Seq_Num(1476) >> _______| CRC 16(0x69F5) Time Stamp(0013 . 460 590 088 s) >> _______|_______________________________________________________________________ >> Packet(177946) Downstream 2.5(x1) TLP(309) Mem MRd(32)(00:00000) Length(1) >> _______| RequesterID(002:00:0) Tag(19) Address(C01000C0) 1st BE(1111) >> _______| Last BE(0000) LCRC(0x91BDA1F5) Time Stamp(0013 . 460 595 936 s) >> _______|_______________________________________________________________________ >> Packet(177947) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(309) >> _______| CRC 16(0x25C6) Time Stamp(0013 . 460 596 368 s) >> _______|_______________________________________________________________________ >> Packet(177950) Upstream 2.5(x1) TLP(1477) Cpl CplD(10:01010) Length(1) >> _______| RequesterID(002:00:0) Tag(19) CompleterID(003:00:0) Status(SC) >> BCM(0) >> _______| Byte Cnt(4) Lwr Addr(0x40) Data(1 dword) LCRC(0x8FE0D922) >> _______| Time Stamp(0013 . 460 597 304 s) >> _______|_______________________________________________________________________ >> Packet(177952) Downstream 2.5(x1) DLLP ACK AckNak_Seq_Num(1477) >> _______| CRC 16(0xC8EE) Time Stamp(0013 . 460 598 840 s) >> _______|_______________________________________________________________________ >> Packet(177999) Downstream 2.5(x1) TLP(310) Mem MWr(32)(10:00000) Length(1) >> _______| RequesterID(002:00:0) Tag(0) Address(C0103818) 1st BE(1111) >> _______| Last BE(0000) Data(1 dword) LCRC(0xA898D9A1) >> _______| Time Stamp(0013 . 460 687 936 s) >> _______|_______________________________________________________________________ >> Packet(178001) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(310) >> _______| CRC 16(0xC6EA) Time Stamp(0013 . 460 688 384 s) >> _______|_______________________________________________________________________ >> Packet(178004) Upstream 2.5(x1) TLP(1478) Mem MRd(32)(00:00000) Length(4) >> _______| RequesterID(003:00:0) Tag(4) Address(0EAFB990) 1st BE(1111) >> _______| Last BE(1111) LCRC(0xB54722D2) Time Stamp(0013 . 460 689 312 s) >> _______|_______________________________________________________________________ >> Packet(178006) Downstream 2.5(x1) TLP(311) Cpl CplD(10:01010) Length(4) >> _______| RequesterID(003:00:0) Tag(4) CompleterID(002:00:0) Status(SC) >> BCM(0) >> _______| Byte Cnt(16) Lwr Addr(0x10) Data(4 dwords) LCRC(0xFE303776) >> _______| Time Stamp(0013 . 460 690 288 s) >> _______|_______________________________________________________________________ >> Packet(178007) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(311) >> _______| CRC 16(0x67F1) Time Stamp(0013 . 460 690 776 s) >> _______|_______________________________________________________________________ >> Packet(178008) Downstream 2.5(x1) DLLP ACK AckNak_Seq_Num(1478) >> _______| CRC 16(0x2BC2) Time Stamp(0013 . 460 690 824 s) >> _______|_______________________________________________________________________ >> Packet(178010) Upstream 2.5(x1) TLP(1479) Mem MRd(64)(01:00000) Length(32) >> _______| RequesterID(003:00:0) Tag(11) Address(AFECEB87:A9D88B00) 1st >> BE(1100) >> _______| Last BE(0011) LCRC(0x6BE341C9) Time Stamp(0013 . 460 691 680 s) >> _______|_______________________________________________________________________ >> Packet(178011) Upstream 2.5(x1) TLP(1480) Mem MRd(64)(01:00000) Length(32) >> _______| RequesterID(003:00:0) Tag(8) Address(AFECEB87:A9D88B7C) 1st >> BE(1100) >> _______| Last BE(0011) LCRC(0xAA5647BD) Time Stamp(0013 . 460 691 808 s) >> _______|_______________________________________________________________________ >> Packet(178012) Upstream 2.5(x1) TLP(1481) Mem MRd(64)(01:00000) Length(32) >> _______| RequesterID(003:00:0) Tag(9) Address(AFECEB87:A9D88BF8) 1st >> BE(1100) >> _______| Last BE(0011) LCRC(0xEEB1F63F) Time Stamp(0013 . 460 692 120 s) >> _______|_______________________________________________________________________ >> Packet(178013) Upstream 2.5(x1) TLP(1482) Mem MRd(64)(01:00000) Length(32) >> _______| RequesterID(003:00:0) Tag(10) Address(AFECEB87:A9D88C74) 1st >> BE(1100) >> _______| Last BE(0011) LCRC(0xA508142C) Time Stamp(0013 . 460 692 248 s) >> _______|_______________________________________________________________________ >> Packet(178014) Downstream 2.5(x1) TLP(312) Cpl Cpl(00:01010) Length(0) >> _______| RequesterID(003:00:0) Tag(11) CompleterID(002:00:0) Status(UR)-BAD >> _______| BCM(0) Byte Cnt(124) Lwr Addr(0x02) LCRC(0xCE5540D2) >> _______| Time Stamp(0013 . 460 692 328 s) >> _______|_______________________________________________________________________ >> Packet(178015) Downstream 2.5(x1) TLP(313) Cpl Cpl(00:01010) Length(0) >> _______| RequesterID(003:00:0) Tag(8) CompleterID(002:00:0) Status(UR)-BAD >> _______| BCM(0) Byte Cnt(124) Lwr Addr(0x7E) LCRC(0x9FE2487D) >> _______| Time Stamp(0013 . 460 692 456 s) >> _______|_______________________________________________________________________ >> Packet(178016) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(312) >> _______| CRC 16(0x086E) Time Stamp(0013 . 460 692 760 s) >> _______|_______________________________________________________________________ >> Packet(178017) Downstream 2.5(x1) TLP(314) Cpl Cpl(00:01010) Length(0) >> _______| RequesterID(003:00:0) Tag(9) CompleterID(002:00:0) Status(UR)-BAD >> _______| BCM(0) Byte Cnt(124) Lwr Addr(0x7A) LCRC(0x097BF4DE) >> _______| Time Stamp(0013 . 460 692 776 s) >> _______|_______________________________________________________________________ >> Packet(178018) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(313) >> _______| CRC 16(0xA975) Time Stamp(0013 . 460 692 888 s) >> _______|_______________________________________________________________________ >> Packet(178019) Downstream 2.5(x1) TLP(315) Cpl Cpl(00:01010) Length(0) >> _______| RequesterID(003:00:0) Tag(10) CompleterID(002:00:0) Status(UR)-BAD >> _______| BCM(0) Byte Cnt(124) Lwr Addr(0x76) LCRC(0x64BDF921) >> _______| Time Stamp(0013 . 460 692 904 s) >> _______|_______________________________________________________________________ >> Packet(178020) Upstream 2.5(x1) TLP(1483) Msg Msg(01:10000) >> _______| Msg Routing(To RC) Length(0) RequesterID(003:00:0) Tag(31) >> _______| Message Code(ERR_FATAL) LCRC(0xCDA53E96) >> _______| Time Stamp(0013 . 460 693 184 s) >> _______|_______________________________________________________________________ >> Packet(178021) Downstream 2.5(x1) DLLP ACK AckNak_Seq_Num(1482) >> _______| CRC 16(0xA771) Time Stamp(0013 . 460 693 208 s) >> _______|_______________________________________________________________________ >> Packet(178023) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(314) >> _______| CRC 16(0x4A59) Time Stamp(0013 . 460 693 280 s) >> _______|_______________________________________________________________________ >> Packet(178024) Upstream 2.5(x1) TLP(1484) Msg Msg(01:10000) >> _______| Msg Routing(To RC) Length(0) RequesterID(003:00:0) Tag(31) >> _______| Message Code(ERR_FATAL) LCRC(0x86D9ACB6) >> _______| Time Stamp(0013 . 460 693 312 s) >> _______|_______________________________________________________________________ >> Packet(178025) Upstream 2.5(x1) DLLP ACK AckNak_Seq_Num(315) >> _______| CRC 16(0xEB42) Time Stamp(0013 . 460 693 408 s) >> _______|_______________________________________________________________________ >> Packet(178026) Upstream 2.5(x1) TLP(1485) Msg Msg(01:10000) >> _______| Msg Routing(To RC) Length(0) RequesterID(003:00:0) Tag(31) >> _______| Message Code(ERR_FATAL) LCRC(0xC5120A31) >> _______| Time Stamp(0013 . 460 693 632 s) >> _______|_______________________________________________________________________ >> Packet(178028) Upstream 2.5(x1) TLP(1486) Msg Msg(01:10000) >> _______| Msg Routing(To RC) Length(0) RequesterID(003:00:0) Tag(31) >> _______| Message Code(ERR_FATAL) LCRC(0x41499062) >> _______| Time Stamp(0013 . 460 693 792 s) >> _______|_______________________________________________________________________ >> Packet(178029) Downstream 2.5(x1) DLLP ACK AckNak_Seq_Num(1486) >> _______| CRC 16(0x231F) Time Stamp(0013 . 460 694 704 s) >> _______|_______________________________________________________________________ >> >> -- >> To unsubscribe from this list: send the line "unsubscribe netdev" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> . >> > > . > ------------------------------------------------------------------------------ Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
