OK. I have the following in /boot/loader.conf (and rebooted)
hw.em.rxd=1024
hw.em.txd=1024
Should this be hw.em2.rxd? Is it set per interface or across all
interfaces?
nmbcluster=262144
# sysctl dev.em.2.stats=1
Feb 22 16:29:57 inet-gw kernel: em2: Defer count = 20
Feb 22 16:29:57 inet-gw kernel: em2: Missed Packets = 119947
Feb 22 16:29:57 inet-gw kernel: em2: Receive No Buffers = 276762
Feb 22 16:29:57 inet-gw kernel: em2: Receive Length Errors = 0
Feb 22 16:29:57 inet-gw kernel: em2: Receive errors = 0
Feb 22 16:29:57 inet-gw kernel: em2: Crc errors = 0
Feb 22 16:29:57 inet-gw kernel: em2: Alignment errors = 0
Feb 22 16:29:57 inet-gw kernel: em2: Collision/Carrier extension errors
= 0
Feb 22 16:29:57 inet-gw kernel: em2: RX overruns = 21
Feb 22 16:29:57 inet-gw kernel: em2: watchdog timeouts = 47
Feb 22 16:29:57 inet-gw kernel: em2: RX MSIX IRQ = 0 TX MSIX IRQ = 0
LINK MSIX IRQ = 0
Feb 22 16:29:57 inet-gw kernel: em2: XON Rcvd = 22
Feb 22 16:29:57 inet-gw kernel: em2: XON Xmtd = 8349
Feb 22 16:29:57 inet-gw kernel: em2: XOFF Rcvd = 31
Feb 22 16:29:57 inet-gw kernel: em2: XOFF Xmtd = 15779
Feb 22 16:29:57 inet-gw kernel: em2: Good Packets Rcvd = 966101852
Feb 22 16:29:57 inet-gw kernel: em2: Good Packets Xmtd = 755993237
Feb 22 16:29:57 inet-gw kernel: em2: TSO Contexts Xmtd = 0
Feb 22 16:29:57 inet-gw kernel: em2: TSO Contexts Failed = 0
still seeing the watchdog timer and link up/down messages.
Should I try going higher than 1024 on the hw.em.rxd? I'm not sure the
next time I can schedule another reboot on this production server.
---- Kirk
Kirk Davis
Senior Network Analyst, ITS
Edmonton Public Schools
One Kingsway Ave.
Edmonton, Alberta, Canada
T5H 4G9
phone: 1-780-429-8308
________________________________
From: Jack Vogel [mailto:jfvo...@gmail.com]
Sent: Monday, February 22, 2010 3:45 PM
To: Kirk Davis
Cc: Mike Tancsa; freebsd-net@freebsd.org
Subject: Re: Intel em0: watchdog timeout
OK, so you are still failing to get mbufs in the RX side,
increase the nmbcluster
value, and then what size is your RX ring (number of rx
descriptors)?
If you havent already done so, change that to 1024.
I am developing a change in the RX code right now that will help
this situation, but am doing so in the 10G driver, once its
solid there
I will be backporting it into the 1G drivers, it will make
discards
almost unnecessary.
Jack
On Mon, Feb 22, 2010 at 1:43 PM, Kirk Davis <kirk.da...@epsb.ca>
wrote:
> -----Original Message-----
> From: Mike Tancsa [mailto:m...@sentex.net]
> Subject: Re: Intel em0: watchdog timeout
>
> At 03:46 PM 2/22/2010, Kirk Davis wrote:
> >Does this need to be done in loader.conf? It doesn't
seem
> to take from
> >the command line.
> ># sysctl dev.em.2.stats=1
> >dev.em.2.stats: -1 -> -1
> >
> ># sysctl dev.em.2.stats
> >dev.em.2.stats: -1
>
> Hi,
> After you issue those commands, the driver
will spit out a
> lot of useful stats to syslog. It will report
something like the
> following in /var/log/messages
>
> Feb 22 16:06:31 offsite kernel: em0: Excessive
collisions = 0
> Feb 22 16:06:31 offsite kernel: em0: Sequence errors =
0
> Feb 22 16:06:31 offsite kernel: em0: Defer count = 0
> Feb 22 16:06:31 offsite kernel: em0: Missed Packets =
0
> Feb 22 16:06:31 offsite kernel: em0: Receive No
Buffers = 0
> Feb 22 16:06:31 offsite kernel: em0: Receive Length
Errors = 0
> Feb 22 16:06:31 offsite kernel: em0: Receive errors =
0
> Feb 22 16:06:31 offsite kernel: em0: Crc errors = 0
> Feb 22 16:06:31 offsite kernel: em0: Alignment errors
= 0
> Feb 22 16:06:31 offsite kernel: em0: Collision/Carrier
> extension errors = 0
> Feb 22 16:06:31 offsite kernel: em0: RX overruns = 0
> Feb 22 16:06:31 offsite kernel: em0: watchdog timeouts
= 0
> Feb 22 16:06:31 offsite kernel: em0: RX MSIX IRQ = 0
TX MSIX IRQ = 0
> LINK MSIX IRQ = 0
> Feb 22 16:06:31 offsite kernel: em0: XON Rcvd = 0
> Feb 22 16:06:31 offsite kernel: em0: XON Xmtd = 0
> Feb 22 16:06:31 offsite kernel: em0: XOFF Rcvd = 0
> Feb 22 16:06:31 offsite kernel: em0: XOFF Xmtd = 0
> Feb 22 16:06:31 offsite kernel: em0: Good Packets Rcvd
= 2559032551
> Feb 22 16:06:31 offsite kernel: em0: Good Packets Xmtd
= 1568751141
> Feb 22 16:06:31 offsite kernel: em0: TSO Contexts Xmtd
= 0
> Feb 22 16:06:31 offsite kernel: em0: TSO Contexts
Failed = 0
Thanks Mike and Jack. I don't know why I didn'ty notice
the output in
/var/log/messages
Here is the output for the two interfaces that are
causing this issue.
Feb 22 13:33:52 inet-gw kernel: em0: Excessive
collisions = 0
Feb 22 13:33:52 inet-gw kernel: em0: Sequence errors = 0
Feb 22 13:33:52 inet-gw kernel: em0: Defer count = 0
Feb 22 13:33:52 inet-gw kernel: em0: Missed Packets =
24296
Feb 22 13:33:52 inet-gw kernel: em0: Receive No Buffers
= 0
Feb 22 13:33:52 inet-gw kernel: em0: Receive Length
Errors = 0
Feb 22 13:33:52 inet-gw kernel: em0: Receive errors = 0
Feb 22 13:33:52 inet-gw kernel: em0: Crc errors = 0
Feb 22 13:33:52 inet-gw kernel: em0: Alignment errors =
0
Feb 22 13:33:52 inet-gw kernel: em0: Collision/Carrier
extension errors
= 0
Feb 22 13:33:52 inet-gw kernel: em0: RX overruns = 0
Feb 22 13:33:52 inet-gw kernel: em0: watchdog timeouts =
6
Feb 22 13:33:52 inet-gw kernel: em0: RX MSIX IRQ = 0 TX
MSIX IRQ = 0
LINK MSIX IRQ = 0
Feb 22 13:33:52 inet-gw kernel: em0: XON Rcvd = 0
Feb 22 13:33:52 inet-gw kernel: em0: XON Xmtd = 0
Feb 22 13:33:52 inet-gw kernel: em0: XOFF Rcvd = 0
Feb 22 13:33:52 inet-gw kernel: em0: XOFF Xmtd = 0
Feb 22 13:33:52 inet-gw kernel: em0: Good Packets Rcvd =
424303810
Feb 22 13:33:52 inet-gw kernel: em0: Good Packets Xmtd =
576529136
Feb 22 13:33:52 inet-gw kernel: em0: TSO Contexts Xmtd =
0
Feb 22 13:33:52 inet-gw kernel: em0: TSO Contexts Failed
= 0
Feb 22 13:34:12 inet-gw kernel: em2: Excessive
collisions = 0
Feb 22 13:34:12 inet-gw kernel: em2: Sequence errors = 0
Feb 22 13:34:12 inet-gw kernel: em2: Defer count = 20
Feb 22 13:34:12 inet-gw kernel: em2: Missed Packets =
68059
Feb 22 13:34:12 inet-gw kernel: em2: Receive No Buffers
= 275612
Feb 22 13:34:12 inet-gw kernel: em2: Receive Length
Errors = 0
Feb 22 13:34:12 inet-gw kernel: em2: Receive errors = 0
Feb 22 13:34:12 inet-gw kernel: em2: Crc errors = 0
Feb 22 13:34:12 inet-gw kernel: em2: Alignment errors =
0
Feb 22 13:34:12 inet-gw kernel: em2: Collision/Carrier
extension errors
= 0
Feb 22 13:34:12 inet-gw kernel: em2: RX overruns = 17
Feb 22 13:34:12 inet-gw kernel: em2: watchdog timeouts =
38
Feb 22 13:34:12 inet-gw kernel: em2: RX MSIX IRQ = 0 TX
MSIX IRQ = 0
LINK MSIX IRQ = 0
Feb 22 13:34:12 inet-gw kernel: em2: XON Rcvd = 21
Feb 22 13:34:12 inet-gw kernel: em2: XON Xmtd = 8344
Feb 22 13:34:12 inet-gw kernel: em2: XOFF Rcvd = 30
Feb 22 13:34:12 inet-gw kernel: em2: XOFF Xmtd = 9159
Feb 22 13:34:12 inet-gw kernel: em2: Good Packets Rcvd =
713607509
Feb 22 13:34:12 inet-gw kernel: em2: Good Packets Xmtd =
569694020
Feb 22 13:34:12 inet-gw kernel: em2: TSO Contexts Xmtd =
0
Feb 22 13:34:12 inet-gw kernel: em2: TSO Contexts Failed
= 0
Feb 22 13:35:10 inet-gw kernel: em2: Excessive
collisions = 0
Feb 22 13:35:10 inet-gw kernel: em2: Sequence errors = 0
Feb 22 13:35:10 inet-gw kernel: em2: Defer count = 20
Feb 22 13:35:10 inet-gw kernel: em2: Missed Packets =
68059
Feb 22 13:35:10 inet-gw kernel: em2: Receive No Buffers
= 275612
Feb 22 13:35:10 inet-gw kernel: em2: Receive Length
Errors = 0
Feb 22 13:35:10 inet-gw kernel: em2: Receive errors = 0
Feb 22 13:35:10 inet-gw kernel: em2: Crc errors = 0
Feb 22 13:35:10 inet-gw kernel: em2: Alignment errors =
0
Feb 22 13:35:10 inet-gw kernel: em2: Collision/Carrier
extension errors
= 0
Feb 22 13:35:10 inet-gw kernel: em2: RX overruns = 17
Feb 22 13:35:10 inet-gw kernel: em2: watchdog timeouts =
38
Feb 22 13:35:10 inet-gw kernel: em2: RX MSIX IRQ = 0 TX
MSIX IRQ = 0
LINK MSIX IRQ = 0
Feb 22 13:35:10 inet-gw kernel: em2: XON Rcvd = 21
Feb 22 13:35:10 inet-gw kernel: em2: XON Xmtd = 8344
Feb 22 13:35:10 inet-gw kernel: em2: XOFF Rcvd = 30
Feb 22 13:35:10 inet-gw kernel: em2: XOFF Xmtd = 9159
Feb 22 13:35:10 inet-gw kernel: em2: Good Packets Rcvd =
715555016
Feb 22 13:35:10 inet-gw kernel: em2: Good Packets Xmtd =
571157561
Feb 22 13:35:10 inet-gw kernel: em2: TSO Contexts Xmtd =
0
Feb 22 13:35:10 inet-gw kernel: em2: TSO Contexts Failed
= 0
Feb 22 13:39:12 inet-gw kernel: em2: Excessive
collisions = 0
Feb 22 13:39:12 inet-gw kernel: em2: Sequence errors = 0
Feb 22 13:39:12 inet-gw kernel: em2: Defer count = 20
Feb 22 13:39:12 inet-gw kernel: em2: Missed Packets =
68059
Feb 22 13:39:12 inet-gw kernel: em2: Receive No Buffers
= 275612
Feb 22 13:39:12 inet-gw kernel: em2: Receive Length
Errors = 0
Feb 22 13:39:12 inet-gw kernel: em2: Receive errors = 0
Feb 22 13:39:12 inet-gw kernel: em2: Crc errors = 0
Feb 22 13:39:12 inet-gw kernel: em2: Alignment errors =
0
Feb 22 13:39:12 inet-gw kernel: em2: Collision/Carrier
extension errors
= 0
Feb 22 13:39:12 inet-gw kernel: em2: RX overruns = 17
Feb 22 13:39:12 inet-gw kernel: em2: watchdog timeouts =
38
Feb 22 13:39:12 inet-gw kernel: em2: RX MSIX IRQ = 0 TX
MSIX IRQ = 0
LINK MSIX IRQ = 0
Feb 22 13:39:12 inet-gw kernel: em2: XON Rcvd = 21
Feb 22 13:39:12 inet-gw kernel: em2: XON Xmtd = 8344
Feb 22 13:39:12 inet-gw kernel: em2: XOFF Rcvd = 30
Feb 22 13:39:12 inet-gw kernel: em2: XOFF Xmtd = 9159
Feb 22 13:39:12 inet-gw kernel: em2: Good Packets Rcvd =
723521981
Feb 22 13:39:12 inet-gw kernel: em2: Good Packets Xmtd =
577211431
Feb 22 13:39:12 inet-gw kernel: em2: TSO Contexts Xmtd =
0
Feb 22 13:39:12 inet-gw kernel: em2: TSO Contexts Failed
= 0
Can this be the problem? "Receive No Buffers = 275612"
---- Kirk
Kirk Davis
Senior Network Analyst, ITS
Edmonton Public Schools
One Kingsway Ave.
Edmonton, Alberta, Canada
T5H 4G9
phone: 1-780-429-8308
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
