Re: Recurring trace from tcp_fragment()

2015-06-04 Thread Yuchung Cheng
On Thu, Jun 4, 2015 at 1:56 PM, Martin KaFai Lau wrote: > > On Thu, Jun 04, 2015 at 01:10:26PM -0700, Grant Zhang wrote: > > Hi Martin, > > > > Thank you! My net.ipv4.tcp_mtu_probing is 1. After turning it off, > > the WARN_ON stack is gone. > Thanks for confirming it. > > > Could you elaborate a

Re: Recurring trace from tcp_fragment()

2015-06-04 Thread Martin KaFai Lau
On Thu, Jun 04, 2015 at 01:10:26PM -0700, Grant Zhang wrote: > Hi Martin, > > Thank you! My net.ipv4.tcp_mtu_probing is 1. After turning it off, > the WARN_ON stack is gone. Thanks for confirming it. > Could you elaborate a bit on why this setting relates to the WARN_ON > trace? The WARN_ON is c

Re: Recurring trace from tcp_fragment()

2015-06-04 Thread Grant Zhang
Hi Martin, Thank you! My net.ipv4.tcp_mtu_probing is 1. After turning it off, the WARN_ON stack is gone. Could you elaborate a bit on why this setting relates to the WARN_ON trace? And what are the pros/cons for disabling mtu_probing? Thanks, Grant On 6/4/15 12:38 PM, Martin KaFai Lau wro

Re: Recurring trace from tcp_fragment()

2015-06-04 Thread Martin KaFai Lau
Hi Grant, On Thu, Jun 04, 2015 at 09:35:04AM -0700, Grant Zhang wrote: > Hi Neal, > > Unfortunately with the patch we still see the same stack trace. > Attached is the TcpExtTCPSACKReneging with the patch, captured with > 60 seconds interval. Its value is incremented at an similar speed as > befor

Re: Recurring trace from tcp_fragment()

2015-06-04 Thread Neal Cardwell
On Thu, Jun 4, 2015 at 12:35 PM, Grant Zhang wrote: > Hi Neal, > > Unfortunately with the patch we still see the same stack trace. Attached is > the TcpExtTCPSACKReneging with the patch, captured with 60 seconds interval. > Its value is incremented at an similar speed as before, about 30/minute. >

Re: Recurring trace from tcp_fragment()

2015-06-04 Thread Grant Zhang
Hi Neal, Unfortunately with the patch we still see the same stack trace. Attached is the TcpExtTCPSACKReneging with the patch, captured with 60 seconds interval. Its value is incremented at an similar speed as before, about 30/minute. If you want to collect any other data, please feel free t

Re: Recurring trace from tcp_fragment()

2015-05-30 Thread Neal Cardwell
On Sat, May 30, 2015 at 2:52 PM, Grant Zhang wrote: > Thank you Neal. Most likely I will test the patch on Monday and report > back the result. > > As for the TcpExtTCPSACKReneging counter, attached is the captured > counter value on a 1-second interval for 10 minutes. OK, great. Those TcpExtTCPS

Re: Recurring trace from tcp_fragment()

2015-05-30 Thread Grant Zhang
Thank you Neal. Most likely I will test the patch on Monday and report back the result. As for the TcpExtTCPSACKReneging counter, attached is the captured counter value on a 1-second interval for 10 minutes. Thanks, Grant reneg.log Description: Binary data > On May 30, 2015, at 10:29 AM,

Re: Recurring trace from tcp_fragment()

2015-05-30 Thread Neal Cardwell
On Fri, May 29, 2015 at 3:53 PM, Grant Zhang wrote: > Hi Neal, > > I will be more happy to test the patch. Please send it my way. Great. Thank you so much for being willing to do this. Attached is a patch for testing. I generated it and tested it relative to Linux v3.14.39, since your stack trace

Re: Recurring trace from tcp_fragment()

2015-05-29 Thread Grant Zhang
Hi Neal, I will be more happy to test the patch. Please send it my way. Thanks, Grant > On May 29, 2015, at 12:46 PM, Neal Cardwell wrote: > > On Fri, May 29, 2015 at 3:21 PM, Grant Zhang wrote: >> We have multiple machines running into the following trace repeatedly. The >> trace shows up

Re: Recurring trace from tcp_fragment()

2015-05-29 Thread Neal Cardwell
On Fri, May 29, 2015 at 3:21 PM, Grant Zhang wrote: > We have multiple machines running into the following trace repeatedly. The > trace shows up every couple of seconds on our production machines. > ... > May 29 18:14:04 cache-fra1230 kernel:[3080455.796188] [] > tcp_fragment+0x2e4/0x2f0 > May

Recurring trace from tcp_fragment()

2015-05-29 Thread Grant Zhang
We have multiple machines running into the following trace repeatedly. The trace shows up every couple of seconds on our production machines. May 29 18:14:04 cache-fra1230 kernel:[3080455.796143] WARNING: CPU: 7 PID: 0 at net/ipv4/tcp_output.c:1082 tcp_fragment+0x2e4/0x2f0() May 29 18:14:04 cach