On Sat, Oct 3, 2015 at 6:46 PM, Ben Greear <gree...@candelatech.com> wrote: > > > On 10/03/2015 09:29 AM, Neal Cardwell wrote: >> >> On Fri, Oct 2, 2015 at 8:21 PM, Ben Greear <gree...@candelatech.com> >> wrote: >>> >>> Gah, seems 'cubic' related. That is the default tcp cong ctrl >>> I was using (same in 3.17, for that matter). >> >> >> There have been recent changes to CUBIC that may account for this. If >> you could repeat your test with more instrumentation, eg "nstat", that >> would be very helpful. >> >> nstat > /dev/null >> # run one test >> nstat >> >> Also, if you could take a sender-side tcpdump trace of the test, that >> would be very useful (default capture length, grabbing just headers, >> is fine). > > > Here is nstat output: > > [root@ben-ota-1 ~]# nstat > #kernel > IpInReceives 14507 0.0 > IpInDelivers 14507 0.0 > IpOutRequests 49531 0.0 > TcpActiveOpens 3 0.0 > TcpPassiveOpens 2 0.0 > TcpInSegs 14498 0.0 > TcpOutSegs 50269 0.0 > UdpInDatagrams 9 0.0 > UdpOutDatagrams 1 0.0 > TcpExtDelayedACKs 43 0.0 > TcpExtDelayedACKLost 5 0.0 > TcpExtTCPHPHits 483 0.0 > TcpExtTCPPureAcks 918 0.0 > TcpExtTCPHPAcks 12758 0.0 > TcpExtTCPDSACKOldSent 5 0.0 > TcpExtTCPRcvCoalesce 49 0.0 > TcpExtTCPAutoCorking 3 0.0 > TcpExtTCPOrigDataSent 49776 0.0 > TcpExtTCPHystartTrainDetect 1 0.0 > TcpExtTCPHystartTrainCwnd 16 0.0 > IpExtInBcastPkts 8 0.0 > IpExtInOctets 2934274 0.0 > IpExtOutOctets 74817312 0.0 > IpExtInBcastOctets 640 0.0 > IpExtInNoECTPkts 14911 0.0 > [root@ben-ota-1 ~]# > > > And, you can find the pcap here: > > http://www.candelatech.com/downloads/cubic.pcap.bz2 > > Let me know if you need anything else.
Thanks! This is very useful. It looks like the sender is sending 3 (and later 4) packets every ~1.5ms for the entirety of the trace. 3 packets per burst is usually a hint that this may be related to TSQ. This slow-and-steady behavior triggers CUBIC's Hystart Train Detection to enter congestion avoidance at a cwnd of 16, which probably in turn leads to slow cwnd growth, since the sending is not cwnd-limited, but probably TSQ-limited, so cwnd does not grow in congestion avoidance mode. Probably most of the other congestion control modules do better because they stay in slow-start, which has a more aggressive criterion for growing cwnd. So this is probably at root due to the known issue with an interaction between the ath10k driver and the following change in 3.19: 605ad7f tcp: refine TSO autosizing There has been a lot of discussion about how to address the TSQ-related issues with this driver. For example, you might consider: https://patchwork.ozlabs.org/patch/438322/ But I am not sure of the latest status of that effort. Perhaps someone on the ath10k list will know. neal -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html