On 15 September 2015 at 04:16, Hans Petter Selasky <[email protected]> wrote: > On 09/14/15 20:35, Adrian Chadd wrote: >> >> Hi, >> >> So what's the actual behaviour of the new tso logic before and after >> the above change in tsomax? > > > Hi, > > The behaviour is the same, only the limits have changed a bit. > >> like, what are the actual packet sizes >> >> being sent up to the hardware? > > > It is not about the packet sizes, it is about the number of packets we > transmit per TSO block. > >> Is TSO or the TCP stack so fragile that >> a slight change in how packets are broken up results in ridiculously >> less throughput? It's only a few bytes. > > > Network adapters which support TSO typically has a hard limit on the number > of mbufs it can load. When we exceed this limit, m_defrag() or packet drop > is next. It is the responsibility of the TCP stack to generated mbuf chains > which are within the network adapter given limits. Previously only the > length was accounted for. Now we also account for the number of segments, > because there are many ways a 64K bytes long mbuf chain can be generated.
I know all of this. What I'm asking is different - what about the change(s) that broke/fixed the Xen network performance actually caused the change in behaviour? I know things are sensitive to how the mbufs are broken up - I'd like to see exactly this. Whilst futzing around at Netflix on TCP behaviour, I found with everything lined up correctly I'd get 40gbit TCP over a lot of sockets on sandy bridge hardware - but only as long as the TCP send buffer size resulted in nicely 4k segments. Some (larger) send buffer sizes resulted in the stack settling on non-4k segments for whatever reason, which drastically changed TSO performance. So, subtle changes had much bigger effects. I'm asking what specifically is going on here. +1 on the "going to make things more correct" path, but -2 on the "subtle changes and big unintended, un-described effects" results :( -adrian _______________________________________________ [email protected] mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "[email protected]"
