Hi Andy, On Tue, Aug 15, 2017 at 01:31:36PM -0700, Andy Zhou wrote: > On Tue, Aug 15, 2017 at 9:13 AM, Simon Horman > <[email protected]> wrote: > > From: Pieter Jansen van Vuuren <[email protected]> > > > > Adds a config parameter that determines if a bond will use recirculation > > in the kernel datapath when implementing a bond in balance-tcp mode. > > Using recirc for bonding is a lower level decision. I am not sure > OVSDB is right level > for exposing this detail.
I do see that this configuration is arguably more detailed than existing bond parameters set via OVSDB, but it is still a bond parameter so perhaps it isn't entirely out of place here. > > The default for enable-recirc is "true", resulting in the traditional > > implementation of a bond in balance-tcp mode. Setting enable-recirc to > > false results in datapath rules that do not rely on the recirculation > > action. > > > > example usage: > > ovs-vsctl set port bond0 other_config:enable-recirc=false > > > > Advantages: > > - Allows TC offloading of OVS bonds on hardware which > > does not support recirculation > > Not sure it is an advantage. Bonding with recirc allows the first flow to > be a mega flow. without megaflow, all bond traffic has to be micro > flows, which was how OVS implements bonding. The recirc was introduced to > solve the flow explosion problem > > Without supporting recirc, I'd image TC offloading would suffer the same > issue. May be it is correct not to offload the bond flows, at least not > before recirc is supported in TC. > > Another way to look at this is that when TC ran out of flow space, and > bonding is supported by the kernel module. This configuration would > prevent kernel module from using recirc. I believe that your points regarding flow explosion and fallback to the kernel datapath are well made. However, I also believe there is hardware where using recirculation is not practical. For that reason I think it is worth exploring this proposal further. > > - Appears to result in lower latency (in systems using few flows). > > Not sure this is true in general. When using recirc with proper > megaflow, the latency should > improve for new microflows since upcalls can be omitted. That may be true in terms of flow setup cost but I believe that the per-packet cost is going to be higher when recirculation is used. > > Quick ping test results in: > > > > other_config:enable-recirc=false > > rtt min/avg/max/mdev = 0.039/0.193/7.612/1.059 ms > > > > other_config:enable-recirc=true > > rtt min/avg/max/mdev = 0.038/0.321/14.091/1.967 ms > > > > When setting up the flow, 'recirc" requires 2 miss upcalls, thus set > up latency is larger. > However, once flows are set up, the latency difference should be minimal. > > I'd expect the max latency to be different with or without recirc. > Notice the minimal latency difference is small. > I'd expect the average latency to converge to minimal for sufficiently > large number of ping packets. Latency may not be the best way to measure this but having to classify each packet twice, as I believe is the case when recirculation is used, must cost something. > > More comprehensive testing is in progress. > > > > Signed-off-by: Pieter Jansen van Vuuren > > <[email protected]> > > Signed-off-by: Simon Horman <[email protected]> _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
