On Fri, Apr 17, 2020 at 12:56 PM Han Zhou <[email protected]> wrote: > > > > On Tue, Apr 7, 2020 at 7:03 AM Maciej Jozefczyk <[email protected]> wrote: > > > > Hello! > > > > I would like to ask you to clarify how the OVN Load balancing algorithm > > works. > > > > Based on the action [1]: > > 1) If connection is alive the same 'backend' will be chosen, > > > > 2) If it is a new connection the backend will be chosen based on > > selection_method=dp_hash [2]. > > Based on changelog the dp_hash uses '5 tuple hash' [3]. > > The hash is calculated based on values: source and destination IP, source > > port, protocol and arbitrary value - 42. [4] > > Based on that information we could name it SOURCE_IP_PORT. > > > > Unfortunately we recently got a bug report in OVN Octavia provider driver > > project, that the Load Balancing in OVN > > works differently [5]. The report shows even when the test uses the same > > source ip and port, but new TCP connection, > > traffic is randomly distributed, but based on [2] it shouldn't? > > > > Is it a bug? Is something else taken to account while creating a hash? Can > > it be fixed in OVS/OVN? > > > > > > > > Thanks, > > Maciej > > > > > > [1] https://github.com/ovn-org/ovn/blob/branch-20.03/lib/actions.c#L1017 > > [2] https://github.com/ovn-org/ovn/blob/branch-20.03/lib/actions.c#L1059 > > [3] > > https://github.com/openvswitch/ovs/blob/d58b59c17c70137aebdde37d3c01c26a26b28519/NEWS#L364-L371 > > [4] > > https://github.com/openvswitch/ovs/blob/74286173f4d7f51f78e9db09b07a6d4d65263252/lib/flow.c#L2217 > > [5] https://bugs.launchpad.net/neutron/+bug/1871239 > > > > -- > > Best regards, > > Maciej Józefczyk > > _______________________________________________ > > discuss mailing list > > [email protected] > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > Hi Maciej, > > Thanks for reporting. It is definitely strange that same 5-tuple flow > resulted in hitting different backends. I didn't observed such behavior > before (maybe I should try again myself to confirm). Can you make sure during > the testing the group bucket didn't change? You can do so by: > # ovs-ofctl dump-groups br-int > and also check the group stats and see if multiple buckets has counter > increased during the test > # ovs-ofctl dump-group-stats br-int [group] > > For the 5-tuple hash function you are seeing flow_hash_5tuple(), it is using > all the 5-tuples. It adds both ports (src and dst) at once: > /* Add both ports at once. */ > hash = hash_add(hash, > ((const uint32_t *)flow)[offsetof(struct flow, tp_src) > / sizeof(uint32_t)]); > > The tp_src is the start of the offset, and the size is 32, meaning both src > and dst, each is 16 bits. (Although I am not sure if dp_hash method is using > this function or not. Need to check more code) > > BTW, I am not sure why Neutron give it the name SOURCE_IP_PORT. Shall it be > called just 5-TUPLE, since protocol, destination IP and PORT are also > considered in the hash. >
Hi Maciej and Han, I did some testing and I can confirm as you're saying. OVN is not choosing the same backend with the src ip, src port fixed. I think there is an issue with OVN on how it is programming the group flows. OVN is setting the selection_method as dp_hash. But when ovs-vswitchd receives the GROUP_MOD openflow message, I noticed that the selection_method is not set. From the code I see that selection_method will be encoded only if ovn-controller uses openflow version 1.5 [1] Since selection_method is NULL, vswitchd uses the dp_hash method [2]. dp_hash means it uses the hash calculated by the datapath. In the case of kernel datapath, from what I understand it uses skb_get_hash(). I modified the vswitchd code to use the selection_method "hash" if selection_method is not set. In this case the load balancer works as expected. For a fixed src ip, src port, dst ip and dst port, the group action is selecting the same bucket always. [3] I think we need to fix a few issues in OVN - Use openflow 1.5 so that ovn can set selection_method - Use "hash" method if dp_hash is not choosing the same bucket for 5-tuple hash. - May be provide the option for the CMS to choose an algorithm i.e. to use dp_hash or hash. I'll look into it on how to support this. [1] - https://github.com/openvswitch/ovs/blob/master/lib/ofp-group.c#L2120 https://github.com/openvswitch/ovs/blob/master/lib/ofp-group.c#L2082 [2] - https://github.com/openvswitch/ovs/blob/master/ofproto/ofproto-dpif.c#L5108 [3] - https://github.com/openvswitch/ovs/blob/master/ofproto/ofproto-dpif-xlate.c#L4553 Thanks Numan > Thanks, > Han > _______________________________________________ > discuss mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss _______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
