On Thu, Jul 20, 2017 at 10:21 AM, Ilya Maximets <[email protected]> wrote: > There are 3 constraints for moving hashes from one slave to another: > > 1. The load difference is larger than ~3% of one slave's load. > 2. The load difference between slaves exceeds 100000 bytes. > 3. Moving of the hash makes the load difference lower by > 10%. > > In current implementation if one of the slaves goes DOWN state, all > the hashes assigned to it will be moved to other slaves. After that, > if slave will go UP it will wait for rebalancing to get some hashes. > But in case where we have more than 10 equally loaded hashes it > will never fit constraint #3, because each hash will handle less than > 10% of the load. Situation become worse when number of flows grows > higher and it's almost impossible to migrate any hash when all the > 256 hash entries are used which is very likely when we have few > hundreds/thousands of flows. > > As a result, if one of the slaves goes down and up while traffic > flows, it will never be used again for packet transmission. > Situation will not be fixed even if we'll stop traffic completely > and start it again because first two constraints will block > rebalancing on the earlier stages while we have low amount of traffic. > > Moving of one hash if destination has no hashes as it was before > commit c460a6a7bc75 ("ofproto/bond: simplify rebalancing logic") > will not help because having one hash isn't enough to make load > difference less than 10% of total load and this slave will > handle only that one hash forever. > > To fix this lets try to move few hashes simultaniously to fit > constraint #3.
Thanks for working on this. Current interface tries to only rebalance one hash bucket at a time. There are two reasons I can think of: In general, It is good idea to keep flow binding stable. Moving flow introduces side effects, such as packet re-ordering, that impacts application performances significantly. The second reason is the bandwidth each flow consumes tends to be more dynamic in practice. It is usually pointless to use a snapshot of bandwidth consumption to come up a "perfect" bond port binding. For example, TCP flows rate adapt to available bandwidth. It is probably better to make smaller changes in each rebalancing step. Is it still possible to address the issue mentioned while maintaining current interface? Can we find some way to break the tie in case current algorithm fail to nominate any bucket to move? Thanks. _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
