On 4/21/21 9:40 AM, Dumitru Ceara wrote: > On 4/20/21 8:54 PM, Han Zhou wrote: >> On Tue, Apr 20, 2021 at 12:43 AM Dumitru Ceara <[email protected]> wrote: >>> >>> On 4/19/21 7:24 PM, Han Zhou wrote: >>>> On Mon, Apr 19, 2021 at 12:27 AM Dumitru Ceara <[email protected]> >> wrote: >>>>> >>>>> On 4/16/21 7:16 PM, Mark Michelson wrote: >>>>>> Thanks for the explanation Dumitru. It made good sense out of what had >>>>>> to be a difficult thing to debug. I'm assuming you went with >>>>>> unconditional monitoring since this will work with any change to a DP >>>>>> group, and not just the addition new datapaths to the group. >>>>> >>>>> There's also a case I didn't mention in the commit log (but I can add >> it >>>>> if you think it's worth adding): >>>>> >>>>> If a logical flow LF1 is not shared between datapaths, so it's only >>>>> applied to DP1, when DP2 is added that needs an identical flow, >>>>> ovn-northd might generate a fresh new DPG = {.datapaths=[DP1, DP2]}, >> and >>>>> create a new LF1' applied on datapath_group=DPG. >>>>> >>>>> Without unconditional monitoring for datapath groups there's no easy >> way >>>>> to get the notification about DPG and LF1' being added in the same >>>>> jsonrpc update from SB with the delete for LF. >>>>> >>>>> Given that the number of datapath groups is expected to be relatively >>>>> low compared to the total number of logical flows, unconditional >>>>> monitoring seems like the best solution. >>>>> >>>>>> >>>>>> Acked-by: Mark Michelson <[email protected]> >>>>> >>>>> Thanks! >>>>> >>>> >>>> +1 (although late) >>>> >>>> I am catching up the DPG implementation, and here are some thoughts on >> the >>>> use cases. For my understanding, except for the DPG that contains all >> DPs, >>>> most DPGs are mapping to port-groups, which usually map to >>>> tenants/namespaces. The conditional monitoring feature (when >>>> ovn-monitor-all is disabled) is useful mainly when there are a large >> number >>>> of small tenants and they don't need to talk to each other, so their >>>> workloads may reside on different HVs and DPs, so conditional monitoring >>>> can result in a smaller number of lflows on each HV in this scenario. >>>> However, when DPG related flows are unconditionally monitored, it makes >>>> conditional monitoring not as useful as it should be when there are a >> lot >>>> of ACLs, because ACLs are the major user of the port-group based DPGs, >>>> which may contribute to a significant portion of lflows. In such >> scenarios >>>> (when there are many small tenants), maybe it is better to disable DPG >> (and >>>> disable ovn-monitor-all) to benefit more from the conditional monitoring >>> >>> I think DPGs may be useful in this case too (many small tenants) with >>> conditional monitoring enabled. >>> >>> There's a reasonable number of "trivial" logical flows, e.g., the flows >>> that advance to next table by default, that are shared between all >>> logical datapaths and DPGs will optimize those. >>> >>> For example, on one such large downstream OpenStack deployment we saw >>> IIRC a reduction of ~20% w.r.t. number of logical flows and SB size when >>> enabling DPG. It's not a huge factor but it's still a relevant >>> reduction in number of objects in the SB that need to be sent to all >>> ovn-controllers. >> >> It is true that DPG will still reduce SB size on the central nodes, but my >> above point is about the lflows being conditionally monitored by each >> individual HV, because usually central DB size is not a concern if >> ovn-monitor-all is disabled. In the particular scenario of large amount of >> small tenants, without DPG, a HV may be interested in only a small amount >> of DPs which has a small amount of flows in total. With DPG enabled, >> although sharing the common flows between the DPs would reduce some >> redundant flows, but since the number of interested DPs is small for each >> HV, this savings may be small. On the other hand, since all PG related ACL >> flows for all tenants (all DPs) are now monitored, each HV could have a >> huge increase of unnecessary ACL flows from unrelated tenants. That's why I >> think in such a scenario DPG should be disabled. > > I see, you're right, makes sense. Another way to make this work in all > scenarios is to add two-level deep conditional monitoring to the IDL and > ovsdb-server, that is, allow ovn-controller to conditionally monitor > "all logical flows that refer to DPG that contain datapaths equal to a > specific value".
Another (probably, easier in implementation?) option is to avoid flow updates if we have in-flight condition changes, i.e. if we do not have a full view over logical flows. > >> >>> >>>> feature. In the contrary use cases when there are few but large tenants, >>>> each of them maps to a large port-group, crossing a large amount of >>>> datapaths, then enabling DPG may be a better choice, and at the same >> time >>>> enabling ovn-monitor-all may be recommended. >>>> >>>> Is the above understanding correct? >>>> >>> >>> I think so. >>> >>> Maybe it makes sense to try to add some guidelines about when to enable >>> the datapath groups feature and how to combine it with conditional >>> monitoring, maybe as part of the OVN documentation? What do you think? >>> >> Yes, I agree we should document the guidelines. >> >>> Thanks, >>> Dumitru >>> >>> >> _______________________________________________ >> dev mailing list >> [email protected] >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
