On 4/20/21 8:54 PM, Han Zhou wrote: > On Tue, Apr 20, 2021 at 12:43 AM Dumitru Ceara <[email protected]> wrote: >> >> On 4/19/21 7:24 PM, Han Zhou wrote: >>> On Mon, Apr 19, 2021 at 12:27 AM Dumitru Ceara <[email protected]> > wrote: >>>> >>>> On 4/16/21 7:16 PM, Mark Michelson wrote: >>>>> Thanks for the explanation Dumitru. It made good sense out of what had >>>>> to be a difficult thing to debug. I'm assuming you went with >>>>> unconditional monitoring since this will work with any change to a DP >>>>> group, and not just the addition new datapaths to the group. >>>> >>>> There's also a case I didn't mention in the commit log (but I can add > it >>>> if you think it's worth adding): >>>> >>>> If a logical flow LF1 is not shared between datapaths, so it's only >>>> applied to DP1, when DP2 is added that needs an identical flow, >>>> ovn-northd might generate a fresh new DPG = {.datapaths=[DP1, DP2]}, > and >>>> create a new LF1' applied on datapath_group=DPG. >>>> >>>> Without unconditional monitoring for datapath groups there's no easy > way >>>> to get the notification about DPG and LF1' being added in the same >>>> jsonrpc update from SB with the delete for LF. >>>> >>>> Given that the number of datapath groups is expected to be relatively >>>> low compared to the total number of logical flows, unconditional >>>> monitoring seems like the best solution. >>>> >>>>> >>>>> Acked-by: Mark Michelson <[email protected]> >>>> >>>> Thanks! >>>> >>> >>> +1 (although late) >>> >>> I am catching up the DPG implementation, and here are some thoughts on > the >>> use cases. For my understanding, except for the DPG that contains all > DPs, >>> most DPGs are mapping to port-groups, which usually map to >>> tenants/namespaces. The conditional monitoring feature (when >>> ovn-monitor-all is disabled) is useful mainly when there are a large > number >>> of small tenants and they don't need to talk to each other, so their >>> workloads may reside on different HVs and DPs, so conditional monitoring >>> can result in a smaller number of lflows on each HV in this scenario. >>> However, when DPG related flows are unconditionally monitored, it makes >>> conditional monitoring not as useful as it should be when there are a > lot >>> of ACLs, because ACLs are the major user of the port-group based DPGs, >>> which may contribute to a significant portion of lflows. In such > scenarios >>> (when there are many small tenants), maybe it is better to disable DPG > (and >>> disable ovn-monitor-all) to benefit more from the conditional monitoring >> >> I think DPGs may be useful in this case too (many small tenants) with >> conditional monitoring enabled. >> >> There's a reasonable number of "trivial" logical flows, e.g., the flows >> that advance to next table by default, that are shared between all >> logical datapaths and DPGs will optimize those. >> >> For example, on one such large downstream OpenStack deployment we saw >> IIRC a reduction of ~20% w.r.t. number of logical flows and SB size when >> enabling DPG. It's not a huge factor but it's still a relevant >> reduction in number of objects in the SB that need to be sent to all >> ovn-controllers. > > It is true that DPG will still reduce SB size on the central nodes, but my > above point is about the lflows being conditionally monitored by each > individual HV, because usually central DB size is not a concern if > ovn-monitor-all is disabled. In the particular scenario of large amount of > small tenants, without DPG, a HV may be interested in only a small amount > of DPs which has a small amount of flows in total. With DPG enabled, > although sharing the common flows between the DPs would reduce some > redundant flows, but since the number of interested DPs is small for each > HV, this savings may be small. On the other hand, since all PG related ACL > flows for all tenants (all DPs) are now monitored, each HV could have a > huge increase of unnecessary ACL flows from unrelated tenants. That's why I > think in such a scenario DPG should be disabled.
I see, you're right, makes sense. Another way to make this work in all scenarios is to add two-level deep conditional monitoring to the IDL and ovsdb-server, that is, allow ovn-controller to conditionally monitor "all logical flows that refer to DPG that contain datapaths equal to a specific value". > >> >>> feature. In the contrary use cases when there are few but large tenants, >>> each of them maps to a large port-group, crossing a large amount of >>> datapaths, then enabling DPG may be a better choice, and at the same > time >>> enabling ovn-monitor-all may be recommended. >>> >>> Is the above understanding correct? >>> >> >> I think so. >> >> Maybe it makes sense to try to add some guidelines about when to enable >> the datapath groups feature and how to combine it with conditional >> monitoring, maybe as part of the OVN documentation? What do you think? >> > Yes, I agree we should document the guidelines. > >> Thanks, >> Dumitru >> >> > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
