On Tue, Apr 20, 2021 at 12:43 AM Dumitru Ceara <[email protected]> wrote: > > On 4/19/21 7:24 PM, Han Zhou wrote: > > On Mon, Apr 19, 2021 at 12:27 AM Dumitru Ceara <[email protected]> wrote: > >> > >> On 4/16/21 7:16 PM, Mark Michelson wrote: > >>> Thanks for the explanation Dumitru. It made good sense out of what had > >>> to be a difficult thing to debug. I'm assuming you went with > >>> unconditional monitoring since this will work with any change to a DP > >>> group, and not just the addition new datapaths to the group. > >> > >> There's also a case I didn't mention in the commit log (but I can add it > >> if you think it's worth adding): > >> > >> If a logical flow LF1 is not shared between datapaths, so it's only > >> applied to DP1, when DP2 is added that needs an identical flow, > >> ovn-northd might generate a fresh new DPG = {.datapaths=[DP1, DP2]}, and > >> create a new LF1' applied on datapath_group=DPG. > >> > >> Without unconditional monitoring for datapath groups there's no easy way > >> to get the notification about DPG and LF1' being added in the same > >> jsonrpc update from SB with the delete for LF. > >> > >> Given that the number of datapath groups is expected to be relatively > >> low compared to the total number of logical flows, unconditional > >> monitoring seems like the best solution. > >> > >>> > >>> Acked-by: Mark Michelson <[email protected]> > >> > >> Thanks! > >> > > > > +1 (although late) > > > > I am catching up the DPG implementation, and here are some thoughts on the > > use cases. For my understanding, except for the DPG that contains all DPs, > > most DPGs are mapping to port-groups, which usually map to > > tenants/namespaces. The conditional monitoring feature (when > > ovn-monitor-all is disabled) is useful mainly when there are a large number > > of small tenants and they don't need to talk to each other, so their > > workloads may reside on different HVs and DPs, so conditional monitoring > > can result in a smaller number of lflows on each HV in this scenario. > > However, when DPG related flows are unconditionally monitored, it makes > > conditional monitoring not as useful as it should be when there are a lot > > of ACLs, because ACLs are the major user of the port-group based DPGs, > > which may contribute to a significant portion of lflows. In such scenarios > > (when there are many small tenants), maybe it is better to disable DPG (and > > disable ovn-monitor-all) to benefit more from the conditional monitoring > > I think DPGs may be useful in this case too (many small tenants) with > conditional monitoring enabled. > > There's a reasonable number of "trivial" logical flows, e.g., the flows > that advance to next table by default, that are shared between all > logical datapaths and DPGs will optimize those. > > For example, on one such large downstream OpenStack deployment we saw > IIRC a reduction of ~20% w.r.t. number of logical flows and SB size when > enabling DPG. It's not a huge factor but it's still a relevant > reduction in number of objects in the SB that need to be sent to all > ovn-controllers.
It is true that DPG will still reduce SB size on the central nodes, but my above point is about the lflows being conditionally monitored by each individual HV, because usually central DB size is not a concern if ovn-monitor-all is disabled. In the particular scenario of large amount of small tenants, without DPG, a HV may be interested in only a small amount of DPs which has a small amount of flows in total. With DPG enabled, although sharing the common flows between the DPs would reduce some redundant flows, but since the number of interested DPs is small for each HV, this savings may be small. On the other hand, since all PG related ACL flows for all tenants (all DPs) are now monitored, each HV could have a huge increase of unnecessary ACL flows from unrelated tenants. That's why I think in such a scenario DPG should be disabled. > > > feature. In the contrary use cases when there are few but large tenants, > > each of them maps to a large port-group, crossing a large amount of > > datapaths, then enabling DPG may be a better choice, and at the same time > > enabling ovn-monitor-all may be recommended. > > > > Is the above understanding correct? > > > > I think so. > > Maybe it makes sense to try to add some guidelines about when to enable > the datapath groups feature and how to combine it with conditional > monitoring, maybe as part of the OVN documentation? What do you think? > Yes, I agree we should document the guidelines. > Thanks, > Dumitru > > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
