On Mon, Mar 8, 2021, at 16:04, Maxime Coquelin wrote: > Hi Gaetan, > > On 2/10/21 4:33 PM, Gaetan Rivet wrote: > > Expose functions reporting user configuration of offloading threads, as > > well as utility functions for multithreading. > > > > This will only expose the configuration knob to the user, while no > > datapath will implement the multiple thread request. > > > > This will allow implementations to use this API for offload thread > > management in relevant layers before enabling the actual dataplane > > implementation. > > > > The offload thread ID is lazily allocated and can as such be in a > > different order than the offload thread start sequence. > > > > The RCU thread will sometime access hardware-offload objects from > > a provider for reclamation purposes. In such case, it will get > > a default offload thread ID of 0. Care must be taken that using > > this thread ID is safe concurrently with the offload threads. > > > > Signed-off-by: Gaetan Rivet <[email protected]> > > Reviewed-by: Eli Britstein <[email protected]> > > --- > > lib/netdev-offload-provider.h | 1 + > > lib/netdev-offload.c | 88 ++++++++++++++++++++++++++++++++++- > > lib/netdev-offload.h | 19 ++++++++ > > vswitchd/vswitch.xml | 16 +++++++ > > 4 files changed, 122 insertions(+), 2 deletions(-) > > > > diff --git a/lib/netdev-offload-provider.h b/lib/netdev-offload-provider.h > > index 4815a1bd2..f8b66961d 100644 > > --- a/lib/netdev-offload-provider.h > > +++ b/lib/netdev-offload-provider.h > > @@ -84,6 +84,7 @@ struct netdev_flow_api { > > struct dpif_flow_stats *); > > > > /* Get the number of flows offloaded to netdev. > > + * 'n_flows' is an array of counters, one per offload thread. > > * Return 0 if successful, otherwise returns a positive errno value. */ > > int (*flow_get_n_flows)(struct netdev *, uint64_t *n_flows); > > > > diff --git a/lib/netdev-offload.c b/lib/netdev-offload.c > > index 3092f8d55..c30377b91 100644 > > --- a/lib/netdev-offload.c > > +++ b/lib/netdev-offload.c > > @@ -60,6 +60,12 @@ VLOG_DEFINE_THIS_MODULE(netdev_offload); > > > > static bool netdev_flow_api_enabled = false; > > > > +#define DEFAULT_OFFLOAD_THREAD_NB 1 > > +#define MAX_OFFLOAD_THREAD_NB 10 > > Out of curiosity, is there a rationale in setting the maximum number of > threads to 10? > > Maxime
Hi Maxime, The limit is mostly arbitrary. I didn't want to allow setting outlandish values, some memory allocations depends on it. In our driver, scaling is effective up to 4 threads at most, and one core use-case of OVS for us, Bluefield, has 8 cores. 10 threads seemed a reasonable limit. I think it's good to have a margin, to test with higher-than-effective numbers in development, while in production we expect on our end to run 4 threads in general. Maybe the limit could be removed, similar to n-handle-threads and n-revalidator-threads. On that point: those two will adapt to each other and depend on the number of CPU cores. Maybe we might also want to allow some budget for offload thread(s), and possibly ct_clean as well. -- Gaetan Rivet _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
