On Wed, Sep 19, 2018 at 02:47:03PM +0200, Matteo Croce wrote:
> When using the kernel datapath, OVS allocates a pool of sockets to handle
> netlink events. The number of sockets is: ports * n-handler-threads, where
> n-handler-threads is user configurable and defaults to 3/4*number of cores.
>
> This because vswitchd starts n-handler-threads threads, each one with a
> netlink socket for every port of the switch. Every thread then, starts
> listening on events on its set of sockets with epoll().
>
> On setup with lot of CPUs and ports, the number of sockets easily hits
> the process file descriptor limit, and ovs-vswitchd will exit with -EMFILE.
>
> Change the number of allocated sockets to just one per port by moving
> the socket array from a per handler structure to a per datapath one,
> and let all the handlers share the same sockets by using EPOLLEXCLUSIVE
> epoll flag which avoids duplicate events, on systems that support it.
Thanks a lot for working on this. OVS clearly uses too many fds in some
situations and we need to fix it.
I noticed that the check for EPOLLEXCLUSIVE is only a build-time check,
using #ifdef. This will only be effective if the build system is where
OVS runs and only if the build system uses its own kernel headers for
the build. Usually, in OVS, we instead try to always enable a feature
at build time, e.g. through something like:
#ifndef EPOLLEXCLUSIVE
#define EPOLLEXCLUSIVE (whatever)
#endif
and then at runtime test whether the feature is actually available and
try to use it.
Is EPOLLEXCLUSIVE safe, that is, can we miss events or end up sleeping
before processing them? If it is safe, then what invariants does OVS
need to maintain to assure them?
With this commit, the "hash" argument to dpif_port_get_pid() is never
used. The overall design of the function can be simplified. That could
be folded into this patch or go in a later patch.
Does this patch maintain the same level of thread safety as before?
Thanks,
Ben.
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
