Hi Milind,

(I'm just a user not a developer so take what I say with a grain of salt :-).

AFAICT the agent/containerisation code is not NUMA-aware, so it
probably wouldn't be trivial.

Also, there are some obvious limitations with this: for example
binding processes to a specific NUMA node means that you might not
benefit from CPU bursting (e.g. if there's some available CPU on
another NUMA node).
Also NUMA binding has actually quite a few possible settings: for
example you might also want to bind the memory allocations, etc, which
means a simple flag might not be enough to achieve what you want.

One possibility I can think of might be to write your own executor -
we wrote our own executor at work for various reasons.
It's a bit of work, but it would give you unlimited flexibility in how
you start your tasks, bind them etc.

Also out of curiosity - is automatic NUMA balancing enabled on your
agents (kernel.numa_balancing sysctl)?



Le lun. 6 juil. 2020 à 19:36, Milind Chabbi <mil...@uber.com> a écrit :
> Hi,
> I have noticed that without explicit flags, the mesos-agent does not restrict 
> a cgroup of a container to any CPUSET. This has quite deleterious 
> consequences in our usage model where the OS threads in containerized 
> processes migrate to any NUMA sockets over time and lose locality to memory 
> they allocated with the first touch policy. It would take a lot of effort to 
> specify the exact CPUSET at the container launch time.
> I am wondering if the mesos agent can expose a flag (e.g., 
> --best-effort-numa-locality) so that if the requested number of CPU share and 
> memory demands meet the requirements, then the container can be launched with 
> the cgroup affinity set to a single NUMA socket and avoid any deleterious 
> effects of unrestricted CPU migration.
> -Milind

Reply via email to