Qian Zhang created MESOS-4353:
---------------------------------
Summary: Limit the number of processes created by libprocess
Key: MESOS-4353
URL: https://issues.apache.org/jira/browse/MESOS-4353
Project: Mesos
Issue Type: Improvement
Components: libprocess
Reporter: Qian Zhang
Assignee: Qian Zhang
Currently libprocess will create {{max(8, number of CPU cores)}} processes
during the initialization, see
https://github.com/apache/mesos/blob/0.26.0/3rdparty/libprocess/src/process.cpp#L2146
for details. This should be OK for a normal machine which has no much cores
(e.g., 16, 32), but for a powerful machine which may have a large number of
cores (e.g., an IBM Power machine may have 192 cores), this will cause too much
worker threads which are not necessary.
And since libprocess is widely used in Mesos (master, agent, scheduler,
executor), it may also cause some performance issue. For example, when user
creates a Docker container via Mesos in a Mesos agent which is running on a
powerful machine with 192 cores, the DockerContainerizer in Mesos agent will
create a dedicated executor for the container, and there will be 192 worker
threads in that executor. And if user creates 1000 Docker containers in that
machine, then there will be 1000 executors, i.e., 1000 * 192 worker threads
which is a large number and may thrash the OS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
