[
https://issues.apache.org/jira/browse/MESOS-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109695#comment-15109695
]
Maged Michael commented on MESOS-4353:
--------------------------------------
Some options:
(1) Use an environment variable to set the maximum number of libprocess worker
threads per process. Such value can be set according to the role of the Mesos
process. For example, a Mesos Master process may have a higher maximum number
of libprocess worker threads than a Docker Containerizer.
(2) Automatically detect the type of the Mesos process (master, executor, ...)
and set the maximum number of libprocess worker threads accordingly.
(3) Set a hardwired maximum number of worker threads regardless of the type of
the Mesos process.
I think option (1) is the most flexible.
> Limit the number of processes created by libprocess
> ---------------------------------------------------
>
> Key: MESOS-4353
> URL: https://issues.apache.org/jira/browse/MESOS-4353
> Project: Mesos
> Issue Type: Improvement
> Components: libprocess
> Reporter: Qian Zhang
> Assignee: Qian Zhang
>
> Currently libprocess will create {{max(8, number of CPU cores)}} processes
> during the initialization, see
> https://github.com/apache/mesos/blob/0.26.0/3rdparty/libprocess/src/process.cpp#L2146
> for details. This should be OK for a normal machine which has no much cores
> (e.g., 16, 32), but for a powerful machine which may have a large number of
> cores (e.g., an IBM Power machine may have 192 cores), this will cause too
> much worker threads which are not necessary.
> And since libprocess is widely used in Mesos (master, agent, scheduler,
> executor), it may also cause some performance issue. For example, when user
> creates a Docker container via Mesos in a Mesos agent which is running on a
> powerful machine with 192 cores, the DockerContainerizer in Mesos agent will
> create a dedicated executor for the container, and there will be 192 worker
> threads in that executor. And if user creates 1000 Docker containers in that
> machine, then there will be 1000 executors, i.e., 1000 * 192 worker threads
> which is a large number and may thrash the OS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
