[
https://issues.apache.org/jira/browse/MESOS-6264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549111#comment-15549111
]
Joris Van Remoortere edited comment on MESOS-6264 at 10/5/16 3:45 PM:
----------------------------------------------------------------------
cc [~vinodkone][~jieyu]
The bulk of this comes from loading in {{libmesos.so}}
We do this because the autoconf build treats libmesos as a dynamic dependency.
Since we load libmesos dynamically, there is no chance for the linker to strip
unused code. This means that all of the code in libmesos regardless of use gets
loaded into resident memory.
In contrast the cmake build generates a static library for {{libmesos.a}}. This
is then used to build the {{mesos-executor}} binary without a dynamic
dependency on libmesos. The benefit of this approach is that the linker is able
to strip out all unused code. In an optimized build this is {{~10MB}}.
Some approaches for the quick win are:
# Consider using the cmake build. This only needs to be modified slightly to
strip symbols from the final executor binary {{-s}}.
# Modiy the autoconf build to build a {{libmesos.a}} so that we can statically
link it in to the {{mesos-executor}} binary and allow the linker to strip
unused code.
Regardless of the above approach, {{libmesos}} would still be by far the
largest contributor of the {{RSS}}. This is for 2 reasons:
# Much of our code is structured such that the linker can't determine if it is
unused. We would need to adjust our patterns such that the unused code analyzer
can do a better job.
# Much of our code is {{inlined}} or written such that it can't be optimized. 2
examples are:
##
https://github.com/apache/mesos/blob/9beb8eae6408249cdb3e2f16ba68b31a00d3452c/3rdparty/libprocess/include/process/mime.hpp#L35-L154
This code could be moved to a {{.cpp}} file and should be a {{static const
std::unordered_map<string, string>}} that we {{insert(begin(), end())}} into
{{types}}. This would reduce the size of libmesos by {{~20KB}}!
##
https://github.com/apache/mesos/blob/9beb8eae6408249cdb3e2f16ba68b31a00d3452c/3rdparty/libprocess/include/process/http.hpp#L453-L517
This code and sibling {{struct Request}} have auto-generated {{inlined}}
destructors. These are very expensive. Just declaring and then defining in the
{{.cpp}} the default destructor can remove another {{~20KB}} each from
libmesos. There are plenty of other opportunities like this scattered through
the codebase. It's work to find them and the returns are small for each, but
end up adding to much of the {{9MB}} left over.
was (Author: jvanremoortere):
cc [~vinodkone][~jieyu]
The bulk of this comes from loading in {{libmesos.so}}
We do this because the autoconf build treats libmesos as a dynamic dependency.
Since we load libmesos dynamically, there is no chance for the linker to strip
unused code. This means that all of the code in libmesos regardless of use gets
loaded into resident memory.
In contrast the cmake build generates a static library for {{libmesos.a}}. This
is then used to build the {{mesos-executor}} binary without a dynamic
dependency on libmesos. The benefit of this approach is that the linker is able
to strip out all unused code. In an optimized build this is {{~10MB}}.
Some approaches for the quick win are:
# Consider using the cmake build. This only needs to be modified slightly to
strip symbols from the final executor binary {{-s}}.
# Modiy the autoconf build to build a {{libmesos.a}} so that we can statically
link it in to the {{mesos-executor}} binary and allow the linker to strip
unused code.
Regardless of the above approach, {{libmesos}} would still be by far the
largest contributor of the {{RSS}}. This is for 2 reasons:
# Much of our code is structured such that the linker can't determine if it is
unused. We would need to adjust our patterns such that the unused code analyzer
can do a better job.
# Much of our code is {{inlined}} or written such that it can't be optimized. 2
examples are:
##
https://github.com/apache/mesos/blob/9beb8eae6408249cdb3e2f16ba68b31a00d3452c/3rdparty/libprocess/include/process/mime.hpp#L35-L154
This code could be moved to a {{.cpp}} file and should be a {{static const
std::unordered_map<string, string>}} that we {{insert(begin(), end())}} into
{{types}}. This would reduce the size of libmesos by {{~20KB}}!
##
https://github.com/apache/mesos/blob/master/3rdparty/libprocess/include/process/http.hpp#L453-L517
This code and sibling {{struct Request}} have auto-generated {{inlined}}
destructors. These are very expensive. Just declaring and then defining in the
{{.cpp}} the default destructor can remove another {{~20KB}} each from
libmesos. There are plenty of other opportunities like this scattered through
the codebase. It's work to find them and the returns are small for each, but
end up adding to much of the {{9MB}} left over.
> Investigate the high memory usage of the default executor.
> ----------------------------------------------------------
>
> Key: MESOS-6264
> URL: https://issues.apache.org/jira/browse/MESOS-6264
> Project: Mesos
> Issue Type: Bug
> Reporter: Anand Mazumdar
> Labels: mesosphere
> Fix For: 1.1.0
>
> Attachments: pmap_output_for_the_default_executor.txt
>
>
> It seems that a default executor with two sleep tasks is using ~32 mb on
> average and can sometimes lead to it being killed for some tests like
> {{SlaveRecoveryTest/0.ROOT_CGROUPS_ReconnectDefaultExecutor}} on our internal
> CI. Attached the {{pmap}} output for the default executor. Please note that
> the command executor memory usage is also pretty high (~26 mb).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
