[
https://issues.apache.org/jira/browse/IMPALA-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535031#comment-16535031
]
Tim Armstrong commented on IMPALA-7239:
---------------------------------------
It looks like there was a known bug that makes walking smaps O(n^2) in the
number of threads. https://lkml.org/lkml/2016/1/19/488. It's been fixed in
later kernel versions but it seems like it's present in a lot of CentOS7
kernels. There's also some work to produce a summarised version of smaps that
is cheaper to produce: https://patchwork.kernel.org/patch/9888155/
Anyway, I think we need to turn off the smaps parsing, which means removing
some of the metrics.
> Mitigate ParseSmaps() overhead
> ------------------------------
>
> Key: IMPALA-7239
> URL: https://issues.apache.org/jira/browse/IMPALA-7239
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.10.0, Impala 2.11.0, Impala 3.0, Impala 2.12.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: perf, resource-management
>
> I've heard anecdotes of high system time spent in functions related this the
> smap parsing. It appears that this can be expensive on systems once the
> impalad virtual memory gets fragmented and there are 10s of thousands of maps.
> We can try to mitigate by reducing frequency of the parsing or disabling it
> entirely. I'm not sure if there are cheaper ways to get all of the same
> metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
