[
https://issues.apache.org/jira/browse/IMPALA-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535111#comment-16535111
]
Tim Armstrong commented on IMPALA-7239:
---------------------------------------
It looks like smaps does acquire mmap_sem at various points during the page
table walk. This does affect performance of virtual memory operations. E.g. if
I run the attached program mmap.c and cat /proc/$(pgrep mmap)/smaps in another
terminal, mmap noticably slows down, even on a system without the smaps bug
mentioned above. I think if the bug is present then the smap-producing code is
probably holding mmap_sem while walking all of the threads.
{code}
# In one terminal.
gcc -std=gnu99 mmap.c -o mmap && ./mmap
# In another terminal.
while cat /proc/$(pgrep mmap)/smaps > /dev/null; do echo yes; done
{code}
> Mitigate ParseSmaps() overhead
> ------------------------------
>
> Key: IMPALA-7239
> URL: https://issues.apache.org/jira/browse/IMPALA-7239
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.10.0, Impala 2.11.0, Impala 3.0, Impala 2.12.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: perf, resource-management
> Attachments: mmap.c
>
>
> I've heard anecdotes of high system time spent in functions related this the
> smap parsing. It appears that this can be expensive on systems once the
> impalad virtual memory gets fragmented and there are 10s of thousands of maps.
> We can try to mitigate by reducing frequency of the parsing or disabling it
> entirely. I'm not sure if there are cheaper ways to get all of the same
> metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
