[
https://issues.apache.org/jira/browse/IMPALA-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831068#comment-16831068
]
Ruslan Dautkhanov commented on IMPALA-5229:
-------------------------------------------
We have some performance issues and working RH Support..
{noformat}
I'm surprised that a system with so much RAM and Java isn't already using
HugeTLB.
However the busiest cores might not be the problem and I could be looking in
the wrong place, so we'll await your reply on that to target our investigation
better.
Also, you might want to ask Cloudera if you should be using the classic HugeTLB
sort of pre-allocated HugePages? That will massively reduce your pagetable
sizes and lead to less memory fragmentation too.
{noformat}
How do we enable Huge pages for Impala?
Thank you!
> Try using TCMalloc + Huge Pages for buffers
> -------------------------------------------
>
> Key: IMPALA-5229
> URL: https://issues.apache.org/jira/browse/IMPALA-5229
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.9.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: perf, resource-management
> Fix For: Impala 2.9.0
>
>
> As well as mmap() + huge pages, we could support TCMalloc + huge pages. I
> believe TCMalloc will support this well because:
> * We can allocate huge-page-aligned memory via posix_memalign()
> * TCMalloc, by default, always decommits large allocations upon freeing them.
> So if we undo the HUGEPAGE madvise() before handing the pages back to
> TCMalloc, then TCMalloc will decommit the huge pages backing the allocation,
> and we won't get TCMalloc's page heap into a weird state.
> This could give us the perf benefits of huge pages without some of the
> headaches associated with using mmap() directly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
