[
https://issues.apache.org/jira/browse/IMPALA-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776601#comment-16776601
]
ASF subversion and git services commented on IMPALA-2742:
---------------------------------------------------------
Commit cb4bbbeebecda071f414cd180740ed1d3c594fb0 in impala's branch
refs/heads/2.x from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=cb4bbbe ]
IMPALA-6425: reduce MemPool max chunk size
IMPALA-2742 capped chunk size doubling to 1MB with the idea that 1MB
would be a good value to amortise malloc() overhead. As it turns out,
1MB allocations are significantly more expensive than smaller
allocations because they don't have a central freelist in TCMalloc,
so the previous choice was misinformed.
Note that this doesn't prevent larger allocations from being served from
the MemPool, it just caps the automatic doubling behaviour.
Perf:
Ran single-node TPC-H and targeted perf with scale factor 60. Overall
improvement in mean/geomean of 0-2% and no regressions outside of the
usual variance.
Change-Id: I58cc3031ce592892cb9efe9ab41f07d86468b08c
Reviewed-on: http://gerrit.cloudera.org:8080/10789
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> MemPool allocation growth behavior
> ----------------------------------
>
> Key: IMPALA-2742
> URL: https://issues.apache.org/jira/browse/IMPALA-2742
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.3.0
> Reporter: Martin Grund
> Assignee: Tim Armstrong
> Priority: Blocker
> Labels: resource-management
> Fix For: Impala 2.5.0, Impala 2.3.4
>
>
> Currently, the MemPool would always double the size of the last allocation.
> This can lead to bad behavior if the MemPool transferred the ownership of all
> it's data except the last chunk. In the next allocation, the next allocated
> chunk would double the size of this large chunk which can be undesirable.
> If a MemPool is used without a FreePool there are some alternatives to
> consider:
> - Cap the chunk size at a reasonable value
> - Allocate only doubles of the requested size
> - Instead of doubling pick a slightly smaller factor that is a root of 2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
