Repository: impala Updated Branches: refs/heads/master 64e671987 -> bbb399ddf
IMPALA-6425: reduce MemPool max chunk size IMPALA-2742 capped chunk size doubling to 1MB with the idea that 1MB would be a good value to amortise malloc() overhead. As it turns out, 1MB allocations are significantly more expensive than smaller allocations because they don't have a central freelist in TCMalloc, so the previous choice was misinformed. Note that this doesn't prevent larger allocations from being served from the MemPool, it just caps the automatic doubling behaviour. Perf: Ran single-node TPC-H and targeted perf with scale factor 60. Overall improvement in mean/geomean of 0-2% and no regressions outside of the usual variance. Change-Id: I58cc3031ce592892cb9efe9ab41f07d86468b08c Reviewed-on: http://gerrit.cloudera.org:8080/10789 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/impala/repo Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/bbb399dd Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/bbb399dd Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/bbb399dd Branch: refs/heads/master Commit: bbb399ddf405709ec5a82edb50e39c359f64098a Parents: 64e6719 Author: Tim Armstrong <[email protected]> Authored: Thu Jun 21 15:38:18 2018 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Tue Jun 26 18:26:08 2018 +0000 ---------------------------------------------------------------------- be/src/runtime/mem-pool.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/impala/blob/bbb399dd/be/src/runtime/mem-pool.h ---------------------------------------------------------------------- diff --git a/be/src/runtime/mem-pool.h b/be/src/runtime/mem-pool.h index 2d12da8..03f05e2 100644 --- a/be/src/runtime/mem-pool.h +++ b/be/src/runtime/mem-pool.h @@ -178,8 +178,9 @@ class MemPool { static const int INITIAL_CHUNK_SIZE = 4 * 1024; /// The maximum size of chunk that should be allocated. Allocations larger than this - /// size will get their own individual chunk. - static const int MAX_CHUNK_SIZE = 1024 * 1024; + /// size will get their own individual chunk. Chosen to be small enough that it gets + /// a freelist in TCMalloc's central cache. + static const int MAX_CHUNK_SIZE = 512 * 1024; struct ChunkInfo { uint8_t* data; // Owned by the ChunkInfo.
