Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/9058 )
Change subject: IMPALA-5528: Bump total thread cache size when KRPC is enabled
......................................................................
IMPALA-5528: Bump total thread cache size when KRPC is enabled
KRPC in general tends to put more pressure on the thread
caches due to allocations of more small objects (i.e. <1MB).
While some of them are being addressed in KUDU-1865, it's shown
that the following TCMalloc workarounds will provide reasonable
performance with KRPC:
- TCMALLOC_TRANSFER_NUM_OBJ:
- maximum number of object per classe type to transfer between
thread and central caches.
- the default value of 512 in 2.5.2 seems to cause the spin lock
in the central cache to be held for too long with KRPC. 2.5.90
and latter reverts this value to 32 by default.
- TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES
- total amount of memory allocated to all thread caches in bytes
- the default value is 32MB. We need to bump it to 1GB which is the
internal cap in TCMalloc.
This change bumps the thread cache sizes to 1GB when KRPC is enabled and
FLAGS_TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES has the default value of 0.
GPerfTools/TCMalloc needs to be upgraded to 2.5.90 or above to pick up
the change of the default value of TCMALLOC_TRANSFER_NUM_OBJ. Previous
attempt to upgrade GPerfTools to 2.6.3 failed on certain platforms
(IMPALA-6414).
Also fixes a couple of BE tests to initialize the test environment properly.
Change-Id: I8407528942051fb19a0222491347c9090d4b4b8d
Reviewed-on: http://gerrit.cloudera.org:8080/9058
Reviewed-by: Michael Ho <[email protected]>
Reviewed-by: Sailesh Mukil <[email protected]>
Tested-by: Impala Public Jenkins
---
M be/src/runtime/bufferpool/buffer-allocator-test.cc
M be/src/runtime/bufferpool/free-list-test.cc
M be/src/runtime/bufferpool/suballocator-test.cc
M be/src/runtime/exec-env.cc
4 files changed, 43 insertions(+), 12 deletions(-)
Approvals:
Michael Ho: Looks good to me, but someone else must approve
Sailesh Mukil: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/9058
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8407528942051fb19a0222491347c9090d4b4b8d
Gerrit-Change-Number: 9058
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho <[email protected]>
Gerrit-Reviewer: Sailesh Mukil <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
