Tim Armstrong has uploaded a new patch set (#2). Change subject: IMPALA-5073: Use mmap instead of malloc for buffer pool ......................................................................
IMPALA-5073: Use mmap instead of malloc for buffer pool Allocate with mmap instead of TCMalloc to give more control over memory usage. Also allocate huge pages when possible to reduce TLB pressure. Adds additional memory metrics, since we previously relied on the assumption that all memory was allocated through TCMalloc. memory.total-used and memory.total-reserved track the total across the buffer pool and TCMalloc. When the buffer pool is not present, they just report the TCMalloc values. ASAN still uses malloc() because it doesn't instrument mmap(). Testing: Added some unit tests to test edge cases. Many pre-existing tests also exercise the modified code. Change-Id: Ifbc748f74adcbbdcfa45f3ec7df98284925acbd6 --- M be/src/catalog/catalogd-main.cc M be/src/runtime/bufferpool/buffer-allocator-test.cc M be/src/runtime/bufferpool/buffer-allocator.h M be/src/runtime/bufferpool/buffer-pool.cc M be/src/runtime/bufferpool/buffer-pool.h M be/src/runtime/bufferpool/reservation-tracker.cc M be/src/runtime/bufferpool/reservation-tracker.h M be/src/runtime/bufferpool/system-allocator.cc M be/src/runtime/bufferpool/system-allocator.h M be/src/runtime/exec-env.cc M be/src/statestore/statestored-main.cc M be/src/util/asan.h M be/src/util/memory-metrics.cc M be/src/util/memory-metrics.h M be/src/util/metrics-test.cc M be/src/util/metrics.h M common/thrift/generate_error_codes.py M common/thrift/metrics.json 18 files changed, 355 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/6474/2 -- To view, visit http://gerrit.cloudera.org:8080/6474 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifbc748f74adcbbdcfa45f3ec7df98284925acbd6 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]>
