Yida Wu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23542
Change subject: IMPALA-14493: Cap memory usage of global admission service ...................................................................... IMPALA-14493: Cap memory usage of global admission service The global admission service can OOM on high workload because its process memory tracker is inaccurate and doesn't account for all memory allocations. Ensuring memory tracker accurately accounts for every allocation could be difficult, this patch uses a simpler solution to introduce a hard memory cap using tcmalloc statistics, which accurately reflect the true process memory usage. If a new query is submitted while tcmalloc memory usage is over the process limit, the query will be rejected immediately to protect from OOM. Adds a new flag enable_admission_service_mem_safeguard allowing this feature to be enabled or disabled. By default, this feature is turned on Tests: Added test test_admission_service_low_mem_limit. Passed exhaustive tests. Change-Id: I2ee2c942a73fcd69358851fc2fdc0fc4fe531c73 --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/admissiond-env.cc M be/src/scheduling/admissiond-env.h M tests/custom_cluster/test_admission_controller.py 4 files changed, 57 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/23542/1 -- To view, visit http://gerrit.cloudera.org:8080/23542 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2ee2c942a73fcd69358851fc2fdc0fc4fe531c73 Gerrit-Change-Number: 23542 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu <[email protected]>
