Bikramjeet Vig has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11157
Change subject: IMPALA-7349: Add Admission control support for automatically setting mem_limit With this patch the mem_limit of a query is automatically set based on the mem_limit set in the query options and the mem_estimate calculated by the planner. This mem_limit wil ...................................................................... IMPALA-7349: Add Admission control support for automatically setting mem_limit With this patch the mem_limit of a query is automatically set based on the mem_limit set in the query options and the mem_estimate calculated by the planner. This mem_limit will also be used for admission accounting and consequently for admission control. Moreover three new pool configuration options have been added: min-query-mem-limit-bytes, max-query-mem-limit-bytes => help clamp the mem_limit. If these are set to zero, impala will fall back to traditional behavior (mem_limit not automatically set) strict-min-max-query-mem-limit => if false, the mem_limit defined in the query options is used directly and the max/min limits are not enforced on it. Testing: Added e2e test cases. Change-Id: Ifec00141651982f5975803c2165b7d7a10ebeaa6 --- M be/src/benchmarks/process-wide-locks-benchmark.cc M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator.cc M be/src/runtime/mem-tracker.cc M be/src/runtime/mem-tracker.h M be/src/runtime/query-exec-mgr.cc M be/src/runtime/query-exec-mgr.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-state.cc M be/src/runtime/test-env.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/service/client-request-state.cc M common/thrift/ImpalaInternalService.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/util/RequestPoolService.java A fe/src/test/resources/fair-scheduler-mem-limit-test.xml A fe/src/test/resources/llama-site-mem-limit-test.xml A testdata/workloads/functional-query/queries/QueryTest/admission-max-min-mem-limits.test M tests/custom_cluster/test_admission_controller.py 24 files changed, 858 insertions(+), 202 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/11157/6 -- To view, visit http://gerrit.cloudera.org:8080/11157 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifec00141651982f5975803c2165b7d7a10ebeaa6 Gerrit-Change-Number: 11157 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
