Abhishek Rawat has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/22329 )
Change subject: IMPALA-12494: Clamp MEM_LIMIT_EXECUTORS and MEM_LIMIT_COORDINATORS ...................................................................... IMPALA-12494: Clamp MEM_LIMIT_EXECUTORS and MEM_LIMIT_COORDINATORS 'mem_limit_executors' and 'mem_limit_coordinators' query options are not clamped to any pool configs unlike 'mem_limit'. It's desirable to clamp these query options to pool specific limits so that an admin could put resource pool level clamp limits and end users still have flexibility to override planner's estimates within configured resource pool clamp limits. This patch clamps 'mem_limit_executors' and 'mem_limit_coordinators' to resource pool's ['min-query-mem-limit','max-query-mem-limit'] and ['min-query-mem-limit-coordinators', 'max-query-mem-limit-coordinators'] respectively, if 'clamp-mem-limit-query-option' is set. Testing: - Added be tests and custom cluster tests which validates 'mem_limit_executors' and 'mem_limit_coordinators' clamping behavior. Change-Id: I262910284cbf84b8ba7e0c949bbf053dbfe44a0d --- M be/src/scheduling/admission-controller-test.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M be/src/scheduling/request-pool-service.cc M be/src/scheduling/schedule-state.cc M common/protobuf/admission_control_service.proto M common/thrift/ImpalaInternalService.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/util/RequestPoolService.java M fe/src/test/resources/mem-limit-test-fair-scheduler.xml M fe/src/test/resources/mem-limit-test-llama-site.xml M tests/custom_cluster/test_admission_controller.py M www/admission_controller.tmpl 13 files changed, 484 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/22329/2 -- To view, visit http://gerrit.cloudera.org:8080/22329 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I262910284cbf84b8ba7e0c949bbf053dbfe44a0d Gerrit-Change-Number: 22329 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
