Jason Fehr has posted comments on this change. ( http://gerrit.cloudera.org:8080/22249 )
Change subject: IMPALA-13201: System Table Queries Execute When Admission Queues are Full ...................................................................... Patch Set 20: (7 comments) http://gerrit.cloudera.org:8080/#/c/22249/19/be/src/scheduling/cluster-membership-mgr-test.cc File be/src/scheduling/cluster-membership-mgr-test.cc: http://gerrit.cloudera.org:8080/#/c/22249/19/be/src/scheduling/cluster-membership-mgr-test.cc@290 PS19, Line 290: << "Actual hostnames:" << actual_hostnames.str(); > Space after "Actual hostnames:" isn't needed because actual_hostnames will Done http://gerrit.cloudera.org:8080/#/c/22249/19/be/src/scheduling/cluster-membership-mgr-test.cc@302 PS19, Line 302: << expected << "' in actual coordinators:" << actual_hostnames.str(); > Same here, could be "' in actual coordinators:". Done http://gerrit.cloudera.org:8080/#/c/22249/15/be/src/util/webserver.cc File be/src/util/webserver.cc: http://gerrit.cloudera.org:8080/#/c/22249/15/be/src/util/webserver.cc@717 PS15, Line 717: LOG(INFO) << "Invalid JWT token provided"; > Eh, guess it's a good idea to clean up. Done http://gerrit.cloudera.org:8080/#/c/22249/19/common/thrift/ImpalaInternalService.thrift File common/thrift/ImpalaInternalService.thrift: http://gerrit.cloudera.org:8080/#/c/22249/19/common/thrift/ImpalaInternalService.thrift@236 PS19, Line 236: // Specifies the state of the onlyCoordinators configuration element for a request pool. > Should have a comment. Good catch. Done http://gerrit.cloudera.org:8080/#/c/22249/19/docs/topics/impala_admission_config.xml File docs/topics/impala_admission_config.xml: http://gerrit.cloudera.org:8080/#/c/22249/19/docs/topics/impala_admission_config.xml@187 PS19, Line 187: To use an <codeph><onlyCoordinators></codeph> request pool, set the > typo: "simple" should be "simply", but I'd rather remove it. Done http://gerrit.cloudera.org:8080/#/c/22249/19/docs/topics/impala_admission_config.xml@196 PS19, Line 196: Caution: care must be taken when naming the <codeph><onlyCoordinators></codeph> > Why does this happen? It's a combination of planning and scheduling. Impala allows for specifying the "expected_executor_group_sets" flag which defines groups of isolated executor groups that the planner and scheduler then select from when a query is to be executed. For example, say there are executor group set "small" and "large". Executor groups "small-001", "small-002", and "small-003" will all be considered by the scheduler when the planner determines that executor group set "small" can run they query. Similarly, if the planner determines that "small" is too small, a a "large" executor group needs to run the query, then the scheduler will consider executor groups "large-001", "large-002", and "large-003". If an only coordinator's pool is named "small-onlycoords", it will be considered when the planner selects "small" as the executor group set. I looked into erroring out if an only coordinators request pool overlaps with an executor group set, but it's unfortunately not easy to do that because the executor group sets are maintained in the backend while the request pools are maintained in the frontend. http://gerrit.cloudera.org:8080/#/c/22249/16/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/22249/16/tests/custom_cluster/test_admission_controller.py@1949 PS16, Line 1949: self._start_impala_cluster( : options=[ : "--impalad_args=s{}".format(impalad_admission_ctrl_config_args( : fs_allocation_file="fair-scheduler-onlycoords.xml", : llama_site_file="llama-site-onlycoords.xml"))], : add_impalads=True, : cluster_size=6, : num_coordinators=1, : use_exclusive_coordinators=True, : wait_for_backends=False, : workload_mgmt=True) > What you're running up against here is that add_impalads was hacked in to s https://issues.apache.org/jira/browse/IMPALA-13755 -- To view, visit http://gerrit.cloudera.org:8080/22249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5e0e64db92bdbf80f8b5bd85d001ffe4c8c9ffda Gerrit-Change-Number: 22249 Gerrit-PatchSet: 20 Gerrit-Owner: Jason Fehr <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Wed, 12 Feb 2025 00:20:25 +0000 Gerrit-HasComments: Yes
