Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/21277 )
Change subject: IMPALA-12988: Calculate an unbounded version of CpuAsk ...................................................................... Patch Set 7: (3 comments) http://gerrit.cloudera.org:8080/#/c/21277/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21277/7//COMMIT_MSG@22 PS7, Line 22: During executor group set : selection, Frontend should use the maximum between bounded CpuAsk and : unbounded CpuAsk numbers to avoid assigning a query to a small executor : group set too soon. The last executor group set stays as the "catch-all" : executor group set. This can potentially cause the issue where most queries would get assigned to the larger group-set or `catch-all` group-set. We should probably make this behavior a configurable option to select bounded vs unbounded cpuAsk i.e. to favor smaller group set or the larger group set during assignment. Also, not clear if 'CpuAskUnbounded' could ever be less than 'CpuAsk'? If that's not the case then we could simply use 'CpuAsK' or 'CpuAskUnbounded' when certain config is set to have bounded vs unbounded resources used for determining the group-set assignment. http://gerrit.cloudera.org:8080/#/c/21277/7/common/thrift/Query.thrift File common/thrift/Query.thrift: http://gerrit.cloudera.org:8080/#/c/21277/7/common/thrift/Query.thrift@1037 PS7, Line 1037: Used by admission control to decide which : // executor group to run the query. The decision is already made in the planner, no? Admission Controller will just admit/queue the incoming request to an assigned group-set. Same for the comment related to 'cores_required' option above. http://gerrit.cloudera.org:8080/#/c/21277/7/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/21277/7/fe/src/main/java/org/apache/impala/service/Frontend.java@2304 PS7, Line 2304: coreReqToUse = Math.max(cores_requirement, cores_requirement_unbounded); Is it even possible to have `cores_requirement` to be more than `cores_requirement_unbounded`? -- To view, visit http://gerrit.cloudera.org:8080/21277 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5441e31088f90761062af35862be4ce09d116923 Gerrit-Change-Number: 21277 Gerrit-PatchSet: 7 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: David Rorke <dro...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Comment-Date: Tue, 16 Apr 2024 14:14:43 +0000 Gerrit-HasComments: Yes
