Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21257 )
Change subject: IMPALA-12980: Translate CpuAsk into admission control slots ...................................................................... Patch Set 15: (2 comments) http://gerrit.cloudera.org:8080/#/c/21257/14//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21257/14//COMMIT_MSG@17 PS14, Line 17: underestimation and query overadmission. For example, assume an executor > nit: missing "and"? Done http://gerrit.cloudera.org:8080/#/c/21257/11/tests/custom_cluster/test_executor_groups.py File tests/custom_cluster/test_executor_groups.py: http://gerrit.cloudera.org:8080/#/c/21257/11/tests/custom_cluster/test_executor_groups.py@1245 PS11, Line 1245: # CoreCount={total=16 trace=F15:3+F01:1+F14:3+F03:1+F13:3+F05:1+F12:3+F07:1}, > One thing I noticed is that runtime filters do not seem to be accounted for. Correct, we have not account for runtime filter waiting time yet. This will require deeper analysis to account for since dependent fragment may wait just for short time or not at all. > Is my understanding correct that this is the case where based on the cost we > wanted to run more instances, but it was clamped to the maximum number of > instances in hosts, Correct, it does clam min and max parallelism per node during traverseEffectiveParallelism(). The next patch behind this will somewhat relax this clamping by calculating an unbounded CpuAsk/CoreCount for executor group selection. It will promote query to bigger executor group if the unbounded CpuAsk/CoreCount indicates a potential benefit to do so. -- To view, visit http://gerrit.cloudera.org:8080/21257 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I338ca96555bfe8d07afce0320b3688a0861663f2 Gerrit-Change-Number: 21257 Gerrit-PatchSet: 15 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Comment-Date: Wed, 17 Apr 2024 15:00:27 +0000 Gerrit-HasComments: Yes
