[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Riza Suminto has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. IMPALA-12032: Fix min parallelism bug in PlanFragment When COMPUTE_PROCESSING_COST=true, PROCESSING_COST_MIN_THREADS control the minimum parallelism of query fragment per executor. However, PlanFragment.adjustToMaxParallelism() did not take account of it during adjustment. This patch fix that bug by capping per-node fragment parallelism to PROCESSING_COST_MIN_THREADS if cost based parallelism comes up with number that is lower. Testing: - Set PROCESSING_COST_MIN_THREADS in PlannerTest.testProcessingCost. - Pass test_executor_groups.py. - Add test cases in query-options-test.cc. Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Reviewed-on: http://gerrit.cloudera.org:8080/19663 Reviewed-by: Kurt Deschler Reviewed-by: Wenzhe Zhou Tested-by: Impala Public Jenkins --- M be/src/service/query-options-test.cc M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-processing-cost.test 5 files changed, 451 insertions(+), 430 deletions(-) Approvals: Kurt Deschler: Looks good to me, but someone else must approve Wenzhe Zhou: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: Thank you for the review! I will merge this code patch first and wait for the next one. I think David want to confirm something about the scan costing. -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 01:48:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 01:43:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: Code-Review+2 carry +1 from Kurt -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 20:24:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9186/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 20:25:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 20:04:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 16:04:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/19663/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19663/3//COMMIT_MSG@9 PS3, Line 9: When COMPUTE_PROCESSING_COST=true, PROCESSING_COST_MIN_THREADS control > I don't follow what this is saying. Please make it more chear what you mean Rephrased. -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 15:35:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Hello Andrew Sherman, Kurt Deschler, Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19663 to look at the new patch set (#4). Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. IMPALA-12032: Fix min parallelism bug in PlanFragment When COMPUTE_PROCESSING_COST=true, PROCESSING_COST_MIN_THREADS control the minimum parallelism of query fragment per executor. However, PlanFragment.adjustToMaxParallelism() did not take account of it during adjustment. This patch fix that bug by capping per-node fragment parallelism to PROCESSING_COST_MIN_THREADS if cost based parallelism comes up with number that is lower. Testing: - Set PROCESSING_COST_MIN_THREADS in PlannerTest.testProcessingCost. - Pass test_executor_groups.py. - Add test cases in query-options-test.cc. Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 --- M be/src/service/query-options-test.cc M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-processing-cost.test 5 files changed, 451 insertions(+), 430 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/19663/4 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/19663/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19663/3//COMMIT_MSG@9 PS3, Line 9: PROCESSING_COST_MIN_THREADS suppose to be a lower bound of per-node I don't follow what this is saying. Please make it more chear what you mean by across executor groups and why the lower bound is appropriate. -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 14:05:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 30 Mar 2023 00:07:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12713/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 29 Mar 2023 23:11:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Hello Andrew Sherman, Kurt Deschler, Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19663 to look at the new patch set (#3). Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. IMPALA-12032: Fix min parallelism bug in PlanFragment PROCESSING_COST_MIN_THREADS suppose to be a lower bound of per-node parallelism when CPU costing algorithm adjust fragment parallelism across executor group set. But PlanFragment.adjustToMaxParallelism() did not take account of it during adjustment. This patch fix that bug by capping per-node fragment parallelism to PROCESSING_COST_MIN_THREADS if cost based parallelism comes up with number that is lower. Testing: - Set PROCESSING_COST_MIN_THREADS in PlannerTest.testProcessingCost. - Pass test_executor_groups.py. - Add test cases in query-options-test.cc. Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 --- M be/src/service/query-options-test.cc M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-processing-cost.test 5 files changed, 451 insertions(+), 430 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/19663/3 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12711/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 29 Mar 2023 21:29:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/19663 )
Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment
..
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19663/2/fe/src/main/java/org/apache/impala/planner/CostingSegment.java
File fe/src/main/java/org/apache/impala/planner/CostingSegment.java:
http://gerrit.cloudera.org:8080/#/c/19663/2/fe/src/main/java/org/apache/impala/planner/CostingSegment.java@203
PS2, Line 203: + "maxParallelism={} newParallelism={}
consumerCost={} consumerInstCount={} "
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/19663
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9
Gerrit-Change-Number: 19663
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Wed, 29 Mar 2023 21:08:46 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Hello Andrew Sherman, Kurt Deschler, Abhishek Rawat, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19663 to look at the new patch set (#2). Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. IMPALA-12032: Fix min parallelism bug in PlanFragment PROCESSING_COST_MIN_THREADS suppose to be a lower bound of per-node parallelism when CPU costing algorithm adjust fragment parallelism across executor group set. But PlanFragment.adjustToMaxParallelism() did not take account of it during adjustment. This patch fix that bug by capping per-node fragment parallelism to PROCESSING_COST_MIN_THREADS if cost based parallelism comes up with number that is lower. Testing: - Set PROCESSING_COST_MIN_THREADS in PlannerTest.testProcessingCost. - Pass test_executor_groups.py. - Add test cases in query-options-test.cc. Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 --- M be/src/service/query-options-test.cc M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-processing-cost.test 5 files changed, 451 insertions(+), 430 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/19663/2 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19663 ) Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12710/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 29 Mar 2023 20:14:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/19663 )
Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19663/1/fe/src/main/java/org/apache/impala/planner/CostingSegment.java
File fe/src/main/java/org/apache/impala/planner/CostingSegment.java:
http://gerrit.cloudera.org:8080/#/c/19663/1/fe/src/main/java/org/apache/impala/planner/CostingSegment.java@203
PS1, Line 203: + "maxParallelism={} newParallelism={}
consumerCost={} consumerInstCount={} "
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/19663
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9
Gerrit-Change-Number: 19663
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Wed, 29 Mar 2023 19:55:01 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12032: Fix min parallelism bug in PlanFragment
Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19663 Change subject: IMPALA-12032: Fix min parallelism bug in PlanFragment .. IMPALA-12032: Fix min parallelism bug in PlanFragment PROCESSING_COST_MIN_THREADS suppose to be a lower bound of per-node parallelism when CPU costing algorithm adjust fragment parallelism across executor group set. But PlanFragment.adjustToMaxParallelism() did not take account of it during adjustment. This patch fix that bug by capping per-node fragment parallelism to PROCESSING_COST_MIN_THREADS if cost based parallelism comes up with number that is lower. Testing: - Set PROCESSING_COST_MIN_THREADS in PlannerTest.testProcessingCost. - Pass test_executor_groups.py Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 --- M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-processing-cost.test 4 files changed, 450 insertions(+), 430 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/19663/1 -- To view, visit http://gerrit.cloudera.org:8080/19663 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9 Gerrit-Change-Number: 19663 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto
