Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24665 )
Change subject: IMPALA-15243: Fix costing for CTEs ...................................................................... IMPALA-15243: Fix costing for CTEs Addresses missing elements in plan costing when COMPUTE_PROCESSING_COST=true. Notably TPC-DS Q39 contains fragments with CTEConsumerNodes and no ScanNodes, some of which have no child fragments because only the first CTEConsumerNode contains a link to its producer to avoid creating a DAG. This may affect the efficacy of CPC with CTEs. A bunch of other queries also visited some CTEConsumerNodes before the fragment for their CTEProducerNode. We only want to count the contribution of the CTEProducerNode once, so it's safe to skip these. Updates cte-distributed to include parallel planning with costing of a union over the same CTE twice. Change-Id: I21b60af313070029a1fa8bf835107bef88f54912 Reviewed-on: http://gerrit.cloudera.org:8080/24665 Reviewed-by: Michael Smith <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/planner/CostingSegment.java M fe/src/main/java/org/apache/impala/planner/LocalMultiSink.java M fe/src/main/java/org/apache/impala/planner/PlanFragment.java M java/calcite-planner/src/test/java/org/apache/impala/calcite/planner/TpcdsCtePlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q01.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q02.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q03.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q04.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q05.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q06.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q07.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q08.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q09.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q10.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q11.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q12.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q13.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q14a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q14b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q15.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q16.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q17.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q18.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q19.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q20.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q21.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q22.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q23a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q23b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q24a.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q24b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q25.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q26.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q27.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q28.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q29.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q30.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q31.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q32.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q34.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q35.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q36.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q37.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q38.test A testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q39a.test A testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q39b.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q40.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q41.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q42.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q43.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q44.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q45.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q46.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q47.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q48.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q49.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q50.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q51.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q52.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q53.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q54.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q55.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q57.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q58.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q59.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q62.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q63.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q64.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q65.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q66.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q67.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q68.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q69.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q70.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q71.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q72.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q73.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q74.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q75.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q76.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q77.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q78.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q79.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q80.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q81.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q82.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q83.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q84.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q85.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q86.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q87.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q88.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q89.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q90.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q91.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q92.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q93.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q94.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q95.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q96.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q97.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q98.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cte/tpcds-q99.test M testdata/workloads/functional-query/queries/QueryTest/cte-distributed.test 108 files changed, 116,666 insertions(+), 92 deletions(-) Approvals: Michael Smith: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/24665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I21b60af313070029a1fa8bf835107bef88f54912 Gerrit-Change-Number: 24665 Gerrit-PatchSet: 19 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Stamatis Zampetakis <[email protected]>
