Steve Carlin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24261 )
Change subject: IMPALA-14849: Upgrade Calcite planner to 1.42 ...................................................................... IMPALA-14849: Upgrade Calcite planner to 1.42 Upgraded to Calcite 1.42. Some problems addressed: - getMaxNumPrecision and getMaxNumScale were made final methods in Calcite causing a compilation error. - A Jira fix changed the interval expression to use CHECKED_TIMES rather than TIMES for the operator. - The avatica dependency is not needed. - 1.42 added new support for some correlated queries which now compile in Calcite and will no longer fallback to the original planner. - The support of some new correlated queries uncovered some incorrect results which can be found in subquery.test and grouping-sets.test. IMPALA-15122 and IMPALA-15124 have been filed to deal with these incorrect results in the original planner. - Some tests for spilling had to be commented out. The queries in the file used to fallback to the original planner. They succeed now. But the plans generated are different. The original planner produces anti-join queries and the Calcite planner does not. This results in different memory usage. The spilling tests are specifically designed to handle this. IMPALA-15123has been filed to ensure these tests will eventually work for the Calcite planner. - This is a small performance regression in analytic-rank-pushdown-calcite.test. IMPALA-15125 has been filed. A Filter/Values RelNode is not being reduced. This will have a minimal performance impact, so it should be ok to do the upgrade with this issue. Change-Id: I4bfb3271282f3dfdd250fa61bce1492e5dd499f8 Reviewed-on: http://gerrit.cloudera.org:8080/24261 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Aman Sinha <[email protected]> --- M java/calcite-planner/pom.xml M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/IntervalExpr.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaRexExecutor.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java M testdata/workloads/functional-planner/queries/PlannerTest/calcite/analytic-rank-pushdown-calcite.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q91.test M testdata/workloads/functional-query/queries/QueryTest/calcite_subquery.test M testdata/workloads/functional-query/queries/QueryTest/grouping-sets.test M testdata/workloads/functional-query/queries/QueryTest/spilling-naaj.test M testdata/workloads/functional-query/queries/QueryTest/subquery.test M tests/common/skip.py M tests/query_test/test_spilling.py 13 files changed, 81 insertions(+), 86 deletions(-) Approvals: Impala Public Jenkins: Verified Aman Sinha: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/24261 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4bfb3271282f3dfdd250fa61bce1492e5dd499f8 Gerrit-Change-Number: 24261 Gerrit-PatchSet: 10 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Anonymous Coward (816) Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Stamatis Zampetakis <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]>
