Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23724 )
Change subject: IMPALA-14429: Calcite planner: change mechanism for parsing tinyint, smallint ...................................................................... IMPALA-14429: Calcite planner: change mechanism for parsing tinyint, smallint Calcite and Impala have an inconsistency with datatypes for small integers. Calcite will treat all integers that fit within an integer as an integer. Impala treats integers more granularly, that is, '1' will be a tinyint and '1872' will be a smallint. CALCITE-7120 has been filed within Calcite so that eventually Calcite will be compatible with Impala. For Calcite v1.37, the coercenodes module handles the inconsistency. This is done after the RelNodes are created. In Calcite 1.41, some code was added at the AST level that broke Impala since it is too late to manipulate the fields after RelNode creation. A new mechanism to handle the inconsistency has been added with this commit. The internal Validator for Calcite calls 'deriveType' which derives the proper type and this method has been overriden. At RexBuilder time (where the RexNodes for the RelNodes are created), the makeLiteral method is overridden so that the proper type is created. ImpalaRexBuilder derives from RexBuilder and provides this method. A "postAnalysis_" variable within the ImpalaRexBuilder ensures that the makeLiteral logic only gets applied at analysis time. At optimization time, the Impala code knows the explicit type of the literal that is being created, so the RelDataType of the parameter passed in is used. Testing is already in place for this as we have tests for dealing with different size constants. This commit is proactive to the upgrade to 1.41 Change-Id: I67b6f7711093a4b8488beee0893aea3c72239eb0 Reviewed-on: http://gerrit.cloudera.org:8080/23724 Reviewed-by: Michael Smith <[email protected]> Tested-by: Michael Smith <[email protected]> --- A java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaRexBuilder.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlValidatorImpl.java M testdata/workloads/functional-planner/queries/PlannerTest/calcite/analytic-rank-pushdown-calcite.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q33.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q40.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q43-verbose.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q43.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q56.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q60.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q61.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q89.test M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q91.test 13 files changed, 260 insertions(+), 104 deletions(-) Approvals: Michael Smith: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/23724 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I67b6f7711093a4b8488beee0893aea3c72239eb0 Gerrit-Change-Number: 23724 Gerrit-PatchSet: 12 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Pranav Lodha <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]>
