Steve Carlin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23724
Change subject: IMPALA-14429: Calcite planner: change mechanism for parsing tinyint, smallint ...................................................................... IMPALA-14429: Calcite planner: change mechanism for parsing tinyint, smallint Calcite and Impala have an inconsistency with datatypes for small integers. Calcite will treat all integers that fit within an integer as an integer. Impala treats integers more granularly, that is, '1' will be a tinyint and '1872' will be a smallint. CALCITE-7120 has been filed within Calcite so that eventually Calcite can deal will be compatible with Impala. For Calcite v1.37, the coercenodes module handles the inconsistency. This is done after the RelNodes are created. In Calcite 1.41, some code was added at the AST level that broke Impala since it is too late to manipulate the fields after RelNode creation. A new mechanism to handle the inconsistency has been added with this commit. The internal Validator for Calcite calls 'deriveType' which derives the proper type and this method has been overriden. At RexBuilder time (where the RexNodes for the RelNodes are created), the makeLiteral method is overridden so that the proper type is created. ImpalaRexBuilder derives from RexBuilder and provides this method. One extra note about the overridden makeLiteral method. While we need to intercept all makeLiteral calls at validation time, we do not want to intercept the call at optimization time. At validation time, we ignore the provided datatype, but at optimization time, we want to use the provided datatype. A specific example is in constant folding, where the expression '1 + 2' will be simplified to a 3:SMALLINT, not a 3:TINYINT based on Impala upcasting rules. The default RexBuilder is therefore needed and we use the ReplaceRelOptClusterShuttle to ensure that we are using a cluster that contains the default RexBuilder. Testing is already in place for this as we have tests for dealing with different size constants. This commit is proactive to the upgrade to 1.41 Change-Id: I67b6f7711093a4b8488beee0893aea3c72239eb0 --- A java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaRexBuilder.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ReplaceRelOptClusterShuttle.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlValidatorImpl.java 5 files changed, 305 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/23724/2 -- To view, visit http://gerrit.cloudera.org:8080/23724 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I67b6f7711093a4b8488beee0893aea3c72239eb0 Gerrit-Change-Number: 23724 Gerrit-PatchSet: 2 Gerrit-Owner: Steve Carlin <[email protected]>
