[
https://issues.apache.org/jira/browse/IMPALA-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18060495#comment-18060495
]
ASF subversion and git services commented on IMPALA-14429:
----------------------------------------------------------
Commit 0f53e31363dddad918c5f5cf103697b4624d9ede in impala's branch
refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0f53e3136 ]
IMPALA-14429: Calcite planner: change mechanism for parsing tinyint, smallint
Calcite and Impala have an inconsistency with datatypes for small integers.
Calcite will treat all integers that fit within an integer as an integer.
Impala treats integers more granularly, that is, '1' will be a tinyint and
'1872' will be a smallint. CALCITE-7120 has been filed within Calcite so
that eventually Calcite will be compatible with Impala.
For Calcite v1.37, the coercenodes module handles the inconsistency. This is
done after the RelNodes are created.
In Calcite 1.41, some code was added at the AST level that broke Impala
since it is too late to manipulate the fields after RelNode creation.
A new mechanism to handle the inconsistency has been added with this commit.
The internal Validator for Calcite calls 'deriveType' which derives the proper
type and this method has been overriden.
At RexBuilder time (where the RexNodes for the RelNodes are created), the
makeLiteral method is overridden so that the proper type is created.
ImpalaRexBuilder derives from RexBuilder and provides this method.
A "postAnalysis_" variable within the ImpalaRexBuilder ensures that the
makeLiteral logic only gets applied at analysis time. At optimization time,
the Impala code knows the explicit type of the literal that is being
created, so the RelDataType of the parameter passed in is used.
Testing is already in place for this as we have tests for dealing with
different size constants. This commit is proactive to the upgrade to
1.41
Change-Id: I67b6f7711093a4b8488beee0893aea3c72239eb0
Reviewed-on: http://gerrit.cloudera.org:8080/23724
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Michael Smith <[email protected]>
> Calcite Planner: 1.40 breaks coercion of smaller numeric datatypes
> ------------------------------------------------------------------
>
> Key: IMPALA-14429
> URL: https://issues.apache.org/jira/browse/IMPALA-14429
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Steve Carlin
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> In Calcite v1.37, we use coercenodes to change smaller datatypes to match
> Impala types. For instance, the numeric '1' needs to be a tinyint in Impala,
> whereas Calcite makes it an int (CALCITE-7120 is filed for this problem).
> On upgrade from Calcite 1.37 to Calcite 1.40, the mechanism in coercenodes
> fails because of work Calcite did in the AST portion of their code. So we
> need to come up with another way to do this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
