[
https://issues.apache.org/jira/browse/KYLIN-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17795292#comment-17795292
]
zhong.zhu commented on KYLIN-5747:
----------------------------------
h1.Root Cause
When the plus sign is used in a row, and the arguments are all constants,
calcite will do constant folding, and when calculating the value, it will
convert the expression into java code, and each plus sign will determine
whether it needs to be converted into a custom plus function or whether it
should be used directly with the java additions.
into a custom plus function has three conditions: 1. plus left is not a basic
type; 2. plus right is a bigDecimal type; 3. (more complex), any one of them
can be satisfied
For '1' + 3 + '3', '1' + 3 is of type double, and '3' is of type string, so the
second plus sign does not satisfy all three conditions, which results in the
expression translated into java code as plus('1' + 3) + '3'.
h1.Dev Design
1. In calcite to generate java code, to determine the + sign needs to be
converted to plus() it or directly use the addition in java, add a condition,
when the parameters on both sides of the plus sign for the string type or
numerical type, directly use plus()
i.e. fix the '1' + 3 + '3' in the previous sql, the java code is plus('1' + 3)
+ '3'.
After the fix it is plus(plus('1' + 3), '3')
2. Change the return value of the custom plus function in calcite from Double
to bigDecimal.
The reason:
- When calcite does constant folding to generate java code, it does isNullable
derivation, and when the nullable of a call is false, it does an automatic
unboxing.
- The expression 'a' + 3 is considered by calcite to be non-nullable when it
does the nullable derivation because the two arguments are constants and
neither of them is null, so the whole thing is considered non-null!
- In spark, 'a' + 3 results in null, so in our implementation of the plus
method, 'a' + 3 also results in null
- In summary, when plus(string, number) returns Double, the java code for 'a' +
3 + '3' is actually plus(plus('a' + 3).doubleValue(), '3'), which then throws
an NPE when the calculation is performed.
The logic of isNullable involves a wider scope, and it is risky to modify it
directly, so here we change the return value of plus to BigDecimal, to avoid
unboxing.
> Calcite constant folding, adding strings to numbers, results not as expected
> when multiple plus signs are used together
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-5747
> URL: https://issues.apache.org/jira/browse/KYLIN-5747
> Project: Kylin
> Issue Type: Bug
> Affects Versions: 5.0-beta
> Reporter: zhong.zhu
> Assignee: zhong.zhu
> Priority: Critical
> Fix For: 5.0.0
>
>
> Phenomenon:
> When more than one plus sign is used in a row and the parameters on both
> sides of the plus sign are constants, the result is not as expected
> '1' + 3 + 3 → 7 (correct)
> '1' + 3 + '3' → 4.03 (wrong result)
> '1' + '3' + 'a' → error
> When multiple plus signs are used in a row, and the arguments on both sides
> of the plus sign are constants, and the first plus sign results in null, the
> use of plus signs in a row is not supported.
> e.g. 'q' + 1 + 1 -> error
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
