GitHub user caneGuy opened a pull request:
https://github.com/apache/spark/pull/20508
[SPARK-23335][SQL] Should not convert to double when there is an Integraâ¦
â¦l value in BinaryArithmetic which will loss precison
## What changes were proposed in this pull request?
For below expression:
`select conv('ffffffffffffffff',16,10) % 2;`
it will return 0.
```
0: jdbc:hive2://xxx:16> select conv('ffffffffffffffff',16,10) % 2;
+----------------------------------------------------------------------------------------------+--+
| (CAST(conv(ffffffffffffffff, 16, 10) AS DOUBLE) % CAST(CAST(2 AS
DECIMAL(20,0)) AS DOUBLE)) |
+----------------------------------------------------------------------------------------------+--+
| 0.0 |
+----------------------------------------------------------------------------------------------+--+
```
It caused by:
```
case a @ BinaryArithmetic(left @ StringType(), right) =>
a.makeCopy(Array(Cast(left, DoubleType), right))
case a @ BinaryArithmetic(left, right @ StringType()) =>
a.makeCopy(Array(left, Cast(right, DoubleType)))
```
This patch fix this by add rule check when has an intergral type in
BinaryArithmetic operator,we should not convert value to double.
Result as below:
```
0: jdbc:hive2://xxx:16> select conv('ffffffffffffffff',16,10) % 2;
+-----------------------------------------------------------------------------------------------------------------------------------+--+
| (CAST(CAST(conv(ffffffffffffffff, 16, 10) AS DECIMAL(38,0)) AS
DECIMAL(38,0)) % CAST(CAST(2 AS DECIMAL(38,0)) AS DECIMAL(38,0))) |
+-----------------------------------------------------------------------------------------------------------------------------------+--+
| 1
|
+-----------------------------------------------------------------------------------------------------------------------------------+--+
```
## How was this patch tested?
Exist tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/caneGuy/spark zhoukang/fix-castasdouble
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20508.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20508
----
commit 1a2c62f6e2725cbbdc44c464c7fc0b9358e064b2
Author: zhoukang <zhoukang199191@...>
Date: 2018-02-05T10:52:40Z
[SPARK-MI][SQL] Should not convert to double when there is an Integral
value in BinaryArithmetic which will loss precison
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
