Aihua Xu commented on HIVE-13423:

[~xuefuz] and [~ctang.ma] Initially I remember we had issues with GroupBy on 
this decimal data type, but I couldn't see such issue any more (seems it has 
been fixed by HIVE-6459).

But we still have a small issue that when the sum overflows, it will produce 
corrupted intermediate file and give ArrayIndexOutOfBoundsException. 

Can you help take a look at the simple fix or do you have a better idea?

> Handle the overflow case for decimal datatype for sum()
> -------------------------------------------------------
>                 Key: HIVE-13423
>                 URL: https://issues.apache.org/jira/browse/HIVE-13423
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>         Attachments: HIVE-13423.1.patch
> When a column col1 defined as decimal and if the sum of the column overflows, 
> we will try to increase the decimal precision by 10. But if it's reaching 38 
> (the max precision), the overflow still could happen. Right now, if such case 
> happens, the following exception will throw since hive is writing incorrect 
> data.
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
>         at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:314)
>  ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:219)
>  ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
>  ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> {noformat}

This message was sent by Atlassian JIRA

Reply via email to