Xuefu Zhang commented on HIVE-13423:

[~ctang.ma]/[~aihuaxu], as to sacrificing scale for bigger integer part, it 
doesn't seem to be a viable option. The precision/scale of the result type of 
sum udf is determined statically as result metadata. That is, the result type 
of sum(decimal(p, s)) is decimal(p+10, s), which is decided before seeing any 
actual data. Thus, at run time when the data is actually processed, we cannot 
return the result of decimal( p+10+d, s-d) because the data (result) doesn't 
conform to the metadata (type decimal(p+10, s).

Please feel free to check standards or what other dbs are dong. As far as I 
know, there is no standard that permits this.

> Handle the overflow case for decimal datatype for sum()
> -------------------------------------------------------
>                 Key: HIVE-13423
>                 URL: https://issues.apache.org/jira/browse/HIVE-13423
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>         Attachments: HIVE-13423.1.patch
> When a column col1 defined as decimal and if the sum of the column overflows, 
> we will try to increase the decimal precision by 10. But if it's reaching 38 
> (the max precision), the overflow still could happen. Right now, if such case 
> happens, the following exception will throw since hive is writing incorrect 
> data.
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
>         at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:314)
>  ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:219)
>  ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>         at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
>  ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> {noformat}

This message was sent by Atlassian JIRA

Reply via email to