[ 
https://issues.apache.org/jira/browse/HIVE-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

vinay kant garg updated HIVE-19523:
-----------------------------------
    Description: 
This issue is related to the unnecessary truncation of zero's while serializing 
BigDecimal object to HiveDecimal object.

BigDecimal object has info about scale and precision still serializing ended up 
modifying scale  and precision metadata as well truncation of zero's.

Eg: if our BigDecimal val is : 47.6300, scale = 4 and precision = 6. If I am 
serializing this to HiveDecimal by using API hive expose :

static HiveDecimal create(BigDecimal b)

static HiveDecimal create(BigDecimal b, boolean allowRounding)

our output will be : val = 47.63, scale = 2 and precision = 4.

or if IP: val = 47.00, scale = 2 and precision = 4 then

       OP: val = 47, scale = 0 and precision = 2

*In above example if we see there is no DATA CORRUPTION because 47.6300 is 
equivalent to 47.63 or 47.00 is equivalent to 47 but later on if this data is 
used to identify whether it is integer or decimal then we may run into weird 
issues because 47.00 is decimal but 47 is an integer*.

I am able to reproduce this issue even with standalone program:

*import java.math.BigDecimal;*
*import org.apache.hadoop.hive.common.type.HiveDecimal;*

*public class testHivedecimal {*

*public static void main(String[] argv){*
 *BigDecimal bd = new BigDecimal ("47.6300");*
 *bd.setScale(4);*
 *System.out.println("bigdecimal object created and value is : " + 
bd.toString());*
 *System.out.println("precision : "+ bd.precision());*
 *System.out.println("scale : "+ bd.scale());*
 *HiveDecimal hv = HiveDecimal.create(bd);*
 *String str = hv.toString();*
 *System.out.println("value after serialization of bigdecimal to hivedecimal : 
" + str);*
 *System.out.println("precision after hivedecimal : "+ hv.precision());*
 *System.out.println("scale after hivedecimal : "+ hv.scale());*
 *}*
*}*

you can use above program to reproduce issue:

To Compile:  

1) use jdk8_u72

2) command : javac -cp hive-common-1.2.1000.2.6.0.3-8.jar:. 
testHivedecimal.java 

To Run:

1) use jdk8_u72

2) command : java -cp hive-common-1.2.1000.2.6.0.3-8.jar:. testHivedecimal

 

  was:
This issue is related to the unnecessary truncation of zero's while serializing 
BigDecimal object to HiveDecimal object.

BigDecimal object has info about scale still serializing ended up modifying 
scale metadata as well truncation of zero's.

Eg: if our BigDecimal val is : 47.6300 and scale = 4 and if I am serializing 
this to HiveDecimal by using API hive expose :

static HiveDecimal create(BigDecimal b)

static HiveDecimal create(BigDecimal b, boolean allowRounding)

our output will be : val = 47.63 and scale = 2.

or if IP: val = 47.00 and scale = 2 then

       OP: val = 47 and scale = 0

*In above example if we see there is no DATA CORRUPTION because 47.6300 is 
equivalent to 47.63 or 47.00 is equivalent to 47 but later on if this data is 
used to identify whether it is integer or decimal then we may run into weird 
issues because 47.00 is decimal but 47 is an integer*.


I am able to reproduce this issue even with standalone program:

*import java.math.BigDecimal;*
*import org.apache.hadoop.hive.common.type.HiveDecimal;*

*public class testHivedecimal {*

*public static void main(String[] argv){*
 *BigDecimal bd = new BigDecimal ("47.6300");*
 *bd.setScale(4);*
 *System.out.println("bigdecimal object created and values are : " + 
bd.toString());*
 *System.out.println();*
 *HiveDecimal hv = HiveDecimal.create(bd);*
 *String str = hv.toString();*
 *System.out.println("values after serialization of bigdecimal to hivedecimal : 
" + str);*
 *}*
*}*

you can use above program to reproduce issue:

To Compile:  

1) use jdk8_u72

2) command : javac -cp hive-common-1.2.1000.2.6.0.3-8.jar:. 
testHivedecimal.java 

To Run:

1) use jdk8_u72

2) command : java -cp hive-common-1.2.1000.2.6.0.3-8.jar:. testHivedecimal

 


> Decimal truncation for trailing zeros in Hive 1.2.1
> ---------------------------------------------------
>
>                 Key: HIVE-19523
>                 URL: https://issues.apache.org/jira/browse/HIVE-19523
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 1.2.1, 2.3.1
>            Reporter: vinay kant garg
>            Priority: Critical
>
> This issue is related to the unnecessary truncation of zero's while 
> serializing BigDecimal object to HiveDecimal object.
> BigDecimal object has info about scale and precision still serializing ended 
> up modifying scale  and precision metadata as well truncation of zero's.
> Eg: if our BigDecimal val is : 47.6300, scale = 4 and precision = 6. If I am 
> serializing this to HiveDecimal by using API hive expose :
> static HiveDecimal create(BigDecimal b)
> static HiveDecimal create(BigDecimal b, boolean allowRounding)
> our output will be : val = 47.63, scale = 2 and precision = 4.
> or if IP: val = 47.00, scale = 2 and precision = 4 then
>        OP: val = 47, scale = 0 and precision = 2
> *In above example if we see there is no DATA CORRUPTION because 47.6300 is 
> equivalent to 47.63 or 47.00 is equivalent to 47 but later on if this data is 
> used to identify whether it is integer or decimal then we may run into weird 
> issues because 47.00 is decimal but 47 is an integer*.
> I am able to reproduce this issue even with standalone program:
> *import java.math.BigDecimal;*
> *import org.apache.hadoop.hive.common.type.HiveDecimal;*
> *public class testHivedecimal {*
> *public static void main(String[] argv){*
>  *BigDecimal bd = new BigDecimal ("47.6300");*
>  *bd.setScale(4);*
>  *System.out.println("bigdecimal object created and value is : " + 
> bd.toString());*
>  *System.out.println("precision : "+ bd.precision());*
>  *System.out.println("scale : "+ bd.scale());*
>  *HiveDecimal hv = HiveDecimal.create(bd);*
>  *String str = hv.toString();*
>  *System.out.println("value after serialization of bigdecimal to hivedecimal 
> : " + str);*
>  *System.out.println("precision after hivedecimal : "+ hv.precision());*
>  *System.out.println("scale after hivedecimal : "+ hv.scale());*
>  *}*
> *}*
> you can use above program to reproduce issue:
> To Compile:  
> 1) use jdk8_u72
> 2) command : javac -cp hive-common-1.2.1000.2.6.0.3-8.jar:. 
> testHivedecimal.java 
> To Run:
> 1) use jdk8_u72
> 2) command : java -cp hive-common-1.2.1000.2.6.0.3-8.jar:. testHivedecimal
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to