[jira] [Commented] (LIVY-771) Thrift server returns decimal value with trailing zeros removed
[ https://issues.apache.org/jira/browse/LIVY-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112604#comment-17112604 ] Wing Yew Poon commented on LIVY-771: https://github.com/apache/incubator-livy/pull/296/ > Thrift server returns decimal value with trailing zeros removed > --- > > Key: LIVY-771 > URL: https://issues.apache.org/jira/browse/LIVY-771 > Project: Livy > Issue Type: Bug > Components: Thriftserver >Affects Versions: 0.6.0, 0.7.0 >Reporter: Wing Yew Poon >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > I have been doing some Hive interop testing. I have a Hive table with a > column, f_decimal decimal(5, 2). > I run a query to select f_decimal from the table. I run this in Hive (HS2) > through beeline, as well as in the Livy Thrift server through beeline. In the > case of Hive, the decimal values are shown with the correct scale (2 places > to the right of the decimal point) in all cases, while in the case of the > Livy TS, the decimal values that have trailing zeros are shown with trailing > zeros removed. E.g., 9.40 in Hive vs 9.4 in Livy; 713.00 in Hive vs 713 in > Livy. (I did not actually have this in my table, but 120.00 in Hive would be > 1.2E+2 in Livy.) > I would argue that this is incorrect behavior (a bug). If the column type is > decimal(5, 2), then the values should be shown with the correct scale. Also, > the results should, where possible, be the same in Livy as in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (LIVY-771) Thrift server returns decimal value with trailing zeros removed
Wing Yew Poon created LIVY-771: -- Summary: Thrift server returns decimal value with trailing zeros removed Key: LIVY-771 URL: https://issues.apache.org/jira/browse/LIVY-771 Project: Livy Issue Type: Bug Components: Thriftserver Affects Versions: 0.7.0, 0.6.0 Reporter: Wing Yew Poon I have been doing some Hive interop testing. I have a Hive table with a column, f_decimal decimal(5, 2). I run a query to select f_decimal from the table. I run this in Hive (HS2) through beeline, as well as in the Livy Thrift server through beeline. In the case of Hive, the decimal values are shown with the correct scale (2 places to the right of the decimal point) in all cases, while in the case of the Livy TS, the decimal values that have trailing zeros are shown with trailing zeros removed. E.g., 9.40 in Hive vs 9.4 in Livy; 713.00 in Hive vs 713 in Livy. (I did not actually have this in my table, but 120.00 in Hive would be 1.2E+2 in Livy.) I would argue that this is incorrect behavior (a bug). If the column type is decimal(5, 2), then the values should be shown with the correct scale. Also, the results should, where possible, be the same in Livy as in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (LIVY-771) Thrift server returns decimal value with trailing zeros removed
[ https://issues.apache.org/jira/browse/LIVY-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112537#comment-17112537 ] Wing Yew Poon commented on LIVY-771: I'll create a PR with a fix. > Thrift server returns decimal value with trailing zeros removed > --- > > Key: LIVY-771 > URL: https://issues.apache.org/jira/browse/LIVY-771 > Project: Livy > Issue Type: Bug > Components: Thriftserver >Affects Versions: 0.6.0, 0.7.0 >Reporter: Wing Yew Poon >Priority: Major > > I have been doing some Hive interop testing. I have a Hive table with a > column, f_decimal decimal(5, 2). > I run a query to select f_decimal from the table. I run this in Hive (HS2) > through beeline, as well as in the Livy Thrift server through beeline. In the > case of Hive, the decimal values are shown with the correct scale (2 places > to the right of the decimal point) in all cases, while in the case of the > Livy TS, the decimal values that have trailing zeros are shown with trailing > zeros removed. E.g., 9.40 in Hive vs 9.4 in Livy; 713.00 in Hive vs 713 in > Livy. (I did not actually have this in my table, but 120.00 in Hive would be > 1.2E+2 in Livy.) > I would argue that this is incorrect behavior (a bug). If the column type is > decimal(5, 2), then the values should be shown with the correct scale. Also, > the results should, where possible, be the same in Livy as in Hive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (LIVY-754) precision and scale are not encoded in decimal type
[
https://issues.apache.org/jira/browse/LIVY-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Gaido resolved LIVY-754.
--
Fix Version/s: 0.8.0
Assignee: Wing Yew Poon
Resolution: Fixed
Issue resolved by PR: https://github.com/apache/incubator-livy/pull/288.
> precision and scale are not encoded in decimal type
> ---
>
> Key: LIVY-754
> URL: https://issues.apache.org/jira/browse/LIVY-754
> Project: Livy
> Issue Type: Bug
> Components: Thriftserver
>Affects Versions: 0.7.0
>Reporter: Wing Yew Poon
>Assignee: Wing Yew Poon
>Priority: Major
> Fix For: 0.8.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The Livy Thrift server support for decimal type in 0.7 is inadequate.
> Before LIVY-699, decimal is mapped to the catch-all string type. With
> LIVY-699, decimal is mapped to a decimal type that is inadequate in that it
> does not encode the precision and scale. The type in Livy is represented by a
> BasicDataType case class which contains a String field, name; and a DataType
> (an enum) field, dataType. In the case of decimal, the dataType is
> DataType.DECIMAL. The precision and scale of the decimal is not encoded.
> When the DataType is converted to a TTypeDesc for sending a Thrift response
> to a client request for result set metadata, the TTypeDesc contains a
> TPrimitiveTypeEntry(TTypeId.DECIMAL_TYPE) without TTypeQualifiers (which are
> needed to capture the precision and scale). This results in problems for
> clients. E.g., if we connect to the Thrift server in beeline and do a select
> from a table with column of decimal type, we get
> {noformat}
> java.lang.NullPointerException
> at org.apache.hive.jdbc.JdbcColumn.columnPrecision(JdbcColumn.java:310)
> at
> org.apache.hive.jdbc.JdbcColumn.columnDisplaySize(JdbcColumn.java:262)
> at
> org.apache.hive.jdbc.HiveResultSetMetaData.getColumnDisplaySize(HiveResultSetMetaData.java:63)
> at
> org.apache.hive.beeline.IncrementalRows.(IncrementalRows.java:57)
> at
> org.apache.hive.beeline.IncrementalRowsWithNormalization.(IncrementalRowsWithNormalization.java:47)
> at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2322)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1026)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1215)
> at org.apache.hive.beeline.Commands.sql(Commands.java:1144)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1497)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1355)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1134)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082)
> at
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
> {noformat}
> Note: You have to use "--verbose" with beeline to see the stack trace for the
> NPE.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
