[
https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502440#comment-13502440
]
Shreepadma Venugopalan commented on HIVE-3678:
----------------------------------------------
@Ashutosh: My answers are inline.
We can add two more column in M*ColumnStatistics table of type BigDecimal:
BigDecimalLowValue and BigDecimalHighValue. But is BigDecimal type supported
consistently across different DBs?
Agreed, BigDecimal is not consistently supported across DBs. Hence we can't add
a BigDecimal column consistently across DB vendors and versions easily.
We can have these two columns of type Double, but then we loose precision.
Yes, we can store BigDecimal and Long as Double but we will lose precision.
We can store as plain strings in column of type varchar.
The maximum number of digits after the decimal point in a BigDecimal number is
unlimited for all practical purpose. If we stored it in a varchar, it could
result in truncation of some digits following the decimal point in some cases,
but this seems to be the only practical solution.
We can store in json format in column of type varchar.
The maximum number of digits in a BigDecimal number after the decimal point is
unlimited for all practical purposes (Java allows nearly 2 billion digits after
the decimal point). At this time, we collect MIN, MAX column values for numeric
columns. If we stored BigDecimal value, we may exceed the varchar size limit
and as a result truncate the JSON "blob". This would result in a malformed JSON
object. Additionally we will also lose some of the column statistics.
> Add metastore upgrade scripts for column stats schema changes
> -------------------------------------------------------------
>
> Key: HIVE-3678
> URL: https://issues.apache.org/jira/browse/HIVE-3678
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Shreepadma Venugopalan
> Assignee: Shreepadma Venugopalan
> Fix For: 0.10.0
>
> Attachments: HIVE-3678.1.patch.txt, HIVE-3678.2.patch.txt
>
>
> Add upgrade script for column statistics schema changes for
> Postgres/MySQL/Oracle/Derby
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
