[
https://issues.apache.org/jira/browse/HIVE-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591142#comment-14591142
]
Prasanth Jayachandran commented on HIVE-11031:
----------------------------------------------
[~gopalv] Added some changes to throw when incompatible statistics gets merged.
Also the orc files which does not have stripe statistics will be added to
incompatible file set (ignored from merging).
> ORC concatenation of old files can fail while merging column statistics
> -----------------------------------------------------------------------
>
> Key: HIVE-11031
> URL: https://issues.apache.org/jira/browse/HIVE-11031
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.13.0, 0.14.0, 1.0.0, 1.2.0, 1.1.0, 2.0.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Priority: Critical
> Attachments: HIVE-11031.2.patch, HIVE-11031.patch
>
>
> Column statistics in ORC are optional protobuf fields. Old ORC files might
> not have statistics for newly added types like decimal, date, timestamp etc.
> But column statistics merging assumes column statistics exists for these
> types and invokes merge. For example, merging of TimestampColumnStatistics
> directly casts the received ColumnStatistics object without doing instanceof
> check. If the ORC file contains time stamp column statistics then this will
> work else it will throw ClassCastException.
> Also, the file merge operator swallows the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
