[
https://issues.apache.org/jira/browse/HIVE-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233127#comment-15233127
]
Prasanth Jayachandran commented on HIVE-13223:
----------------------------------------------
ORC reader handles only non-orc files correctly by throwing
FileFormatException. The reader itself does not handle 0 length files. The way
it's handled currently is OrcInputFormat just ignores 0 length files from split
computation as it knows that it cannot be valid orc file. Also there are
filters to prune hidden and _* files which are also not valid orc files. So ORC
reader expects only valid ORC files. I think it should be handled at both
places (split generation and reader) as both can be used together or
independently. I can add a check to ORC reader to throw exception when 0 length
files are encountered.
> HoS may hang for queries that run on 0 splits
> -----------------------------------------------
>
> Key: HIVE-13223
> URL: https://issues.apache.org/jira/browse/HIVE-13223
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Affects Versions: 2.1.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: HIVE-13223.1.patch, HIVE-13223.2.patch, HIVE-13223.patch
>
>
> Can be seen on all timed out tests after HIVE-13040 went in
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
