[ https://issues.apache.org/jira/browse/ORC-285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362759#comment-16362759 ]
Owen O'Malley commented on ORC-285: ----------------------------------- For the Spark project, I looked into the background of this bug. My findings are: After a bit of digging, the problem that ORC-285 fixes was introduced in HIVE-9711. HIVE-9711 is included in Hive 1.2.0, but doesn't create a problem unless the vectorized ORC reader is used. That didn't become the default behavior until HIVE-11417, which was released in Hive 2.1.0. > Empty vector batches of floats or doubles get java.io.EOFException > ------------------------------------------------------------------- > > Key: ORC-285 > URL: https://issues.apache.org/jira/browse/ORC-285 > Project: ORC > Issue Type: Bug > Affects Versions: 1.3.4, 1.4.1 > Reporter: Owen O'Malley > Assignee: Owen O'Malley > Priority: Major > Fix For: 1.4.3, 1.5.0 > > > The FloatTreeReader and DoubleTreeReader both fail if given a set of empty > values. This often happens when the float or double data is inside a list > where all of the values are empty. > The stack trace looks like: > {code} > java.io.IOException: Error reading file: > /Users/piyush.mukati/tmp/11-task_1511779500016_298243_r_000031-r-00031.orc > at > org.apache.orc.impl.RecordReaderImpl.nextBatch(RecordReaderImpl.java:1191) > at org.apache.orc.tools.ScanData.main(ScanData.java:67) > at org.apache.orc.tools.Driver.main(Driver.java:109) > Caused by: java.io.EOFException: Read past EOF for compressed stream Stream > for column 36 kind DATA position: 0 length: 0 range: 0 offset: 0 limit: 0 > at > org.apache.orc.impl.SerializationUtils.readFully(SerializationUtils.java:119) > at > org.apache.orc.impl.SerializationUtils.readLongLE(SerializationUtils.java:102) > at > org.apache.orc.impl.SerializationUtils.readDouble(SerializationUtils.java:98) > at > org.apache.orc.impl.TreeReaderFactory$DoubleTreeReader.nextVector(TreeReaderFactory.java:763) > at > org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextVector(TreeReaderFactory.java:1835) > at > org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextVector(TreeReaderFactory.java:1835) > at > org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextVector(TreeReaderFactory.java:1835) > at > org.apache.orc.impl.TreeReaderFactory$ListTreeReader.nextVector(TreeReaderFactory.java:2003) > at > org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextVector(TreeReaderFactory.java:1835) > at > org.apache.orc.impl.TreeReaderFactory$ListTreeReader.nextVector(TreeReaderFactory.java:2003) > at > org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextVector(TreeReaderFactory.java:1835) > at > org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextBatch(TreeReaderFactory.java:1817) > at > org.apache.orc.impl.RecordReaderImpl.nextBatch(RecordReaderImpl.java:1184) > ... 2 more > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)