[
https://issues.apache.org/jira/browse/HIVE-13617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347973#comment-15347973
]
Prasanth Jayachandran commented on HIVE-13617:
----------------------------------------------
[~sershe] This patch is causing test failures in master. Following 2 failures
are related
{code}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
{code}
Reverting this patch makes these tests pass.
Following exception is thrown
{code}
Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 4
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.rethrowErrorIfAny(LlapInputFormat.java:346)
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.nextCvb(LlapInputFormat.java:302)
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:227)
at
org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:148)
at
org.apache.hadoop.hive.ql.io.BatchToRowReader.ensureBatch(BatchToRowReader.java:167)
at
org.apache.hadoop.hive.ql.io.BatchToRowReader.next(BatchToRowReader.java:140)
at
org.apache.hadoop.hive.ql.io.BatchToRowReader.next(BatchToRowReader.java:78)
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:350)
... 22 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:240)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:417)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:209)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:206)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:90)
... 5 more
{code}
Created HIVE-14089 for tracking.
> LLAP: support non-vectorized execution in IO
> --------------------------------------------
>
> Key: HIVE-13617
> URL: https://issues.apache.org/jira/browse/HIVE-13617
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Fix For: 2.2.0
>
> Attachments: HIVE-13617-wo-11417.patch, HIVE-13617-wo-11417.patch,
> HIVE-13617.01.patch, HIVE-13617.03.patch, HIVE-13617.04.patch,
> HIVE-13617.05.patch, HIVE-13617.06.patch, HIVE-13617.patch, HIVE-13617.patch,
> HIVE-15396-with-oi.patch
>
>
> Two approaches - a separate decoding path, into rows instead of VRBs; or
> decoding VRBs into rows on a higher level (the original LlapInputFormat). I
> think the latter might be better - it's not a hugely important path, and perf
> in non-vectorized case is not the best anyway, so it's better to make do with
> much less new code and architectural disruption.
> Some ORC patches in progress introduce an easy to reuse (or so I hope,
> anyway) VRB-to-row conversion, so we should just use that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
