[
https://issues.apache.org/jira/browse/HIVE-11222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623027#comment-14623027
]
Sergey Shelukhin commented on HIVE-11222:
-----------------------------------------
Assertion error was just to get a better error if the fix doesn't work. Right
now there's blah.getBlah().getBlah() line there and it's not clear what is null.
The race was that we cache metadata with necessary indexes only, not with all
of them, and add additional ones to cached object as needed.
So if query X is reading cols 0,1 and query Y cols 0,2, from the same stripe,
at the same time on the same machine, they both can see nothing in cache; read
in parallel; X would cache with 0,1, then Y would try to cache with 0,2, and
get an old object from X from the cache instead. Then the index would be null
for column 2 when it's needed.
The condition is rare enough that it's simpler, code-wise, to double check and
reread than it is to reconcile.
> LLAP: occasional NPE in parallel queries in ORC reader
> ------------------------------------------------------
>
> Key: HIVE-11222
> URL: https://issues.apache.org/jira/browse/HIVE-11222
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Fix For: llap
>
> Attachments: HIVE-11222.patch
>
>
> {noformat}
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.rethrowErrorIfAny(LlapInputFormat.java:275)
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.nextCvb(LlapInputFormat.java:227)
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:155)
> at
> org.apache.hadoop.hive.llap.io.api.impl.LlapInputFormat$LlapRecordReader.next(LlapInputFormat.java:101)
> at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:350)
> ... 22 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:709)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.determineRgsToRead(OrcEncodedDataReader.java:618)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:195)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:59)
> at
> org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
> ... 4 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
