[
https://issues.apache.org/jira/browse/HIVE-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500376#comment-14500376
]
Sergey Shelukhin commented on HIVE-10217:
-----------------------------------------
I am working on the patch... yes, the approach is to use stream boundaries (in
compressed, CBs can't cross streams so that's implicit) and then chunk up large
streams (at least to max allocation). There are some assumptions that break;
e.g. you always read full CBs in compressed case, and in uncompressed you can
read any bytes in case of RG filter.
> LLAP: Support caching of uncompressed ORC data
> ----------------------------------------------
>
> Key: HIVE-10217
> URL: https://issues.apache.org/jira/browse/HIVE-10217
> Project: Hive
> Issue Type: Sub-task
> Affects Versions: llap
> Reporter: Gopal V
> Assignee: Sergey Shelukhin
> Fix For: llap
>
>
> {code}
> Caused by: java.io.IOException: ORC compression buffer size (0) is smaller
> than LLAP low-level cache minimum allocation size (131072). Decrease the
> value for hive.llap.io.cache.orc.alloc.min
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:137)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:48)
> at
> org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
