[
https://issues.apache.org/jira/browse/HIVE-24721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-24721:
------------------------------------------
Description:
Before bumping to ORC-1.6, LLAP_ALLOCATOR_MAX_ALLOC value was also used as the
ORC CacheWriter buffer size.
As per ORC-238, the max bufferSize argument can be up to 2^(3*8 - 1) -- i.e.,
less than 8Mb.
In HIVE-23553 we decouple the two configuration (LLAP max alloc and CacheWriter
buffer size) -- this ticket is to investigate if there is a need for these two
conf to convergence.
More details:
https://github.com/apache/hive/pull/1823#pullrequestreview-575698916
was:
Before bumping to ORC-1.6, LLAP_ALLOCATOR_MAX_ALLOC value was also used as the
ORC CacheWriter buffer size.
As per ORC-238, the max bufferSize argument can be up to 2^(3*8 - 1) -- i.e.,
less than 8Mb and since we enforce the size to be power of 2 the next available
is 4Mb.
In HIVE-23553 we decouple the two configuration (LLAP max alloc and CacheWriter
buffer size) with the first one being 16Mb, and the latter 4Mb -- this ticket
is to investigate if there is a need for these two conf to convergence.
More details:
https://github.com/apache/hive/pull/1823#pullrequestreview-575698916
> Align CacheWriter bufferSize with Llap max alloc
> ------------------------------------------------
>
> Key: HIVE-24721
> URL: https://issues.apache.org/jira/browse/HIVE-24721
> Project: Hive
> Issue Type: Sub-task
> Reporter: Panagiotis Garefalakis
> Priority: Major
>
> Before bumping to ORC-1.6, LLAP_ALLOCATOR_MAX_ALLOC value was also used as
> the ORC CacheWriter buffer size.
> As per ORC-238, the max bufferSize argument can be up to 2^(3*8 - 1) -- i.e.,
> less than 8Mb.
> In HIVE-23553 we decouple the two configuration (LLAP max alloc and
> CacheWriter buffer size) -- this ticket is to investigate if there is a need
> for these two conf to convergence.
> More details:
> https://github.com/apache/hive/pull/1823#pullrequestreview-575698916
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
