[
https://issues.apache.org/jira/browse/HIVE-23606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-23606:
----------------------------------
Labels: pull-request-available (was: )
> LLAP: Delay In DirectByteBuffer Clean Up For EncodedReaderImpl
> --------------------------------------------------------------
>
> Key: HIVE-23606
> URL: https://issues.apache.org/jira/browse/HIVE-23606
> Project: Hive
> Issue Type: Bug
> Components: llap
> Affects Versions: 3.0.0
> Reporter: Syed Shameerur Rahman
> Assignee: Syed Shameerur Rahman
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-23606.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> DirectByteBuffler are only cleaned up when there is Full GC or manually
> invoked cleaner method of DirectByteBuffer, Since full GC may take some time
> to kick in, In the meanwhile the native memory usage of LLAP daemon process
> might shoot up and this will force the YARN pmem monitor to kill the
> container running the daemon.
> HIVE-16180 tried to solve this problem, but the code structure got messed up
> after HIVE-15665
> The IdentityHashMap (toRelease) is initialized in
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java#L409
> , but it is getting re-initialized inside the method
> getDataFromCacheAndDisk()
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java#L633
> which makes it local to that method hence the original toRelease
> IdentityHashMap remains empty.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
