[
https://issues.apache.org/jira/browse/HIVE-18269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289635#comment-16289635
]
Sergey Shelukhin commented on HIVE-18269:
-----------------------------------------
Interesting... CV memory use may be hard to estimate. Maybe the backpressure
can be based on list length, start at relatively low value, and then if
backpressure was triggered before and the list has emptied (causing operators
to wait) the value would go up?
> LLAP: Fast llap io with slow processing pipeline can lead to OOM
> ----------------------------------------------------------------
>
> Key: HIVE-18269
> URL: https://issues.apache.org/jira/browse/HIVE-18269
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Prasanth Jayachandran
> Assignee: Prasanth Jayachandran
> Attachments: Screen Shot 2017-12-13 at 1.15.16 AM.png
>
>
> pendingData linked list in Llap IO elevator (LlapRecordReader.java) may have
> grow indefinitely when Llap IO is faster than processing pipeline. Since we
> don't have backpressure to slow down the IO, this can lead to indefinite
> growth of pending data leading to severe GC pressure and eventually lead to
> OOM.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)