[ 
https://issues.apache.org/jira/browse/HIVE-18269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289635#comment-16289635
 ] 

Sergey Shelukhin commented on HIVE-18269:
-----------------------------------------

Interesting... CV memory use may be hard to estimate. Maybe the backpressure 
can be based on list length, start at relatively low value, and then if 
backpressure was triggered before and the list has emptied (causing operators 
to wait) the value would go up?

> LLAP: Fast llap io with slow processing pipeline can lead to OOM
> ----------------------------------------------------------------
>
>                 Key: HIVE-18269
>                 URL: https://issues.apache.org/jira/browse/HIVE-18269
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: Screen Shot 2017-12-13 at 1.15.16 AM.png
>
>
> pendingData linked list in Llap IO elevator (LlapRecordReader.java) may have 
> grow indefinitely when Llap IO is faster than processing pipeline. Since we 
> don't have backpressure to slow down the IO, this can lead to indefinite 
> growth of pending data leading to severe GC pressure and eventually lead to 
> OOM.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to