[
https://issues.apache.org/jira/browse/TEZ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516264#comment-14516264
]
Hitesh Shah commented on TEZ-711:
---------------------------------
[~rajesh.balamohan] [~sseth] is this still valid?
> Fix memory leak when not reading from inputs due to caching
> -----------------------------------------------------------
>
> Key: TEZ-711
> URL: https://issues.apache.org/jira/browse/TEZ-711
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Rohini Palaniswamy
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: OOM-threaddump-711-5-patch.txt,
> OOM-threaddump-till-TEZ-752.txt, TEZ-711.5.txt, TEZ-711.wip.1.txt,
> TEZ-711.wip.2.txt, TEZ-711.wip.3.txt, TEZ-711.wip.4.txt
>
>
> When you are reading from inputs and caching objects with vertex scope, you
> don't have to read the input again when container is reused. But it allocates
> memory and that leaks causing OOM. KeyValueReader does not have a API to
> close the reader to clear allotted memory without reading from it. Also if
> there was a option to pre-close inputs in Processor and not fetch input at
> all over the wire and do shuffle/sort it would be a good optimization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
