[ https://issues.apache.org/jira/browse/TEZ-711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516291#comment-14516291 ]
Siddharth Seth commented on TEZ-711: ------------------------------------ Yes it is. > Fix memory leak when not reading from inputs due to caching > ----------------------------------------------------------- > > Key: TEZ-711 > URL: https://issues.apache.org/jira/browse/TEZ-711 > Project: Apache Tez > Issue Type: Bug > Affects Versions: 0.2.0 > Reporter: Rohini Palaniswamy > Assignee: Siddharth Seth > Priority: Critical > Attachments: OOM-threaddump-711-5-patch.txt, > OOM-threaddump-till-TEZ-752.txt, TEZ-711.5.txt, TEZ-711.wip.1.txt, > TEZ-711.wip.2.txt, TEZ-711.wip.3.txt, TEZ-711.wip.4.txt > > > When you are reading from inputs and caching objects with vertex scope, you > don't have to read the input again when container is reused. But it allocates > memory and that leaks causing OOM. KeyValueReader does not have a API to > close the reader to clear allotted memory without reading from it. Also if > there was a option to pre-close inputs in Processor and not fetch input at > all over the wire and do shuffle/sort it would be a good optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)