[
https://issues.apache.org/jira/browse/TEZ-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935959#comment-14935959
]
Gopal V commented on TEZ-2850:
------------------------------
I've been trying to understand why we even have a reference to IFileInputStream
from the Segment.
The shuffleToMemory() should throw away the IFileInputStream as soon as it
copies the data into memory.
>From my understanding of the merger code, for in-memory segments, this buffer
>is assumed to be already thrown away after the reader pulls it into memory.
Only disk segments should be having 4kb chunks attached to them (a total of 4Mb
with a 100 sort factor).
> Tez MergeManager OOM for small Map Outputs
> ------------------------------------------
>
> Key: TEZ-2850
> URL: https://issues.apache.org/jira/browse/TEZ-2850
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Saikat
> Assignee: Saikat
> Attachments: OOM_1.png, OOM_2.png, OOM_3.png, TEZ-2850_test.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
