[
https://issues.apache.org/jira/browse/TEZ-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277905#comment-14277905
]
Rajesh Balamohan commented on TEZ-1945:
---------------------------------------
SPILLED_RECORDS can be > 0 as it is accounted for in finalMerge (mem + disk).
Will create a separate JIRA for post_merge_mem_limit.
> Remove 2 GB memlimit restriction in MergeManager
> ------------------------------------------------
>
> Key: TEZ-1945
> URL: https://issues.apache.org/jira/browse/TEZ-1945
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1945.1.patch
>
>
> In certain situations (data coming in larger chunks, but yet to complete),
> fetchers might wait in MerManager.waitForShuffleToMergeMemory() for memory to
> become available.
> Removing the 2 GB resitrction on MergeManager.memlimit would help in such
> situations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
