[
https://issues.apache.org/jira/browse/PIG-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366602#comment-15366602
]
Rohini Palaniswamy commented on PIG-4626:
-----------------------------------------
Just came across one user job where they were setting 80% of heap as io.sort.mb
as they had lot of spill. Job was still slow because we were setting 0.5 for
tez.task.scale.memory.reserve-fraction and it used a smaller sort buffer. So
apart from looking at whether the job has POPartialAgg or Replicated join and
reducing the percentage, we also need to see how much is io.sort.mb part of
heap and increase the value if it is really high.
> [Pig on Tez] OOM in case of multiple outputs and POPartialAgg
> -------------------------------------------------------------
>
> Key: PIG-4626
> URL: https://issues.apache.org/jira/browse/PIG-4626
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.17.0
>
>
> tez.task.scale.memory.reserve-fraction is 0.3 by default which assumes 70%
> of memory is available for io.sort.mb. If map is configured as 1G Xmx and
> io.sort.mb as 256MB, and there is group by on 3 different keys it has 3
> different outputs and Tez's WeightedScalingMemoryDistributor allocates 256MB
> to each of the outputs which leaves very less memory for Pig and POPartialAgg
> (which requires 20% of memory) and sometime leads to OOM. This also causes
> SpillableMemoryManager to be invoked often as its threshold is set to 70% of
> memory.
> Need to set tez.task.scale.memory.reserve-fraction as 0.5 for all pig jobs
> and 0.6 in case there is POPartialAgg in combiner plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
