[
https://issues.apache.org/jira/browse/TEZ-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145104#comment-15145104
]
Bikas Saha commented on TEZ-3114:
---------------------------------
Backpressure looks reasonable. Though separately, if we can cut flab from
eventmetadata itself then that would also help with the "in the moment" memory
usage. Looks like we have identified some overweight elements in there.
> Shuffle OOM due to EventMetaData flood
> --------------------------------------
>
> Key: TEZ-3114
> URL: https://issues.apache.org/jira/browse/TEZ-3114
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: TEZ-3114.001.patch, TEZ-3114.002.patch
>
>
> A task encountered an OOM during shuffle, and investigation of the heap dump
> showed a lot of memory being consumed by almost 3.5 million EventMetaData
> objects. Auto-parallelism had reduced the number of tasks in the vertex to 1
> and there were 2000 upstream tasks to shuffle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
