[
https://issues.apache.org/jira/browse/TEZ-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145141#comment-15145141
]
Siddharth Seth commented on TEZ-3114:
-------------------------------------
bq. @ConfigurationScope(Scope.VERTEX)
Don't think this will be respected at the moment. Please add a comment to
indicate this. I think [~rohini] had filed a jira about ensuring some of these
task properties are respected correctly (currently setup once per container)
Rest looks good, much simpler than what I was expecting it to be :) Could you
please add a test. There's already one around fetching again when maxEvents are
received.
> Shuffle OOM due to EventMetaData flood
> --------------------------------------
>
> Key: TEZ-3114
> URL: https://issues.apache.org/jira/browse/TEZ-3114
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: TEZ-3114.001.patch, TEZ-3114.002.patch
>
>
> A task encountered an OOM during shuffle, and investigation of the heap dump
> showed a lot of memory being consumed by almost 3.5 million EventMetaData
> objects. Auto-parallelism had reduced the number of tasks in the vertex to 1
> and there were 2000 upstream tasks to shuffle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
