[
https://issues.apache.org/jira/browse/TEZ-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190424#comment-15190424
]
Jonathan Eagles commented on TEZ-3145:
--------------------------------------
Tried the above patch out at scale and it puts too much pressure on the AM for
large jobs. Filed TEZ-3163 which reduces the messages by a very small amount,
but more importantly reduces processing time of the messages. If they can't be
smaller at least they can be faster. Combined with message throttling, this
should be sufficient for handling the needed scale for now. I would like to
keep this jira open for now to try a few more ideas.
> Reduce message size when empty partitions is high
> -------------------------------------------------
>
> Key: TEZ-3145
> URL: https://issues.apache.org/jira/browse/TEZ-3145
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-3145.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
