[
https://issues.apache.org/jira/browse/TEZ-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185900#comment-15185900
]
Siddharth Seth commented on TEZ-3145:
-------------------------------------
[~jeagles] - have you seen issues with the size of the events sent to tasks ?
Deserializing messages in the AM, and reconstructing them can be quite
expensive. Single process doing all of this - and will likely create temporary
duplicate events which can cause AM memory pressure.
Also, the Edge code should not refer to anything in tez-runtime-library. The
payload is meant to be specific to Inputs/Outputs. This flow will not work if
there were to be an alternate payload format.
> Reduce message size when empty partitions is high
> -------------------------------------------------
>
> Key: TEZ-3145
> URL: https://issues.apache.org/jira/browse/TEZ-3145
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-3145.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
