[
https://issues.apache.org/jira/browse/PIG-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235986#comment-15235986
]
Rohini Palaniswamy commented on PIG-4844:
-----------------------------------------
bq. Can we separate it into two ticket? The PigContext change is more
fundamental
sure. Will be easy to revert or fix if there are any issues with PigContext
changes.
bq. Did you run through unit tests and e2e tests?
Yes. Already deployed in our research clusters as well.
> Tez AM runs out of memory when vertex has high number of outputs
> ----------------------------------------------------------------
>
> Key: PIG-4844
> URL: https://issues.apache.org/jira/browse/PIG-4844
> Project: Pig
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.16.0
>
> Attachments: PIG-4844-1-nowhitespacechanges.patch, PIG-4844-1.patch
>
>
> AM runs out of memory when trying to respond to getTask() calls from
> container for a vertex with large number of outputs (usually the case with
> multi-query when you group by on multiple dimensions). Problem is with the
> size of payload config associated with PigProcessor, Input and Output. When
> there is >10 outputs size of the payload considerably increases causing
> memory pressure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
