[
https://issues.apache.org/jira/browse/PIG-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated PIG-1815:
-------------------------------
Attachment: yourkit_combiner_hprof.jpg
> pig task retains used instances of PhysicalPlan
> -----------------------------------------------
>
> Key: PIG-1815
> URL: https://issues.apache.org/jira/browse/PIG-1815
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0, 0.9.0
>
> Attachments: yourkit_combiner_hprof.jpg
>
>
> map tasks of a pig query ran out of memory because there were too many
> (thousands) instances of combiner PhysicalPlan in memory. Each physical plan
> (except the last?) was linked to older one as shown in the yourkit snapshot
> that I am attaching.
> This problem was noticed with 0.8 because of the split combination feature,
> that resulted in each map having larger inputs. The query also had large
> physical plan because of multi-query, it had 17 MR jobs merged into one
> during the multi-query optimization phase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
