[
https://issues.apache.org/jira/browse/PIG-3365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lorand Bendig updated PIG-3365:
-------------------------------
Attachment: PIG-3365-2.patch
[~rohini], thanks for having look at the patch.
The aim of this initial patch was to follow the logic of Hadoop's ubertask
decision at the Pig side.
{{JobControlCompiler#okToRunLocal}} just takes an additional parameter
{{totalInputFileSize}} from {{JobControlCompiler#getJob}} where the ubertask
decision is done. Because totalInputFileSize is needed for the decision I
passed this parameter to okToRunLocal() so that it won't be recalculated.
I attached a further patch based on your suggestion. As far as I see enabling
uber mode in {{PigInputFormat#getSplits}} will be picked up by the job.
> Run as uber job if there is only one input split
> ------------------------------------------------
>
> Key: PIG-3365
> URL: https://issues.apache.org/jira/browse/PIG-3365
> Project: Pig
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Assignee: Lorand Bendig
> Labels: Performance
> Attachments: PIG-3365-2.patch, PIG-3365.patch
>
>
> Hadoop 2 has support for uber mode (mapreduce.job.ubertask.enable=true) which
> runs the map and reduce on Application Master itself and reduces the overhead
> of launching a separate map/reduce task.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
