[
https://issues.apache.org/jira/browse/PIG-3365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042783#comment-14042783
]
Lorand Bendig commented on PIG-3365:
------------------------------------
[~chitnis], thanks for the suggestions. I'll modify the patch so that
properties will be extracted into static variables. The property
{{dfs.block.size}} is deprecated in Hadoop2, {{dfs.blocksize}} is used instead.
Initially I used this property to retrieve the block size but I remarked that
MAPREDUCE-3610 changed the way how blocksize is determined and also checked
Hadoop's {{JobImpl#makeUberDecision}} and followed that. I'm not quite sure
whether dfs.blocksize returns the current blocksize if the file system isn't
hdfs (E.g: for s3n there's {{fs.s3n.block.size}}). I think if we get blocksize
from the current fs implementation that might be safer. What do you think?
> Run as uber job if there is only one input split
> ------------------------------------------------
>
> Key: PIG-3365
> URL: https://issues.apache.org/jira/browse/PIG-3365
> Project: Pig
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Assignee: Lorand Bendig
> Labels: Performance
> Attachments: PIG-3365.patch
>
>
> Hadoop 2 has support for uber mode (mapreduce.job.ubertask.enable=true) which
> runs the map and reduce on Application Master itself and reduces the overhead
> of launching a separate map/reduce task.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
