[
https://issues.apache.org/jira/browse/PIG-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755748#action_12755748
]
Ying He commented on PIG-961:
-----------------------------
there are a few problems while porting pig to hadoop 21 new API.
1. When running a Task (map or reduce), Two context are created, a JobContext
and a TaskAttempContext. Each context has its own copy of job config. So a
property set from one context can't be accessed from another context. I have
to make the following change to JobContext class to make pig work:
the original code makes a copy of input config:
public JobContext(Configuration conf, JobID jobId) {
this.conf = new org.apache.hadoop.mapred.JobConf(conf);
this.jobId = jobId;
}
I changed it to share the config object
public JobContext(Configuration conf, JobID jobId) {
if (conf instanceof org.apache.hadoop.mapred.JobConf) {
this.conf = (org.apache.hadoop.mapred.JobConf)conf;
}else{
this.conf = new org.apache.hadoop.mapred.JobConf(conf);
}
this.jobId = jobId;
}
2. The "Reporter" object is not visible from JobContext. There is no access to
reporter.incrCounter()
3. JobConf is obsolete. There is not access to some convenient methods such as
getUser(), Instead, I have to use config.get("user.name")
> Integration with Hadoop 21
> --------------------------
>
> Key: PIG-961
> URL: https://issues.apache.org/jira/browse/PIG-961
> Project: Pig
> Issue Type: New Feature
> Reporter: Olga Natkovich
> Assignee: Ying He
> Attachments: hadoop21.jar, PIG-961.patch
>
>
> Hadoop 21 is not yet released but we know that switch to new MR API is coming
> there. This JIRA is for early integration with this API
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
