[ 
https://issues.apache.org/jira/browse/PIG-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755748#action_12755748
 ] 

Ying He commented on PIG-961:
-----------------------------

there are a few problems while porting pig to hadoop 21 new API.

1.  When running a Task (map or reduce), Two context are created, a JobContext 
and a TaskAttempContext. Each context has its own copy of job config. So a 
property set from one context can't be accessed from another context.  I have 
to make the following change to JobContext class to make pig work:

the original code makes a copy of input config:
public JobContext(Configuration conf, JobID jobId) {
       this.conf = new org.apache.hadoop.mapred.JobConf(conf);  
       this.jobId = jobId;
  }

I changed it to share the config object
public JobContext(Configuration conf, JobID jobId) {
          if (conf instanceof org.apache.hadoop.mapred.JobConf) {
                  this.conf = (org.apache.hadoop.mapred.JobConf)conf;
          }else{
                  this.conf = new org.apache.hadoop.mapred.JobConf(conf);
          }
    this.jobId = jobId;
  }

2. The "Reporter" object is not visible from JobContext. There is no access to 
reporter.incrCounter()

3. JobConf is obsolete. There is not access to some convenient methods such as 
getUser(),  Instead, I have to use config.get("user.name")

> Integration with Hadoop 21
> --------------------------
>
>                 Key: PIG-961
>                 URL: https://issues.apache.org/jira/browse/PIG-961
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Olga Natkovich
>            Assignee: Ying He
>         Attachments: hadoop21.jar, PIG-961.patch
>
>
> Hadoop 21 is not yet released but we know that switch to new MR API is coming 
> there. This JIRA is for early integration with this API

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to