[ https://issues.apache.org/jira/browse/PIG-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755748#action_12755748 ]
Ying He commented on PIG-961: ----------------------------- there are a few problems while porting pig to hadoop 21 new API. 1. When running a Task (map or reduce), Two context are created, a JobContext and a TaskAttempContext. Each context has its own copy of job config. So a property set from one context can't be accessed from another context. I have to make the following change to JobContext class to make pig work: the original code makes a copy of input config: public JobContext(Configuration conf, JobID jobId) { this.conf = new org.apache.hadoop.mapred.JobConf(conf); this.jobId = jobId; } I changed it to share the config object public JobContext(Configuration conf, JobID jobId) { if (conf instanceof org.apache.hadoop.mapred.JobConf) { this.conf = (org.apache.hadoop.mapred.JobConf)conf; }else{ this.conf = new org.apache.hadoop.mapred.JobConf(conf); } this.jobId = jobId; } 2. The "Reporter" object is not visible from JobContext. There is no access to reporter.incrCounter() 3. JobConf is obsolete. There is not access to some convenient methods such as getUser(), Instead, I have to use config.get("user.name") > Integration with Hadoop 21 > -------------------------- > > Key: PIG-961 > URL: https://issues.apache.org/jira/browse/PIG-961 > Project: Pig > Issue Type: New Feature > Reporter: Olga Natkovich > Assignee: Ying He > Attachments: hadoop21.jar, PIG-961.patch > > > Hadoop 21 is not yet released but we know that switch to new MR API is coming > there. This JIRA is for early integration with this API -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.