Hey Matt, It looks like this is the problem: https://issues.apache.org/jira/browse/MAPREDUCE-4442
Is it causing the job to fail? We could add a similar workaround to the one described in the JIRA. J On Wed, Aug 7, 2013 at 5:37 PM, Matt Christiansen <[email protected]> wrote: > Hey guys, im getting this error while running a crunch job on our YARN > cluster. It doesn't seem to be serious buts generating a lot of QA > questions: > > > ERROR exec.MRExecutor: Exception thrown fetching job counters for > stage: com.rr.crunch.ads.KeyValuePairsAggregator: > > Avro(/QA/event_logs/2013_08_07/pixel_tracking)+S0+pre-distinct+GBK+post-distinct+S1+SeqFile(/tmp/crunch-1725623982/p1) > java.io.IOException > at > org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:317) > at > org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:337) > at > org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:476) > at org.apache.hadoop.mapreduce.Job$8.run(Job.java:757) > at org.apache.hadoop.mapreduce.Job$8.run(Job.java:754) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) > at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:754) > at > org.apache.crunch.impl.mr.exec.MRExecutor.execute(MRExecutor.java:69) > at org.apache.crunch.impl.mr.MRPipeline.run(MRPipeline.java:153) > at > com.rr.crunch.ads.KeyValuePairsAggregator.process(KeyValuePairsAggregator.java:79) > at > com.rr.crunch.ads.KeyValuePairsAggregator.main(KeyValuePairsAggregator.java:63) > > This is with crunch 0.5.0 but I have tried 0.7.0 and get the same > general error (no message about it being job counters but same strack > trace and in the code its at the job.getJob().getCounters() part) I > have tried both the hadoop-1 and hadoop-2 jars; I was wondering if any > one had come acrossed this before or had any idea > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
