Which version of hadoop-2 are you testing against? Did you start up the mapreduce specific JobHistoryServer?
Thanks, +Vinod On Aug 7, 2013, at 5:37 PM, Matt Christiansen wrote: > Hey guys, im getting this error while running a crunch job on our YARN > cluster. It doesn't seem to be serious buts generating a lot of QA > questions: > > > ERROR exec.MRExecutor: Exception thrown fetching job counters for > stage: com.rr.crunch.ads.KeyValuePairsAggregator: > Avro(/QA/event_logs/2013_08_07/pixel_tracking)+S0+pre-distinct+GBK+post-distinct+S1+SeqFile(/tmp/crunch-1725623982/p1) > java.io.IOException > at > org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:317) > at > org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:337) > at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:476) > at org.apache.hadoop.mapreduce.Job$8.run(Job.java:757) > at org.apache.hadoop.mapreduce.Job$8.run(Job.java:754) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) > at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:754) > at org.apache.crunch.impl.mr.exec.MRExecutor.execute(MRExecutor.java:69) > at org.apache.crunch.impl.mr.MRPipeline.run(MRPipeline.java:153) > at > com.rr.crunch.ads.KeyValuePairsAggregator.process(KeyValuePairsAggregator.java:79) > at > com.rr.crunch.ads.KeyValuePairsAggregator.main(KeyValuePairsAggregator.java:63) > > This is with crunch 0.5.0 but I have tried 0.7.0 and get the same > general error (no message about it being job counters but same strack > trace and in the code its at the job.getJob().getCounters() part) I > have tried both the hadoop-1 and hadoop-2 jars; I was wondering if any > one had come acrossed this before or had any idea -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
