java.lang.NoSuchMethodError: org.apache.hadoop.mapred. jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/ jobcontrol/Job;)Ljava/lang/String;
This error usually happens when you use Pig compiled with -Dhadoopversion=23 with Hadoop 1.x mapreduce or -Dhadoopversion=20 with Hadoop 2.x mapreduce cluster. Regards, Rohini On Thu, Feb 5, 2015 at 7:51 AM, Carlos Balduz Bernal <cbal...@gmail.com> wrote: > Hi, > > Our cluster uses Hortonworks HDP 2.2 with Hadoop 2.6.0 and Pig 0.14 and we > haven't faced any problem yet. > > Regards, > > Carlos Balduz. > > On 5 February 2015 at 16:37, Quang-Nhat HOANG-XUAN <hxquangn...@gmail.com> > wrote: > > > Hi, > > I faced this problem before and still can not fix it. My cluster is > hadoop > > 2.0.0-cdh4.4.0, it can work with Pig 13.0 but can not with Pig 14.0. > > CDH distro is very complicated because it packages both mapreduce1 and > > mapreduce2. > > The only solution I got is waiting for Cloudera to upgrade their Pig or > you > > have to use a Hadoop distro from Apache. > > > > Quang-Nhat > > > > On Thu, Feb 5, 2015 at 4:20 PM, Michael Bohn <michael.b...@freiheit.com> > > wrote: > > > > > We updated (successful) our Pig to 0.12 and it worked. > > > > > > But it shouldn't matter where Hadoop comes from? I guess this problem > > > exists with > > > every combination of Hadoop 2.0 and Pig 0.14? > > > > > > Regards, > > > Michael > > > > > > On 05.02.2015, at 16:15, Russell Jurney <russell.jur...@gmail.com> > > wrote: > > > > > > CDH pig is the only Pig that works with CDH. In fact, it has to be the > > > exact pig for that release of CDH. I tried very, very hard to use pig > 12 > > > with CDH 4.x, and pig 13/14 with CDH 5.x, and I believe it is not > > possible. > > > > > > Bug Cloudera to upgrade their Pig. Sorry :( > > > > > > On Thursday, February 5, 2015, Michael Bohn <michael.b...@freiheit.com > > > > > wrote: > > > > > > Hi, > > > > > > we are trying to use the latest Pig-Version 0.14 with our Hadoop 2.0.0 > > > installation. > > > We are using: 2.0.0-cdh4.7.0.jar > > > > > > This version seems not to be compatible with Pig 0.14. > > > > > > Even this minimal script: > > > > > > D = load 'test.txt'; > > > dump D; > > > > > > and test.txt an empty file, fails with the error: > > > > > > 2015-02-05 12:57:51,133 [main] WARN > org.apache.hadoop.conf.Configuration > > > - dfs.safemode.extension is deprecated. Instead, use > > > dfs.namenode.safemode.extension > > > Pig Stack Trace > > > --------------- > > > ERROR 2998: Unhandled internal error. > > > > > > > > > org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String; > > > > > > java.lang.NoSuchMethodError: > > > > > > > > > org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String; > > > at > > > > > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:325) > > > at > > > > > > > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:196) > > > at > > > > > > > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:280) > > > at org.apache.pig.PigServer.launchPlan(PigServer.java:1390) > > > at > > > > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375) > > > at org.apache.pig.PigServer.storeEx(PigServer.java:1034) > > > at org.apache.pig.PigServer.store(PigServer.java:997) > > > at org.apache.pig.PigServer.openIterator(PigServer.java:910) > > > at > > > > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:746) > > > at > > > > > > > > > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372) > > > at > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) > > > at > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) > > > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) > > > at org.apache.pig.Main.run(Main.java:624) > > > at org.apache.pig.Main.main(Main.java:170) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at org.apache.hadoop.util.RunJar.main(RunJar.java:208) > > > > > > > > > > > > ================================================================================ > > > > > > > > > The release notes promise compatibility with Hadoop 2.X. Is this a bug? > > > Is there a workaround? > > > > > > Best regards and thanks for your help. > > > -- Michael > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Russell Jurney twitter.com/rjurney russell.jur...@gmail.com > > > datasyndrome.com > > > > > > > > > > > >
