After change from hadoop20 to hadoop23 the warning dissapeared but I got
the same exception (Found interface
org.apache.hadoop.mapreduce.JobContext, but class was expected)
I have tried over a fresh install: hadoop 2.2.0 and pig 0.12.1 compiled
by me, no other product nor configuration, just two servers, one master
with ResourceManager and NameNode, one slave with DataNode and NodeManager.
I can't understand why over this fresh cluster Pig 0.12 fails. Here is
the new trace:
2013-10-24 16:10:52,351 [JobControl] ERROR
org.apache.pig.backend.hadoop23.PigJobControl - Error while trying to
run jobs.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at
org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:130)
at
org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
at java.lang.Thread.run(Thread.java:724)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
... 3 more
Caused by: java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.JobContext, but class was expected
at
org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:113)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
at
org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
... 8 more
El 24/10/13 21:33, Prashant Kommireddi escribió:
Yes it does. You need to recompile Pig for hadoop 2
ant clean jar-withouthadoop -Dhadoopversion=23
On Thu, Oct 24, 2013 at 5:37 AM, Claudio Romo Otto <
claudio.r...@namictech.com> wrote:
Does Pig support Hadoop 2.2? When I try Pig 0.12 and Hadoop 2.2 I get an
error even with simple operations like
data = LOAD 'cql://keyspace1/testcf?' USING CqlStorage();
dump data;
I only got a warning first and then and exception:
2013-10-24 09:35:19,300 [main] WARN
org.apache.pig.backend.**hadoop20.PigJobControl
- falling back to default JobControl (not using hadoop 0.20 ?)
java.lang.**NoSuchFieldException: runnerState
at java.lang.Class.**getDeclaredField(Class.java:**1938)
at org.apache.pig.backend.**hadoop20.PigJobControl.<**
clinit>(PigJobControl.java:51)
at org.apache.pig.backend.hadoop.**executionengine.shims.**
HadoopShims.newJobControl(**HadoopShims.java:97)
at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
JobControlCompiler.compile(**JobControlCompiler.java:285)
at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
MapReduceLauncher.launchPig(**MapReduceLauncher.java:177)
at org.apache.pig.PigServer.**launchPlan(PigServer.java:**1264)
at org.apache.pig.PigServer.**executeCompiledLogicalPlan(**
PigServer.java:1249)
at org.apache.pig.PigServer.**storeEx(PigServer.java:931)
at org.apache.pig.PigServer.**store(PigServer.java:898)
at org.apache.pig.PigServer.**openIterator(PigServer.java:**811)
at org.apache.pig.tools.grunt.**GruntParser.processDump(**
GruntParser.java:696)
at org.apache.pig.tools.**pigscript.parser.**PigScriptParser.parse(**
PigScriptParser.java:320)
at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(**
GruntParser.java:194)
at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(**
GruntParser.java:170)
at org.apache.pig.tools.grunt.**Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.**java:538)
at org.apache.pig.Main.main(Main.**java:157)
at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
at sun.reflect.**NativeMethodAccessorImpl.**invoke(**
NativeMethodAccessorImpl.java:**57)
at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
DelegatingMethodAccessorImpl.**java:43)
at java.lang.reflect.Method.**invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.**main(RunJar.java:212)
------------------------------**--------
Backend error message during job submission
------------------------------**-------------
Unexpected System Error Occured: java.lang.**IncompatibleClassChangeError:
Found interface org.apache.hadoop.mapreduce.**JobContext, but class was
expected
at org.apache.pig.backend.hadoop.**executionengine.**
mapReduceLayer.**PigOutputFormat.**setupUdfEnvAndStores(**
PigOutputFormat.java:225)
at org.apache.pig.backend.hadoop.**executionengine.**
mapReduceLayer.**PigOutputFormat.**checkOutputSpecs(**
PigOutputFormat.java:186)
at org.apache.hadoop.mapreduce.**JobSubmitter.checkSpecs(**
JobSubmitter.java:456)
at org.apache.hadoop.mapreduce.**JobSubmitter.**submitJobInternal(
**JobSubmitter.java:342)
at org.apache.hadoop.mapreduce.**Job$10.run(Job.java:1268)
at org.apache.hadoop.mapreduce.**Job$10.run(Job.java:1265)
at java.security.**AccessController.doPrivileged(**Native Method)
at javax.security.auth.Subject.**doAs(Subject.java:415)
at org.apache.hadoop.security.**UserGroupInformation.doAs(**
UserGroupInformation.java:**1491)
at org.apache.hadoop.mapreduce.**Job.submit(Job.java:1265)
at org.apache.hadoop.mapreduce.**lib.jobcontrol.ControlledJob.**
submit(ControlledJob.java:335)
at org.apache.hadoop.mapreduce.**lib.jobcontrol.JobControl.run(**
JobControl.java:240)
at org.apache.pig.backend.**hadoop20.PigJobControl.run(**
PigJobControl.java:121)
at java.lang.Thread.run(Thread.**java:724)
at org.apache.pig.backend.hadoop.**executionengine.**
mapReduceLayer.**MapReduceLauncher$1.run(**MapReduceLauncher.java:257)
Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias data
org.apache.pig.impl.**logicalLayer.**FrontendException: ERROR 1066:
Unable to open iterator for alias data
at org.apache.pig.PigServer.**openIterator(PigServer.java:**836)
at org.apache.pig.tools.grunt.**GruntParser.processDump(**
GruntParser.java:696)
at org.apache.pig.tools.**pigscript.parser.**
PigScriptParser.parse(**PigScriptParser.java:320)
at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(**
GruntParser.java:194)
at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(**
GruntParser.java:170)
at org.apache.pig.tools.grunt.**Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.**java:538)
at org.apache.pig.Main.main(Main.**java:157)
at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
at sun.reflect.**NativeMethodAccessorImpl.**invoke(**
NativeMethodAccessorImpl.java:**57)
at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
DelegatingMethodAccessorImpl.**java:43)
at java.lang.reflect.Method.**invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.**main(RunJar.java:212)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
at org.apache.pig.PigServer.**openIterator(PigServer.java:**828)
... 12 more
