Thanks Serega and Ankit.
According to my understanding it looks that the actual joda-time jar is in
the classpath (otherwise it would have failed saying no classDef found).
But it is not able to find the "now" static method which is part of the
DateTime class.
Not sure if I am missing something.
The transformation I am doing on the date field using joda date time API.
Let me paste the complete stack trace :
2013-08-11 13:01:06,911 [Thread-9] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.NoSuchMethodError:
org.joda.time.DateTime.now(Lorg/joda/time/DateTimeZone;)Lorg/joda/time/DateTime;
at
com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:178)
at
com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:12)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:337)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:381)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:334)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:241)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:465)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:433)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:413)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:257)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:262)
2013-08-11 13:01:10,417 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Ooops! Some job has failed! Specify -stop_on_fail
Thanks again.
DR
On 13 August 2013 11:28, Serega Sheypak <[email protected]> wrote:
> Pig uses reflection. The top exception says that there is no such method
> signature. The problem is in the way you are trying to call method.
> And its better to paste the whole stavktrace
> 13.08.2013 7:39 пользователь "Darpan R" <[email protected]> написал:
>
> > I've a UDF which I use to do custom processing on the records. In the
> eval
> > function I am using a third party jar for processing. I saw the job jar
> > file, but it does not include this dependency. Is there any way to
> include
> > dependent jar in the job jar ? (For testing I am running the cluster in
> the
> > local mode).
> >
> > Or can I use distributed cache to make the dependent jar available to the
> > UDF ?
> >
> > I've tried registering the dependent jars in the pig. For the first
> > registered jar (all udfs are bundled in this jar) I do not face the
> issues.
> > But for the second jar, I am facing issues when UDF tries to access the
> > class from it.
> > REGISTER '/home/user/pig/udfrepository/projectUDF.jar'
> > REGISTER '/home/user/thridpartyjars/xyz.jar';
> > The logs I get on the console are like this :
> > 2013-08-11 10:35:02,485 [Thread-14] WARN
> > org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> > java.lang.NoSuchMethodError:
> > org.xyz.abc.convertToOtherFormat(Lorg/DateTimeZone;)Lorg/DateTime;
> > at com.myproject.MyUDF.exec(MyUDF.java:70)
> > Any help on this is highly appreciated.
> > Thanks in advance.
> >
> > -Darpan
> >
>
