If you run your script using oozie, you can control it using
<property>
<name>mapreduce.task.classpath.user.precedence</name>
<value>true</value>
</property>
Also try to add to add to the top of your script:
set mapreduce.task.classpath.user.precedence=true
2013/8/13 Niels Basjes <[email protected]>
> I've had the exact same problem with my own udf and joda.
> It turn out that an old version of joda is embedded inside the pig
> application.
> My workaround was limitying myself to the old joda api.
>
> Niels
> On Aug 13, 2013 10:31 AM, "Darpan R" <[email protected]> wrote:
>
> > Thanks Serega and Ankit.
> >
> > According to my understanding it looks that the actual joda-time jar is
> in
> > the classpath (otherwise it would have failed saying no classDef found).
> > But it is not able to find the "now" static method which is part of the
> > DateTime class.
> > Not sure if I am missing something.
> >
> > The transformation I am doing on the date field using joda date time API.
> >
> > Let me paste the complete stack trace :
> >
> >
> > 2013-08-11 13:01:06,911 [Thread-9] WARN
> > org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> > java.lang.NoSuchMethodError:
> >
> >
> org.joda.time.DateTime.now(Lorg/joda/time/DateTimeZone;)Lorg/joda/time/DateTime;
> > at
> >
> com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:178)
> > at
> > com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:12)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:337)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:381)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:334)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:241)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:465)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:433)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:413)
> > at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:257)
> > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> > at
> > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
> > at
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:262)
> > 2013-08-11 13:01:10,417 [main] WARN
> >
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - Ooops! Some job has failed! Specify -stop_on_fail
> >
> >
> >
> > Thanks again.
> > DR
> >
> > On 13 August 2013 11:28, Serega Sheypak <[email protected]>
> wrote:
> >
> > > Pig uses reflection. The top exception says that there is no such
> method
> > > signature. The problem is in the way you are trying to call method.
> > > And its better to paste the whole stavktrace
> > > 13.08.2013 7:39 пользователь "Darpan R" <[email protected]> написал:
> > >
> > > > I've a UDF which I use to do custom processing on the records. In the
> > > eval
> > > > function I am using a third party jar for processing. I saw the job
> jar
> > > > file, but it does not include this dependency. Is there any way to
> > > include
> > > > dependent jar in the job jar ? (For testing I am running the cluster
> in
> > > the
> > > > local mode).
> > > >
> > > > Or can I use distributed cache to make the dependent jar available to
> > the
> > > > UDF ?
> > > >
> > > > I've tried registering the dependent jars in the pig. For the first
> > > > registered jar (all udfs are bundled in this jar) I do not face the
> > > issues.
> > > > But for the second jar, I am facing issues when UDF tries to access
> the
> > > > class from it.
> > > > REGISTER '/home/user/pig/udfrepository/projectUDF.jar'
> > > > REGISTER '/home/user/thridpartyjars/xyz.jar';
> > > > The logs I get on the console are like this :
> > > > 2013-08-11 10:35:02,485 [Thread-14] WARN
> > > > org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> > > > java.lang.NoSuchMethodError:
> > > > org.xyz.abc.convertToOtherFormat(Lorg/DateTimeZone;)Lorg/DateTime;
> > > > at com.myproject.MyUDF.exec(MyUDF.java:70)
> > > > Any help on this is highly appreciated.
> > > > Thanks in advance.
> > > >
> > > > -Darpan
> > > >
> > >
> >
>
