I've a UDF which I use to do custom processing on the records. In the eval
function I am using a third party jar for processing. I saw the job jar
file, but it does not include this dependency. Is there any way to include
dependent jar in the job jar ? (For testing I am running the cluster in the
local mode).
Or can I use distributed cache to make the dependent jar available to the
UDF ?
I've tried registering the dependent jars in the pig. For the first
registered jar (all udfs are bundled in this jar) I do not face the issues.
But for the second jar, I am facing issues when UDF tries to access the
class from it.
REGISTER '/home/user/pig/udfrepository/projectUDF.jar'
REGISTER '/home/user/thridpartyjars/xyz.jar';
The logs I get on the console are like this :
2013-08-11 10:35:02,485 [Thread-14] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.NoSuchMethodError:
org.xyz.abc.convertToOtherFormat(Lorg/DateTimeZone;)Lorg/DateTime;
at com.myproject.MyUDF.exec(MyUDF.java:70)
Any help on this is highly appreciated.
Thanks in advance.
-Darpan
