Hi,
Please tell me steps to be followed to add Custom UDF in Hive 0.14 Source Code.
Thanks
Vengatesh Babu
---- On Wed, 24 Dec 2014 20:26:53 +0530 vengatesh.babu
<vengatesh.b...@zohocorp.com> wrote ----
Hi,
In Hive 0.14, I have written custom UDF to concat two strings by checking
"null" string.
This is my code.
package org.apache.hadoop.hive.ql.udf;
import org.apache.hadoop.hive.ql.exec.UDF;
public class ConcatNullCheck extends UDF {
public String evaluate(final String s, final String s1) {
if ((s == null) || (s1 == null) || s.equals("null") || s1.equals("null") )
{
return null;
}
return s+s1;
}
}
Add Jar method:
Above custom udf works fine. when i follow add jar & Create Function.
Add In source method:
I have added the same udf function in source code.
Steps Followed:
1. Add ConcatNullCheck.java in package org.apache.hadoop.hive.ql.udf;
2. Put Entry in FunctionRegistry.java (registerUDF("CONCAT_NULL",
ConcatNullCheck.class, false);)
3. Put Entry in Vectorizer.java
(supportedGenericUDFs.add(ConcatNullCheck.class);)
UDF Function works well for queries which does not require map-reduce job.
But, UDF function not working, Throwing NPE while doing Map-reduce.
Error: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:185)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.generateMapMetaData(MapJoinOperator.java:168)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:213)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1051)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1055)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1055)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1055)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:486)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:176)
... 8 more
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Thanks
Vengatesh Babu K M
