[
https://issues.apache.org/jira/browse/HIVE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-12719:
--------------------------------------
Release Note:
We will get invalid function error when permanent function is used along with
window function because the permanent function is stored as
db_name.function_name in the function registry where as built-in/temporary
function gets stored without any prefix in the function registry.
The fix is done such that the we will first form the qualified name for the
permanent function and then compare if it exists in function registry.
was:
I am facing an issues while calling custom generic UDAF's in HIVE (1.2.1).
Below is my implementation for GenericUDAF : -
/**
*
*/
package com.practice.hive.udaf.test;
import org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.parse.SemanticException;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFParameterInfo;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver2;
import org.apache.hadoop.hive.ql.util.JavaDataModel;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
import org.apache.hadoop.io.LongWritable;
/**
* @author surbhit.shrivastava
*
*/
public class GenericUDAFRegrCount implements GenericUDAFResolver2 {
// This is deprecated will not be called .. but is present for backward
compatibility purpose
@Override
public GenericUDAFEvaluator getEvaluator(TypeInfo[] parameters)
throws SemanticException {
if (parameters.length != 2) {
throw new UDFArgumentTypeException(parameters.length -
1,
"Exactly two arguments are expected.");
}
return new GenericUDAFRegrCountEvaluator();
}
@Override
public GenericUDAFEvaluator getEvaluator(GenericUDAFParameterInfo
paramInfo)
throws SemanticException {
TypeInfo[] parameters = paramInfo.getParameters();
if (parameters.length != 2) {
throw new UDFArgumentTypeException(parameters.length -
1,
"Exactly two arguments are expected.");
}
if (paramInfo.isAllColumns()) {
throw new UDFArgumentTypeException(parameters.length -
1,
"* is not supported");
}
if (paramInfo.isDistinct()) {
throw new UDFArgumentTypeException(parameters.length -
1,
"DISTINCT keyword is not allowed");
}
return new GenericUDAFRegrCountEvaluator();
}
public static class GenericUDAFRegrCountEvaluator extends
GenericUDAFEvaluator {
private LongObjectInspector partialCountAggOI;
private LongWritable result;
@Override
public ObjectInspector init(Mode m, ObjectInspector[]
parameters)
throws HiveException {
super.init(m, parameters);
if (m == Mode.PARTIAL2 || m == Mode.FINAL) {
partialCountAggOI = (LongObjectInspector)
parameters[0];
}
result = new LongWritable(0);
return
PrimitiveObjectInspectorFactory.writableLongObjectInspector;
}
/** class for storing count value. */
@AggregationType(estimable = true)
static class CountAgg extends AbstractAggregationBuffer {
long value;
@Override
public int estimate() {
return JavaDataModel.PRIMITIVES2;
}
}
@Override
public AggregationBuffer getNewAggregationBuffer() throws
HiveException {
CountAgg buffer = new CountAgg();
reset(buffer);
return buffer;
}
@Override
public void reset(AggregationBuffer agg) throws HiveException {
((CountAgg) agg).value = 0;
}
@Override
public void iterate(AggregationBuffer agg, Object[] parameters)
throws HiveException {
if (parameters == null) {
return;
}
if (null != parameters[0] && null != parameters[1]) {
((CountAgg) agg).value++;
}
}
@Override
public void merge(AggregationBuffer agg, Object partial) throws
HiveException {
if (partial != null) {
long p = partialCountAggOI.get(partial);
((CountAgg) agg).value += p;
}
}
@Override
public Object terminate(AggregationBuffer agg) throws
HiveException {
result.set(((CountAgg) agg).value);
return result;
}
@Override
public Object terminatePartial(AggregationBuffer agg)
throws HiveException {
return terminate(agg);
}
}
}
I have registered the function using -
CREATE FUNCTION REGR_COUNT AS
'com.practice.hive.udaf.test.GenericUDAFRegrCount' USING JAR
'hdfs:///data/test/hadoop-sample-0.1-SNAPSHOT.jar';
I get the below errors while trying to execute query -
select regr_count(empid,depid) over() from EMPLOYEE;
FAILED: SemanticException Failed to breakup Windowing invocations into Groups.
At least 1 group must only depend on input columns. Also check for circular
dependencies. Underlying error: Invalid function regr_count
Although the same function works fine when registered as temporary function
using .
CREATE TEMPORARY FUNCTION REGR_COUNT AS
'com.practice.hive.udaf.test.GenericUDAFRegrCount' USING JAR
'hdfs:///data/test/hadoop-sample-0.1-SNAPSHOT.jar';
Any help would be highly appreciated :)
Status: Patch Available (was: Open)
> As a hive user, I am facing issues using permanent UDAF's.
> ----------------------------------------------------------
>
> Key: HIVE-12719
> URL: https://issues.apache.org/jira/browse/HIVE-12719
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 1.2.1
> Reporter: Surbhit
> Assignee: Ganesha Shreedhara
> Attachments: HIVE-12719.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
