Hi Abhishek, The code looks incomplete. See the comment at https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java#L22 Those are all the methods your UDAF class needs to implement but you seem to be missing them.
Mark On Sat, Feb 9, 2013 at 11:08 PM, Abhishek Bhattacharya <abhat...@fiu.edu>wrote: > Thanks for the response. > The link to the code is: > https://github.com/Abhishek2301/Hive/blob/master/src/UDAFTopNPercent.java > Please let me know to fix it! > > Thanks, > Abhishek > > > > On Fri, Feb 8, 2013 at 5:02 PM, Mark Grover > <grover.markgro...@gmail.com>wrote: > >> Abhishek, >> The code doesn't seem to be complete. >> >> Look at >> https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDAFPercentile.javafor >> reference. It has two terminate()'s - one for UDAF and one for the >> Evaluator. >> >> Do you mind posting your complete code on github somewhere so it's easier >> to analyze? >> >> Mark >> >> On Fri, Feb 8, 2013 at 2:05 PM, Abhishek Bhattacharya >> <abhat...@fiu.edu>wrote: >> >>> Hi, >>> >>> I have implemented a simple UDAF for top-n-percent as follows: >>> import java.util.ArrayList; >>> import java.util.Collections; >>> >>> import org.apache.hadoop.hive.ql.exec.UDAF; >>> import org.apache.hadoop.hive.ql.exec.UDAFEvaluator; >>> >>> public class UDAFTopNPercent extends UDAF{ >>> >>> public static class Result { >>> ArrayList<Double> list; >>> double min; >>> } >>> >>> public class TopNPercentEvaluator implements UDAFEvaluator { >>> >>> private Result res; >>> private int rowIndex; >>> private int percent; >>> >>> public TopNPercentEvaluator() { >>> super(); >>> res = new Result(); >>> init(); >>> rowIndex = 0; >>> } >>> @Override >>> public void init() { >>> res.list = new ArrayList<Double>(); >>> res.min = Double.MAX_VALUE; >>> } >>> >>> public boolean iterate(Double rowVal, int pct) { >>> ArrayList<Double> resList = res.list; >>> rowIndex++; >>> resList.add(rowVal); >>> percent = pct; >>> return true; >>> } >>> >>> public ArrayList<Double> terminatePartial() { >>> ArrayList<Double> resList = res.list; >>> Collections.sort(resList); >>> return resList; >>> } >>> >>> public boolean merge(ArrayList<Double> otherList) { >>> ArrayList<Double> resList = res.list; >>> resList.addAll(otherList); >>> return true; >>> } >>> >>> public ArrayList<Double> terminate() { >>> ArrayList<Double> resList = res.list; >>> double num_rows = (double)percent/100.0*rowIndex; >>> Collections.sort(resList); >>> int lastIdx = resList.size()- (int) num_rows; >>> if(lastIdx <= 0) { >>> return resList; >>> } >>> for(int i=0; i<lastIdx; i++) { >>> resList.remove(i); >>> } >>> return resList; >>> } >>> } >>> >>> /** >>> * @param args >>> */ >>> public static void main(String[] args) { >>> // TODO Auto-generated method stub >>> >>> } >>> >>> } >>> >>> But throws some error such as first few lines are: >>> FAILED: Hive Internal Error: >>> java.lang.ClassCastException(org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableFloatObjectInspector >>> cannot be cast to >>> org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector) >>> java.lang.ClassCastException: >>> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableFloatObjectInspector >>> cannot be cast to >>> org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector >>> at >>> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:116) >>> at >>> org.apache.hadoop.hive.ql.udf.generic.GenericUDFUtils$ConversionHelper.<init>(GenericUDFUtils.java:300) >>> at >>> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.init(GenericUDAFBridge.java:129) >>> >>> Please help me to debug this! >>> Is it throwing from returning ArrayList<Double> in terminate()? >>> How should I return a List from UDAF? >>> >>> Thanks, >>> Abhishek >>> >> >> > > > -- > Thanks and Regards, > > Abhishek Bhattacharya > PhD Computer Science > School of Computing and Information Sciences > Florida International University >
