Mayank, do you want to take a look at this one?
JVS
On May 27, 2010, at 8:29 PM, Dilip Joseph wrote:
> I am getting an exception when using the Percentile UDAF on an empty data set
> (details below)? Has anyone seen/solved this before?
>
> Thanks
> Dilip
>
> 1. Create the following table and load it with 4 rows: 10, 20, 30, 40
> CREATE TABLE pct_test (
> val INT
> );
>
> 2. SELECT PERCENTILE(val, 0.5) FROM pct_test; works fine
>
> 3. SELECT PERCENTILE(val, 0.5) FROM pct_test WHERE val > 100; fails with the
> following exception.
>
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:347)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
>
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method
> public boolean
> org.apache.hadoop.hive.ql.udf.UDAFPercentile$PercentileLongEvaluator.iterate(org.apache.hadoop.io.LongWritable,double)
> on object
> org.apache.hadoop.hive.ql.udf.udafpercentile$percentilelongevalua...@ded0f0
> of class org.apache.hadoop.hive.ql.udf.UDAFPercentile$PercentileLongEvaluator
> with arguments {null, null} of size 2
>
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:897)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:539)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:548)
>
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:548)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:548)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:548)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:548)
>
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:324)
> ... 4 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> execute method public boolean
> org.apache.hadoop.hive.ql.udf.UDAFPercentile$PercentileLongEvaluator.iterate(org.apache.hadoop.io.LongWritable,double)
> on object
> org.apache.hadoop.hive.ql.udf.udafpercentile$percentilelongevalua...@ded0f0
> of class org.apache.hadoop.hive.ql.udf.UDAFPercentile$PercentileLongEvaluator
> with arguments {null, null} of size 2
>
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:725)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.iterate(GenericUDAFBridge.java:169)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:139)
>
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:865)
> ... 11 more
> Caused by: java.lang.IllegalArgumentException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:701)
>
> ... 14 more
>
> In this simple example, percentile may not be well defined as there are no
> rows to operate on. However, the problem is that the exception also occurs
> in larger data sets where a few of the multiple maps involved may output 0
> rows.
>
>
