[ https://issues.apache.org/jira/browse/HIVEMALL-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takuya Kitazawa updated HIVEMALL-157: ------------------------------------- Description: Consider the following query: {code:sql} SELECT to_ordered_list(null, null) {code} Here, even though {{to_ordered_list()}} [automatically ignores NULL key and/or value|https://github.com/apache/incubator-hivemall/blob/c2b95783cf9d6fc1646a48ac928e96152eab98c6/core/src/main/java/hivemall/tools/list/UDAFToOrderedList.java#L283-L306], the query throws NPE because of [NULL queue handler access in {{drainQueue()}}|https://github.com/apache/incubator-hivemall/blob/c2b95783cf9d6fc1646a48ac928e96152eab98c6/core/src/main/java/hivemall/tools/list/UDAFToOrderedList.java#L409]: {code} org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1102) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:683) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1060) at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1099) ... 14 more Caused by: java.lang.NullPointerException at hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator$QueueAggregationBuffer.drainQueue(UDAFToOrderedList.java:411) at hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator.terminatePartial(UDAFToOrderedList.java:313) at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.evaluate(GenericUDAFEvaluator.java:204) at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1020) at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1043) ... 15 more {code} Since {{iterate()}} accepts NULL inputs, {{drainQueue()}} should not fail; it has to return something like empty list. was: Consider the following query: {code:sql} SELECT to_ordered_list(null, null) {code} Here, even though {{to_ordered_list()}} [automatically ignores NULL key and/or value|https://github.com/apache/incubator-hivemall/blob/c2b95783cf9d6fc1646a48ac928e96152eab98c6/core/src/main/java/hivemall/tools/list/UDAFToOrderedList.java#L283-L306], the query throws NPE because of [NULL query handler access in {{drainQueue()}}|https://github.com/apache/incubator-hivemall/blob/c2b95783cf9d6fc1646a48ac928e96152eab98c6/core/src/main/java/hivemall/tools/list/UDAFToOrderedList.java#L409]: {code} org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1102) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:683) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1060) at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1099) ... 14 more Caused by: java.lang.NullPointerException at hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator$QueueAggregationBuffer.drainQueue(UDAFToOrderedList.java:411) at hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator.terminatePartial(UDAFToOrderedList.java:313) at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.evaluate(GenericUDAFEvaluator.java:204) at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1020) at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1043) ... 15 more {code} Since {{iterate()}} accepts NULL inputs, {{drainQueue()}} should not fail; it has to return something like empty list. > Null Pointer Exception in to_ordered_list UDAF for empty (partial) result > ------------------------------------------------------------------------- > > Key: HIVEMALL-157 > URL: https://issues.apache.org/jira/browse/HIVEMALL-157 > Project: Hivemall > Issue Type: Bug > Reporter: Takuya Kitazawa > Assignee: Takuya Kitazawa > > Consider the following query: > {code:sql} > SELECT to_ordered_list(null, null) > {code} > Here, even though {{to_ordered_list()}} [automatically ignores NULL key > and/or > value|https://github.com/apache/incubator-hivemall/blob/c2b95783cf9d6fc1646a48ac928e96152eab98c6/core/src/main/java/hivemall/tools/list/UDAFToOrderedList.java#L283-L306], > the query throws NPE because of [NULL queue handler access in > {{drainQueue()}}|https://github.com/apache/incubator-hivemall/blob/c2b95783cf9d6fc1646a48ac928e96152eab98c6/core/src/main/java/hivemall/tools/list/UDAFToOrderedList.java#L409]: > {code} > org.apache.hadoop.hive.ql.metadata.HiveException: > org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1102) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:683) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) > at > org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at > org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1060) > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1099) > ... 14 more > Caused by: java.lang.NullPointerException > at > hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator$QueueAggregationBuffer.drainQueue(UDAFToOrderedList.java:411) > at > hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator.terminatePartial(UDAFToOrderedList.java:313) > at > org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.evaluate(GenericUDAFEvaluator.java:204) > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1020) > at > org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1043) > ... 15 more > {code} > Since {{iterate()}} accepts NULL inputs, {{drainQueue()}} should not fail; it > has to return something like empty list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)