[
https://issues.apache.org/jira/browse/FLINK-13869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-13869:
----------------------------
Description:
In flink, specifying the StreamTableEnvironment through the EnvironmentSetting
using the blink planner, when using the UDAF in hive in the table API, the
error is reported.
The hive function should been make by correct constants and argTypes. Otherwise
it will throw an exception. (See HiveAggSqlFunction)
In this isTableAggregate, it just want to check the aggregate function class
type, so the better way is get the function instead of make a function.
{code:java}
Caused by: java.lang.NullPointerException
at java.util.Arrays.stream(Arrays.java:5004)
at java.util.stream.Stream.of(Stream.java:1000)
at
org.apache.flink.table.types.utils.TypeConversions.fromLogicalToDataType(TypeConversions.java:67)
at
org.apache.flink.table.planner.functions.utils.HiveFunctionUtils.invokeSetArgs(HiveFunctionUtils.java:59)
at
org.apache.flink.table.planner.functions.utils.HiveAggSqlFunction.makeFunction(HiveAggSqlFunction.java:68)
at
org.apache.flink.table.planner.functions.utils.HiveAggSqlFunction.makeFunction(HiveAggSqlFunction.java:47)
at
org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$isTableAggregate$2.apply(AggregateUtil.scala:750)
at
org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$isTableAggregate$2.apply(AggregateUtil.scala:750)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at
org.apache.flink.table.planner.plan.utils.AggregateUtil$.isTableAggregate(AggregateUtil.scala:750)
at
org.apache.flink.table.planner.plan.utils.RelExplainUtil$.streamGroupAggregationToString(RelExplainUtil.scala:346)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecGroupAggregate.explainTerms(StreamExecGroupAggregate.scala:109)
at
org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:307)
at
org.apache.calcite.rel.AbstractRelNode.computeDigest(AbstractRelNode.java:388)
at
org.apache.calcite.rel.AbstractRelNode.recomputeDigest(AbstractRelNode.java:351)
at
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:345)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1668)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:846)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:868)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1939)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:129)
... 60 more
{code}
was:
In flink, specifying the StreamTableEnvironment through the EnvironmentSetting
using the blink planner, when using the UDAF in hive in the table API, the
error is reported.
The hive function should been make by correct constants and argTypes. Otherwise
it will throw an exception. (See HiveAggSqlFunction)
In this isTableAggregate, it just want to check the aggregate function class
type, so the better way is get the function instead of make a function.
Caused by: java.lang.NullPointerExceptionCaused by:
java.lang.NullPointerException at java.util.Arrays.stream(Arrays.java:5004) at
java.util.stream.Stream.of(Stream.java:1000) at
org.apache.flink.table.types.utils.TypeConversions.fromLogicalToDataType(TypeConversions.java:67)
at
org.apache.flink.table.planner.functions.utils.HiveFunctionUtils.invokeSetArgs(HiveFunctionUtils.java:59)
at
org.apache.flink.table.planner.functions.utils.HiveAggSqlFunction.makeFunction(HiveAggSqlFunction.java:68)
at
org.apache.flink.table.planner.functions.utils.HiveAggSqlFunction.makeFunction(HiveAggSqlFunction.java:47)
at
org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$isTableAggregate$2.apply(AggregateUtil.scala:750)
at
org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$isTableAggregate$2.apply(AggregateUtil.scala:750)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at
scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
scala.collection.AbstractTraversable.map(Traversable.scala:104) at
org.apache.flink.table.planner.plan.utils.AggregateUtil$.isTableAggregate(AggregateUtil.scala:750)
at
org.apache.flink.table.planner.plan.utils.RelExplainUtil$.streamGroupAggregationToString(RelExplainUtil.scala:346)
at
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecGroupAggregate.explainTerms(StreamExecGroupAggregate.scala:109)
> Hive functions can not work in blink planner stream mode
> --------------------------------------------------------
>
> Key: FLINK-13869
> URL: https://issues.apache.org/jira/browse/FLINK-13869
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive, Table SQL / Planner
> Affects Versions: 1.9.0
> Reporter: Terry Wang
> Assignee: Jingsong Lee
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Attachments: image-2019-08-27-15-36-57-662.png,
> image-2019-08-27-15-37-11-230.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In flink, specifying the StreamTableEnvironment through the
> EnvironmentSetting using the blink planner, when using the UDAF in hive in
> the table API, the error is reported.
> The hive function should been make by correct constants and argTypes.
> Otherwise it will throw an exception. (See HiveAggSqlFunction)
> In this isTableAggregate, it just want to check the aggregate function class
> type, so the better way is get the function instead of make a function.
> {code:java}
> Caused by: java.lang.NullPointerException
> at java.util.Arrays.stream(Arrays.java:5004)
> at java.util.stream.Stream.of(Stream.java:1000)
> at
> org.apache.flink.table.types.utils.TypeConversions.fromLogicalToDataType(TypeConversions.java:67)
> at
> org.apache.flink.table.planner.functions.utils.HiveFunctionUtils.invokeSetArgs(HiveFunctionUtils.java:59)
> at
> org.apache.flink.table.planner.functions.utils.HiveAggSqlFunction.makeFunction(HiveAggSqlFunction.java:68)
> at
> org.apache.flink.table.planner.functions.utils.HiveAggSqlFunction.makeFunction(HiveAggSqlFunction.java:47)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$isTableAggregate$2.apply(AggregateUtil.scala:750)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$$anonfun$isTableAggregate$2.apply(AggregateUtil.scala:750)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> at
> org.apache.flink.table.planner.plan.utils.AggregateUtil$.isTableAggregate(AggregateUtil.scala:750)
> at
> org.apache.flink.table.planner.plan.utils.RelExplainUtil$.streamGroupAggregationToString(RelExplainUtil.scala:346)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecGroupAggregate.explainTerms(StreamExecGroupAggregate.scala:109)
> at
> org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:307)
> at
> org.apache.calcite.rel.AbstractRelNode.computeDigest(AbstractRelNode.java:388)
> at
> org.apache.calcite.rel.AbstractRelNode.recomputeDigest(AbstractRelNode.java:351)
> at
> org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:345)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1668)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:846)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:868)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1939)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:129)
> ... 60 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
