Hi, GenericUDTFExplode是一个UDTF。 Flink中使用UDTF的方式是标准SQL的方式： "select x from db1.nested, lateral table(explode(a)) as T(x)"
你试下。 [1] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/udfs.html#table-functions Best, Jingsong Lee On Thu, Apr 2, 2020 at 11:22 AM Yaoting Gong <[email protected]> wrote: > 大家后， > > 我们项目目前 集成了HiveModule，遇到一些问题。请教下大家。 > > 在集成 Hive Module 之前，substr，split 都是无法使用的。集成后，验证都是可以的。 > 比如：select split('1,2,2,4',',') > > 但是在使用 explode 函数，select explode(split('1,2,2,4',',')); > 有如下错误： > > > The main method caused an error: SQL validation failed. From line 1, column > 8 to line 1, column 36: *No match found for function signature explode(* > *) * > > *完整堆栈：* > The program finished with the following exception: > > org.apache.flink.client.program.ProgramInvocationException: The main method > caused an error: SQL validation failed. From line 1, column 8 to line 1, > column 36: No match found for function signature explode() > at > > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335) > at > > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205) > at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138) > at > > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664) > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213) > at > > org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895) > at > > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968) > at > org.apache.flink.client.cli.CliFrontend$$Lambda$38/1205406622.call(Unknown > Source) > at > > org.apache.flink.runtime.security.HadoopSecurityContext$$Lambda$39/293907205.run(Unknown > Source) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692) > at > > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968) > Caused by: org.apache.flink.table.api.ValidationException: SQL validation > failed. From line 1, column 8 to line 1, column 36: No match found for > function signature explode() > at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org > > $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130) > at > > org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105) > at > > org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127) > at > > org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66) > at > > org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlQuery(TableEnvironmentImpl.java:464) > at com.sui.bigdata.PlatformEngine$.$anonfun$main$4(PlatformEngine.scala:88) > at > > com.sui.bigdata.PlatformEngine$.$anonfun$main$4$adapted(PlatformEngine.scala:87) > at com.sui.bigdata.PlatformEngine$$$Lambda$765/623545006.apply(Unknown > Source) > at scala.collection.immutable.List.foreach(List.scala:388) > at com.sui.bigdata.PlatformEngine$.main(PlatformEngine.scala:87) > at com.sui.bigdata.PlatformEngine.main(PlatformEngine.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321) > ... 13 more > Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, > column 8 to line 1, column 36: No match found for function signature > explode() > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at > org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834) > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4841) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1773) > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:293) > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587) > at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:479) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4105) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3389) > at > > org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) > at > > org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1008) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:968) > at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:943) > at > > org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:650) > at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org > > $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:126) > ... 28 more > Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match > found for function signature explode() > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > at > org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) > at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) > ... 50 more > > > 简单看了 Hive Functions 加载函数代码， > 1、都进行了加载 > system.registerGenericUDF("split", GenericUDFSplit.class); > system.registerGenericUDTF("explode", GenericUDTFExplode.class); > > 2、FunctionInfo 返回是否 builtIn 逻辑也是一致的 > this.functionType = isNative ? FunctionType.BUILTIN : > FunctionType.TEMPORARY;， 其中 isNative 是单例对象的属性， > // registry for system functions > private static final Registry system = new Registry(true); // isNative > === true > > 3、不在 Hive Module 黑名单中，附黑名单列表： > "dense_rank", > "first_value", > "lag", > "last_value", > "lead", > "rank", > "row_number", > "hop", > "hop_end", > "hop_proctime", > "hop_rowtime", > "hop_start", > "session", > "session_end", > "session_proctime", > "session_rowtime", > "session_start", > "tumble", > "tumble_end", > "tumble_proctime", > "tumble_rowtime", > "tumble_start" > > 不知道是不是还有什么地方使用方式不对？ > > 谢谢解答。 > -- Best, Jingsong Lee
