Hi, Jingsong Li 你好，非常感谢你的解答。问题已经得到处理。
类似写法之前有尝试过，但是没有注意到与hive的差异，包括 T() 这些，所以也是失败的。加上官方文档没有仔细理解。 再次感谢答疑 On Thu, Apr 2, 2020 at 11:27 AM Jingsong Li <[email protected]> wrote: > Hi, > > GenericUDTFExplode是一个UDTF。 > Flink中使用UDTF的方式是标准SQL的方式： > "select x from db1.nested, lateral table(explode(a)) as T(x)" > > 你试下。 > > [1] > > https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/udfs.html#table-functions > > Best, > Jingsong Lee > > On Thu, Apr 2, 2020 at 11:22 AM Yaoting Gong <[email protected]> > wrote: > > > 大家后， > > > > 我们项目目前 集成了HiveModule，遇到一些问题。请教下大家。 > > > > 在集成 Hive Module 之前，substr，split 都是无法使用的。集成后，验证都是可以的。 > > 比如：select split('1,2,2,4',',') > > > > 但是在使用 explode 函数，select explode(split('1,2,2,4',',')); > > 有如下错误： > > > > > > The main method caused an error: SQL validation failed. From line 1, > column > > 8 to line 1, column 36: *No match found for function signature explode(* > > *) * > > > > *完整堆栈：* > > The program finished with the following exception: > > > > org.apache.flink.client.program.ProgramInvocationException: The main > method > > caused an error: SQL validation failed. From line 1, column 8 to line 1, > > column 36: No match found for function signature explode() > > at > > > > > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335) > > at > > > > > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205) > > at > org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138) > > at > > > > > org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664) > > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213) > > at > > > > > org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895) > > at > > > > > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968) > > at > > > org.apache.flink.client.cli.CliFrontend$$Lambda$38/1205406622.call(Unknown > > Source) > > at > > > > > org.apache.flink.runtime.security.HadoopSecurityContext$$Lambda$39/293907205.run(Unknown > > Source) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:422) > > at > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692) > > at > > > > > org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) > > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968) > > Caused by: org.apache.flink.table.api.ValidationException: SQL validation > > failed. From line 1, column 8 to line 1, column 36: No match found for > > function signature explode() > > at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org > > > > > $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130) > > at > > > > > org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105) > > at > > > > > org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127) > > at > > > > > org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66) > > at > > > > > org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlQuery(TableEnvironmentImpl.java:464) > > at > com.sui.bigdata.PlatformEngine$.$anonfun$main$4(PlatformEngine.scala:88) > > at > > > > > com.sui.bigdata.PlatformEngine$.$anonfun$main$4$adapted(PlatformEngine.scala:87) > > at com.sui.bigdata.PlatformEngine$$$Lambda$765/623545006.apply(Unknown > > Source) > > at scala.collection.immutable.List.foreach(List.scala:388) > > at com.sui.bigdata.PlatformEngine$.main(PlatformEngine.scala:87) > > at com.sui.bigdata.PlatformEngine.main(PlatformEngine.scala) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:497) > > at > > > > > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321) > > ... 13 more > > Caused by: org.apache.calcite.runtime.CalciteContextException: From line > 1, > > column 8 to line 1, column 36: No match found for function signature > > explode() > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > > at > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > > at > > > org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) > > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834) > > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4841) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1773) > > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:293) > > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587) > > at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:479) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4105) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3389) > > at > > > > > org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) > > at > > > > > org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1008) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:968) > > at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:943) > > at > > > > > org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:650) > > at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org > > > > > $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:126) > > ... 28 more > > Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No > match > > found for function signature explode() > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > > at > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > > at > > > org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) > > at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) > > ... 50 more > > > > > > 简单看了 Hive Functions 加载函数代码， > > 1、都进行了加载 > > system.registerGenericUDF("split", GenericUDFSplit.class); > > system.registerGenericUDTF("explode", GenericUDTFExplode.class); > > > > 2、FunctionInfo 返回是否 builtIn 逻辑也是一致的 > > this.functionType = isNative ? FunctionType.BUILTIN : > > FunctionType.TEMPORARY;， 其中 isNative 是单例对象的属性， > > // registry for system functions > > private static final Registry system = new Registry(true); // > isNative > > === true > > > > 3、不在 Hive Module 黑名单中，附黑名单列表： > > "dense_rank", > > "first_value", > > "lag", > > "last_value", > > "lead", > > "rank", > > "row_number", > > "hop", > > "hop_end", > > "hop_proctime", > > "hop_rowtime", > > "hop_start", > > "session", > > "session_end", > > "session_proctime", > > "session_rowtime", > > "session_start", > > "tumble", > > "tumble_end", > > "tumble_proctime", > > "tumble_rowtime", > > "tumble_start" > > > > 不知道是不是还有什么地方使用方式不对？ > > > > 谢谢解答。 > > > > > -- > Best, Jingsong Lee >
