liubo1022126 opened a new issue #2653: URL: https://github.com/apache/iceberg/issues/2653
Flink: **flink-1.11.1-bin-scala_2.12** Jar: ./bin/sql-client.sh embedded -j /app/flink-1.11.1/lib/**iceberg-flink-runtime-0.11.0.jar** -j /app/flink-1.11.1/**flink-sql-connector-hive-2.3.6_2.11-1.11.0.jar** shell Hive: **2.3.8** JDK: **1.8.0_202** As [Flink doc](https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/hive_functions.html#hive-user-defined-functions) say, Hive’s UDF and GenericUDF are automatically translated into Flink’s ScalarFunction. So what I do is: 1. Create hive udf in hive like. >create function udf1 as 'com.xxx.xxxx' using jar 'hdfs://icebergHadoop/user/deploy/udf/xxx.jar'; 2. Put the udf xxx.jar in flink lib path. 3. Open flink sql-client. >sql-client.sh embedded 4. Create hive catalog in flink sql-client and use it. > CREATE CATALOG myhive WITH ( > 'type' = 'hive', > 'default-database' = 'default', > 'hive-conf-dir' = '/app/hive/conf' > ); > > use catalog myhive; 5. show functions and use udf1 like . >show functions; > >select udf1('a', 'b'); It works Ok. But when I use iceberg hive catalog As [iceberg hive catalog Doc](https://iceberg.apache.org/flink/#hive-catalog), I can not use hive udf. As what I do is like: 1.Open flink sql-client with iceberg. > export HADOOP_CLASSPATH=`$HADOOP_HOME/bin/hadoop classpath` > > ./bin/sql-client.sh embedded \ > -j <flink-runtime-directory>/iceberg-flink-runtime-xxx.jar \ > -j <hive-bundlded-jar-directory>/flink-sql-connector-hive-2.3.6_2.11-1.11.0.jar \ > shell 2.Create iceberg hive catalog in flink sql-client and use it. > CREATE CATALOG hive_catalog WITH ( > 'type'='iceberg', > 'catalog-type'='hive', > 'uri'='thrift://localhost:9083', > 'clients'='5', > 'property-version'='1', > 'warehouse'='hdfs://nn:8020/warehouse/path' > ); 3.show functions and use udf1 like **(What is not works Ok, I can not see the hive udf which is registered in hive metastore)** >show functions; So how can I use hive udf on iceberg table in Flink, Thx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
