gubinjie opened a new issue, #6663:
URL: https://github.com/apache/hudi/issues/6663
Flink1.13
Hudi: 0.10.1
Compile with:
mvn clean package -DskipITs -Dmaven.test.skip=true -Dhadoop.version=3.2.1
-Pflink-bundle-shade-hive3
Put the compiled hudi-flink_2.11-0.10.1-rc1.jar into ${FLINK_HOME}/lib
Startup method: ./sql-client.sh embedded -j
../lib/hudi-flink_2.11-0.10.1-rc1.jar
Create the table:
CREATE TABLE paat_hudi_flink_test(
id bigint ,
name string,
birthday TIMESTAMP(3),
tsTIMESTAMP(3),
`partition` VARCHAR(20),
primary key(id) not enforced -- the uuid primary key must be specified
)
PARTITIONED BY (`partition`)
with(
'connector'='hudi',
'path' =
'hdfs://emr-cluster/user/hive/hudi/warehouse/ods_hudi.hudi_flink_test/'
, 'hoodie.datasource.write.recordkey.field' = 'id'
, 'write.precombine.field' = 'ts'
, 'write.tasks' = '1'
, 'compaction.tasks' = '1'
, 'write.rate.limit' = '2000'
, 'table.type' = 'MERGE_ON_READ'
, 'compaction.async.enable' = 'true'
, 'compaction.trigger.strategy' = 'num_commits'
, 'compaction.max_memory' = '1024'
, 'changelog.enable' = 'true'
, 'read.streaming.enable' = 'true'
, 'read.streaming.check-interval' = '4'
, 'hive_sync.enable' = 'true'
, 'hive_sync.mode'= 'hms'
, 'hive_sync.metastore.uris' = 'thrift://***:9083'
, 'hive_sync.jdbc_url' = 'jdbc:hive2://****:10000'
, 'hive_sync.table' = 'hudi_flink_test'
, 'hive_sync.db' = 'ods_hudi'
, 'hive_sync.username' = '***'
, 'hive_sync.password' = '***^'
, 'hive_sync.support_timestamp' = 'true'
);
Query: select * from paat_hudi_flink_test;
shows that the class cannot be found:
`2022-09-13 16:24:11,026 ERROR org.apache.flink.table.factories.FactoryUtil
[] - Could not load service provider for factories.
java.util.ServiceConfigurationError:
org.apache.flink.table.factories.Factory: Provider
org.apache.hudi.table.HoodieTableFactory could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
~[?:1.8.0_231]
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
~[?:1.8.0_231]
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
~[?:1.8.0_231]
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
~[?:1.8.0_231]
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
~[?:1.8.0_231]
at java.util.Iterator.forEachRemaining(Iterator.java:116)
~[?:1.8.0_231]
at
org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:503)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:298)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.factories.FactoryUtil.getDynamicTableFactory(FactoryUtil.java:439)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.factories.FactoryUtil.createTableSource(FactoryUtil.java:133)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.plan.schema.CatalogSourceTable.createDynamicTableSource(CatalogSourceTable.java:116)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:82)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3585)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2507)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2144)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2093)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2050)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:663)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:644)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3438)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:570)
~[flink-table_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:169)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:161)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.toQueryOperation(SqlToOperationConverter.java:989)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertSqlQuery(SqlToOperationConverter.java:958)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:283)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:101)
~[flink-table-blink_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.gateway.local.LocalExecutor.lambda$parseStatement$1(LocalExecutor.java:176)
~[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.gateway.context.ExecutionContext.wrapClassLoader(ExecutionContext.java:90)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.gateway.local.LocalExecutor.parseStatement(LocalExecutor.java:176)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.parseCommand(CliClient.java:385)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.executeStatement(CliClient.java:326)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.executeInteractive(CliClient.java:297)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:221)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:151)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at org.apache.flink.table.client.SqlClient.start(SqlClient.java:95)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
Caused by: java.lang.NoClassDefFoundError:
org/apache/hudi/exception/HoodieValidationException
at java.lang.Class.getDeclaredConstructors0(Native Method)
~[?:1.8.0_231]
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
~[?:1.8.0_231]
at java.lang.Class.getConstructor0(Class.java:3075) ~[?:1.8.0_231]
at java.lang.Class.newInstance(Class.java:412) ~[?:1.8.0_231]
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
~[?:1.8.0_231]
... 35 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hudi.exception.HoodieValidationException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
~[?:1.8.0_231]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
~[?:1.8.0_231]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
~[?:1.8.0_231]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
~[?:1.8.0_231]
at java.lang.Class.getDeclaredConstructors0(Native Method)
~[?:1.8.0_231]
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
~[?:1.8.0_231]
at java.lang.Class.getConstructor0(Class.java:3075) ~[?:1.8.0_231]
at java.lang.Class.newInstance(Class.java:412) ~[?:1.8.0_231]
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
~[?:1.8.0_231]
... 35 more
2022-09-13 16:24:11,035 WARN org.apache.flink.table.client.cli.CliClient
[] - Could not execute SQL statement.
org.apache.flink.table.client.gateway.SqlExecutionException: Failed to parse
statement: select * from paat_hudi_flink_test
at
org.apache.flink.table.client.gateway.local.LocalExecutor.parseStatement(LocalExecutor.java:178)
~[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.parseCommand(CliClient.java:385)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.executeStatement(CliClient.java:326)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.executeInteractive(CliClient.java:297)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:221)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:151)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at org.apache.flink.table.client.SqlClient.start(SqlClient.java:95)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at
org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
[flink-sql-client_2.12-1.13.3.jar:1.13.3]
Caused by: org.apache.flink.table.api.ValidationException: Unable to create
a source for reading table
'default_catalog.default_database.paat_hudi_flink_test'.
Table options are:
****`
Is this what is the reason？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
