bithw1 opened a new issue, #12013:
URL: https://github.com/apache/hudi/issues/12013
I am using Hudi 0.15.0 and Flink 1.17.1, following are the steps to
reproduce the problem:
From the flink-sql cli: do the following sql statements
```
CREATE CATALOG hudi_catalog WITH (
'type' = 'hudi',
'mode' = 'hms',
'default-database' = 'default',
'hive.conf.dir' = '/home/hadoop/software/hive-3.1.3/conf',
'table.external' = 'true'
);
create database if not exists hudi_catalog.`default`;
use hudi_catalog.`default`;
CREATE TABLE test_hudi_flink_mor_2 (
a int PRIMARY KEY NOT ENFORCED,
b int,
c int
)
WITH (
'connector' = 'hudi',
'path' = '/tmp/test_hudi_flink_mor_2',
'table.type' = 'MERGE_ON_READ',
'hoodie.datasource.write.keygenerator.class' =
'org.apache.hudi.keygen.ComplexAvroKeyGenerator', --- Using
ComplexAvroKeyGenerator
'hoodie.datasource.write.recordkey.field' = 'a',
'write.precombine.key'='b',
'hoodie.datasource.write.hive_style_partitioning' = 'true',
'hive_sync.conf.dir'='/home/hadoop/software/hive-3.1.3/conf'
);
insert into test_hudi_flink_mor_2 values (1,1,1),(2,2,2);
```
So far so good, then I run `insert into test_hudi_flink_mor_2 select 3,3,
3` on the spark-sql cli, an error occurred, key exception message is as follows:
```
org.apache.hudi.exception.HoodieException: Config conflict(key current
value existing value):
KeyGenerator: org.apache.hudi.keygen.ComplexAvroKeyGenerator
org.apache.hudi.keygen.NonpartitionedAvroKeyGenerator
at
org.apache.hudi.HoodieWriterUtils$.validateTableConfig(HoodieWriterUtils.scala:229)
at
org.apache.hudi.HoodieSparkSqlWriterInternal.writeInternal(HoodieSparkSqlWriter.scala:232)
at
org.apache.hudi.HoodieSparkSqlWriterInternal.write(HoodieSparkSqlWriter.scala:187)
```
When I run `show create table test_hudi_flink_mor_2` on spark-sql cli, it
shows that keygenerator is using ComplexAvroKeyGenerator,but when i look at the
`/tmp/test_hudi_flink_mor_2/.hoodie/hoodie.properties`,
`hoodie.table.keygenerator.class=org.apache.hudi.keygen.NonpartitionedAvroKeyGenerator`,
per the exception, it looks to me that there is a bug here...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
