comien opened a new issue, #11386: URL: https://github.com/apache/seatunnel/issues/11386
### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ### Description ### Search before asking - [x] I searched the issues and found no similar issue. ### SeaTunnel Version 2.3.12 ### Connector Hudi sink ### What happened I use SeaTunnel Hudi sink to write CDC data to MinIO/S3. The Hudi data files and `.hoodie` metadata are created successfully under: ```text s3a://hudistore/warehouse/default/DEPT ``` But Hive Metastore does not contain the table. StarRocks Hudi Catalog cannot see the table because it reads metadata from HMS. I tried adding Hudi hive sync options in HOCON: ```hocon hoodie.datasource.hive_sync.enable = "true" hoodie.datasource.hive_sync.mode = "hms" hoodie.datasource.hive_sync.metastore.uris = "thrift://hive-metastore:9083" hoodie.datasource.hive_sync.database = "default" hoodie.datasource.hive_sync.table = "DEPT" hoodie.datasource.hive_sync.partition_fields = "" hoodie.datasource.hive_sync.auto_create_database = "true" hoodie.datasource.hive_sync.support_timestamp = "true" ``` The job still writes Hudi files, but HMS is not updated. ### Expected behavior One of the following: 1. Hudi sink supports Hive Metastore sync and documents the correct options. 2. Hudi sink explicitly documents that Hive Sync is not supported. 3. Unsupported `hoodie.datasource.hive_sync.*` options should produce a warning or validation error. ### Job config ```hocon sink { hudi { plugin_input = "mapped" table_dfs_path = "s3a://hudistore/warehouse" conf_files_path = "/opt/seatunnel/hudi-conf/core-site.xml;/opt/seatunnel/hudi-conf/hive-site.xml" database = "default" table_name = "DEPT" table_type = "COPY_ON_WRITE" op_type = "upsert" record_key_fields = "DEPTNO" hoodie.datasource.hive_sync.enable = "true" hoodie.datasource.hive_sync.mode = "hms" hoodie.datasource.hive_sync.metastore.uris = "thrift://hive-metastore:9083" hoodie.datasource.hive_sync.database = "default" hoodie.datasource.hive_sync.table = "DEPT" hoodie.datasource.hive_sync.partition_fields = "" hoodie.datasource.hive_sync.auto_create_database = "true" hoodie.datasource.hive_sync.support_timestamp = "true" } } ``` ### Environment - SeaTunnel Engine: Zeta - Deployment: Docker / Kubernetes - Storage: MinIO S3A - Hive Metastore: Hive 3.1.x - Query Engine: StarRocks Hudi Catalog ### Question Does SeaTunnel Hudi sink support Hive Metastore sync in 2.3.12 or later versions? If not, is the recommended production approach to run Apache Hudi HiveSyncTool as a separate post-step after the SeaTunnel Hudi job? ### Usage Scenario _No response_ ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
