Nethaje opened a new issue, #9208:
URL: https://github.com/apache/seatunnel/issues/9208

   ### Search before asking
   
   - [x] I had searched in the 
[issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22)
 and found no similar issues.
   
   
   ### What happened
   
   I am running a MySQL CDC to Apache Iceberg sink. All operations like insert, 
delete, and update are reflecting correctly. However, schema evolution is not 
being handled. There are no errors, but when I delete or alter columns in the 
source MySQL table, the metadata changes are not reflected in Apache Iceberg in 
S3.
   
   ### SeaTunnel Version
   
   2.3.9
   
   ### SeaTunnel Config
   
   ```conf
   env {
     # You can set engine configuration here
     execution.parallelism = 1
     job.mode = "STREAMING"
     checkpoint.interval = 5000
     read_limit.bytes_per_second=7000000
     read_limit.rows_per_second=400
   }
   
   source {
     MySQL-CDC {
       server-id = 5652
       base-url = "REMOVING FOR PRIVACY CONCERN"
       username = "admin"
       password = "REMOVING FOR PRIVACY CONCERN"
       table-names = ["hep_test_db.employee_cdc"]
       startup.mode = "initial"
       schema-changes.enabled = true
     }
   }
   
   sink {
     Iceberg {
       catalog_name="hadoop_Iceberg"
       iceberg.catalog.config={
         type = "hadoop"
         fs.s3a.endpoint = "REMOVING FOR PRIVACY CONCERN"
         warehouse = "s3a://data-platform-hfn//Raw/"   
       }
       namespace="hep_test_db"
       table="employee_cdc"
       iceberg.table.write-props={
         write.format.default="parquet"
         write.target-file-size-bytes=536870912
       }
       iceberg.table.primary-keys="id"
       #iceberg.table.partition-keys="state"
       iceberg.table.upsert-mode-enabled=true
       iceberg.table.schema-evolution-enabled=true
     }
   }
   ```
   
   ### Running Command
   
   ```shell
   cd to apache_seatunnel path
   ./bin/seatunnel.sh --config ./config/Final_Config/final_mysql_cdc -m local
   ```
   
   ### Error Exception
   
   ```log
   No error but my concern is that when updating records, the table name in the 
logs appears as `hadoop_iceberg.hep_test_db.employee_cdc`, which is expected. 
However, when running schema evolution, the table name in the logs is prefixed 
with `MySQL` or starts with `.database.tablename`, as highlighted in the 
screenshot.
   ```
   
   ### Zeta or Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   _No response_
   
   ### Screenshots
   
   
![Image](https://github.com/user-attachments/assets/bb15f4be-94ce-47aa-91d1-86054e50a90a)
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to