MohamedAdelHsn opened a new issue, #7220: URL: https://github.com/apache/iceberg/issues/7220
### Query engine **Flink** ### Question i want to load debezium CDC data from kafka to iceberg which stored in HDFS via flink sql job i follow the official iceberg documents that say below : **Iceberg supports UPSERT based on the primary key when writing data into v2 table format. There are two ways to enable upsert. Enable the UPSERT mode as table-level property write.upsert.enabled ..** when i execute insert ,update, delete in mysql i can see data in synced and changed in hive same with mysql. Please your quick support here and it is really appreciated.... My questions here is: 1. I have created source table for kafka and sink table as iceberg in flink sql my question here is where data stored in flink tables is in in memory or where ? 2. Second if i have 1millon rows with new 1 updated row Is Flink icenerg witer will write all 1million record to iceberg hdfs for the one updated row? 3. is this solution can fit in production or not databases -> CDC [Debezium] -> Kafka -> FLINk Sql source job per each topic -> Flink Sql job that insert * from iceberg_tabl select * from kafka table; -> HDFS -> Iceberg -> HIVE . Thanks for your support -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
