chriszhang08 opened a new issue, #14441: URL: https://github.com/apache/iceberg/issues/14441
### Feature Request / Improvement Hello Apache Iceberg maintainers and community, I am currently using the open-source Iceberg Kafka Connector integrated with the Hadoop catalog for managing Iceberg tables. We are trying to leverage CDC workflows involving equality deletes and automatic upserts. However, we have encountered some ambiguity and inconsistent behavior regarding whether equality deletes are fully supported out-of-the-box, particularly when using the Hadoop catalog as the metadata store. Despite configuring the connector with upsert mode enabled and providing primary key columns, we observe that: Deletes in CDC events sometimes do not generate equality delete files. Metadata commits occasionally skip updates after deletes. Query engines like Dremio do not reflect the expected updates or deletes. Logs indicate successful data writes but missing or delayed metadata updates for deletes. We have reviewed documentation and community discussions indicating evolving and partial support for equality deletes across Iceberg and ecosystem components. However, it is not explicitly clear if: The open-source Iceberg Kafka Connector fully supports equality deletes in combination with the Hadoop catalog. There are any additional required configuration flags or version dependencies for proper equality delete handling. Known limitations or planned improvements exist related to equality deletes in this scenario. Could the maintainers or community kindly provide guidance or confirmation on: Does the current Iceberg Kafka Connector support writing equality deletes to Iceberg tables using the Hadoop catalog natively? Are there specific configuration requirements or recommended versions for this support? Any known limitations, workarounds, or future roadmap items related to equality deletes within this stack? Thank you very much for your help. Any pointers or documentation references would be greatly appreciated. Best regards, Chris ### Query engine None ### Willingness to contribute - [ ] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
