raphaelpfl commented on issue #17542: URL: https://github.com/apache/iceberg/issues/17542#issuecomment-5241644834
Thanks for the detailed response, and for pointing to the deprecation thread and the `ConvertEqualityDeletes` work - I hadn't connected those dots. Looked at both (the dev-list thread and PRs #17142/#17113) and agree with the split: 1. **Silent no-op**: agreed this is a real bug independent of everything else, and I'll follow up on it separately (will file it once I've scoped exactly how `IcebergSinkConfig` currently handles unrecognized properties). 2. **CDC/upsert design**: agreed - investing in equality-deletes as a durable resting state for Kafka Connect would run against where the format is heading, especially given Kafka Connect doesn't have any post-commit maintenance hook today (unlike Flink's `IcebergSink`, which is what makes the `ConvertEqualityDeletes` integration possible there). Building that hook - or deciding Kafka Connect should depend on/wait for the cross-engine index work instead - seems like exactly the kind of question that needs dev-list input, not something to decide unilaterally in this issue. I'll start a `[DISCUSS]` thread on the dev list framed against the eq-delete -> DV direction, as you suggested. Will link it here once it's up. Appreciate you taking the time to lay this out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
