> I prefer to let the developer produce id to dedupe. I think this gives
the developer more opportunity to optimize.
Thinking about it again, I totally agree with Guowei on this. We don't
really need the framework to generate the unique id for Iceberg sink.
De-dup logic is totally internal to
The Flink code is too messy. It uses both Java and scala.
Can you unify the language and use Java only . Is it not tiring to
maintain two sets of codes
Hi, all
>>Just to add to what Aljoscha said regarding the unique id. Iceberg sink
>>checkpoints the unique id into state during snapshot. It also inserts the
>>unique id into the Iceberg snapshot metadata during commit. When a job
>>restores the state after failure, it needs to know if the
