Hi Ankit, You might want to take a look at the Iceberg Secondary Index proposal too:
- Index metadata: https://docs.google.com/document/d/1N6a2IOzC6Qsqv7NBqHKesees4N6WF49YUSIX2FrF7S0 - Key lookup index: https://docs.google.com/document/d/1e0zxK-jA0LBDq8YQlQgFipTHelDFiga8lCkgDTmYub8 Having an available key lookup index could help writing upserts immediately. We are planning to migrate Flink to the new indexes once they are available. The ongoing work could be checked here: - Proposed design: https://docs.google.com/document/d/1Jz4Fjt-6jRmwqbgHX_u0ohuyTB9ytDzfslS7lYraIjk/edit?tab=t.5qlfbj9bk8i3 - Mailing list discussion: https://lists.apache.org/thread/gldqrycmo24r0vo77wzcj9s9b04rtvy7 - First step: Flink 2.1: Add ConvertEqualityDeletes maintenance task - https://github.com/apache/iceberg/pull/15996 I hope this helps. Thanks, Peter Ankit Kumar <[email protected]> ezt írta (időpont: 2026. jún. 2., K, 15:05): > Hi all, > > I was working with CDC use cases on Iceberg and came across this doc: > https://docs.google.com/document/d/1kyyJp4masbd1FrIKUHF1ED_z1hTARL8bNoKCgb7fhSQ/edit?tab=t.0 > and thread here: > https://lists.apache.org/thread/6fhpjszsfxd8p0vfzc3k5vw7zmcyv2mq > > It was a very interesting discussion that was happening, pointing out the > pain problem. Where the real-time query and the query on the mirror table > with respect to cost were the center point of the discussion. > > Is there any conclusion that has been reached on this topic? (Some > solutions were discussed in the thread by @russel and by @Jack Ye) > > My concerns are that some catalogs do not support branching as of now, as > well as Iceberg logical views (for real-time purposes). > > > Thanks >
