Hi All, I'd like to highlight an aspect of the current Events behaviour with respect to multi-table commits that Christopher and I discovered today. The issue existed since the first Events implementation, AFAIK, it just did not come into focus until now, AFAIK.
Consider a multi-table commit request [1]. IcebergCatalogHandler will run each individual table through a separate commit operation on the base catalog [2]. The base catalog will issue events for each table separately [3][4]. However, the overall commit to Polaris Persistence may still fail, e.g. due to concurrent updates [5]. Now, we can have a situation when both before/after events are delivered for a table, but the actual change that triggered the events is _not_ persisted, therefore does not exist in the current state of the Polaris catalog. Thoughts? [1] https://github.com/apache/polaris/blob/f934443114251f85d18c9a51ed61fc49a500a61a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java#L973 [2] https://github.com/apache/polaris/blob/f934443114251f85d18c9a51ed61fc49a500a61a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java#L1051 [3] https://github.com/apache/polaris/blob/f934443114251f85d18c9a51ed61fc49a500a61a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java#L1405 [4] https://github.com/apache/polaris/blob/f934443114251f85d18c9a51ed61fc49a500a61a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalog.java#L1558 [5] https://github.com/apache/polaris/blob/f934443114251f85d18c9a51ed61fc49a500a61a/runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java#L1058 Cheers, Dmitri.
