Hi everyone, Thank you for feedback on the MTT proposal and during community sync. Based on it, Jagdeep and I have iterated on the document and added a second option to use *Catalog CommitSequenceNumbers*. Looking forward to getting more feedback on the proposal, where to add more details or approach/changes to consider. We appreciate everyone's time on this!
The option introduces *Catalog CommitSequenceNumbers(CSNs)*, which allow clients/engines to read a consistent view of multiple tables without needing to register a transaction context with the catalog. This removes the need of registering a transaction context with Catalog, thus removing the need of transaction bookkeeping on the catalog side. For aborting transactions early, clients can use LoadTable with and without CSN to figure out if there is already a conflicting write on any of the tables being modified. Also removed the section where transactions were staging commits on Catalog, and changed the proposal to align with Eduard's PR around staging changes locally before commit ( https://github.com/apache/iceberg/pull/6948). Jagdeep also clarified in an example in a previous email where a workload may require multi table snapshot isolation, even if the tables are being updated without Multi-Table commit API. Though most MTT transactions will commit using the multi table commit API. Maninder, for the approach of "common notion of time between clients and catalog" - I spent some time thinking about it, but cannot find a feasible way to do this. Yes, the catalogs can use a high precision clock, but clients cannot use Catalog Timestamp from API calls to set local clock due to network latency for request/response. For example, different requests to the same Catalog servers can return different timestamps based on network latency. Also what if a client works with more than 1 Catalog. If you want to do a rough write-up or share a reference implementation that uses such an approach, I will be happy to brainstorm it more. Let us know! Here is the link to updated proposal <https://docs.google.com/document/d/1jr4Ah8oceOmo6fwxG_0II4vKDUHUKScb/edit?usp=sharing&ouid=100384647237395649950&rtpof=true&sd=true> Thanks Again! - Drew
