Dmitri, I agreed that we should disable multi-table transactions until the remote catalog(e.g., S3 tables) provides a new commit semantic to support that.
Yufei On Wed, Dec 18, 2024 at 7:50 AM Dmitri Bourlatchkov <dmitri.bourlatch...@dremio.com.invalid> wrote: > Hi All, > > This looks like a valuable feature for users. > > I wonder about multi-table commits, though. These commits are supposed to > be atomic in the Iceberg REST API, but I do not think a simple proxy for > the S3 table API can ensure atomicity. > > Does S3 tables API support multi-table changes (I did not find any)? > > That said, I may still be an option to deny multi-table commits if tables > from distinct domains are involved. IMHO, that is preferable to attempting > proxied changes that are not guaranteed to be atomic as a whole. It will > give users a solid and predictable behaviour at the REST API level. > > Cheers, > Dmitri. > > On Tue, Dec 17, 2024 at 8:00 PM Yufei Gu <flyrain...@gmail.com> wrote: > > > Hi Folks, > > > > I have been thinking about how Polaris integrates with S3 Tables. > > Basically, similar to other federations approach, Polaris could serve as > a > > REST facade for S3 tables, supporting both read and write operations by > > interacting with the S3 table API. Please check more details here, > > https://github.com/apache/polaris/issues/577. > > > > Let me know if you have any thoughts or questions on this! > > > > Yufei > > >
