Hi Qiegang,

Thanks for sharing this and putting together the PoC! Sorry for the
late reply. This would be a great feature to have in Polaris as we
discussed at Coc Glasgow.

I'd suggest starting with a review of the design doc on the dev list
so we can align on the scope, transaction semantics, and API changes.
Once we agree on the design, we can break the implementation into
smaller PRs. I believe this is the design doc, right?
https://docs.google.com/document/d/1ew2dOI_BHQFP1C8Q46Y_m_AvNNH4rzzLKaSsB089GTU/edit?tab=t.0

Sorry if I missed other discussions, some ASF mails have recently
landed in my spam folder.

Yufei

On Mon, Oct 19, 2026 at 12:32 PM Qiegang Long <[email protected]> wrote:
>
> Hi all,
>
> Thanks to everyone who came to the multi-table transactions talk at
> Lakehouse Day in Glasgow, and for the conversations afterwards.
> Several of you suggested continuing here. I'm new to the Polaris
> community: I built this as a proof of concept on Polaris, and this is
> the first time I'm bringing it to the list.
>
> Iceberg already has most of the foundation for transactions engines
> can use: the REST commitTransaction bundle commits several tables
> atomically, and table branches can stage writes. What's missing is the
> engine usable transaction around them: BEGIN ... COMMIT across
> statements, with one consistent read point across tables. The
> discussion of this on the Iceberg list has been quiet for a while, and
> I built the PoC to restart it with something concrete.
>
> The PoC runs Spark and Trino against a Polaris fork: multi-statement
> transactions at READ COMMITTED and REPEATABLE READ, a read point
> shared across engines, and fan-out writes from independent workers
> that commit atomically. After conversations with Iceberg folks at the
> conference, I'm moving the write path onto Iceberg table branches, so
> writes become standard REST updates; I'll follow up in this thread
> with the updated code.
>
> For example, in Spark with the Polaris plugin:
>
>     BEGIN TRANSACTION;
>     INSERT INTO customers VALUES (22, 'joe', '[email protected]');
>     INSERT INTO orders    VALUES (2002, 22, 9.99, 'pending');
>     COMMIT;   -- both tables become visible together, or neither does
>
> Readers in Spark or Trino, at either isolation level, never see the
> order without its customer.
>
> The change to the Iceberg REST surface is small: one optional
> loadTable parameter (the read point). Everything else is Polaris-only
> endpoints (minting a read point, fan-out coordination). So it could
> live in Polaris first as an opt-in extension, with an Iceberg proposal
> later if the community wants one.
>
> Would Polaris be interested in carrying it as an extension first? And
> since I'm new here, I'd appreciate guidance on the usual path for a
> change of this size: for example a design review here first, an issue
> to track it, time at a community sync, or going straight to small PRs.
> I'm happy to work whichever way suits the project. It's an early PoC
> with known gaps (listed in the design doc), and JDBC/PostgreSQL only
> for now.
>
> Talk, slides, demo video, design doc and code:
> https://github.com/qlong/lakehouse-transactions
>
> Thanks,
> Qiegang

Reply via email to