ayushtkn commented on code in PR #4939:
URL: https://github.com/apache/polaris/pull/4939#discussion_r3575719158
##########
polaris-core/src/main/java/org/apache/polaris/core/persistence/PolarisMetaStoreManager.java:
##########
@@ -284,6 +284,30 @@ default BaseResult bootstrapPolarisService(@NonNull
PolarisCallContext callCtx)
@NonNull EntitiesResult updateEntitiesPropertiesIfNotChanged(
@NonNull PolarisCallContext callCtx, @NonNull List<EntityWithPath>
entities);
+ /**
+ * Commits a batch of entity creations and property updates within a single
transaction.
+ *
+ * @param callCtx call context
+ * @param creates entities to create
+ * @param updates entities to update (compare-and-swap)
+ * @return result indicating success or failure
+ */
+ default @NonNull EntitiesResult commitTransactionBatch(
+ @NonNull PolarisCallContext callCtx,
+ @NonNull List<EntityWithPath> creates,
+ @NonNull List<EntityWithPath> updates) {
+ for (EntityWithPath create : creates) {
+ EntityResult result = createEntityIfNotExists(callCtx,
create.catalogPath(), create.entity());
Review Comment:
Thanks for pointing me to the mailing list discussions. I read through the
threads (including `Appendix A/B `and Dennis's Postgres notes) and think I have
some context now.
**My understanding of the ask:** Move away from the
`TransactionWorkspaceMetaStoreManager` pattern (intercepting individual calls,
returning success before persistence, batching at the end). Instead, use a
first-class `Session/ChangeSet` API where batching is the foundational
contract, conflict detection is scoped to an explicit commit boundary, and
backend isolation behavior is well-defined.
**What this PR does toward that:**
Introduced `MetaStoreChangeSet` as a first-class batch payload (instead of
raw create/update lists).
`commitTransaction` now builds an explicit `MetaStoreChangeSet` and commits
via `commitTransactionBatch(changeSet`).
**What is still interim:**
The API is not fully inverted yet —`commitTransactionBatch`(default) still
calls the individual abstract methods, and backends are still built around
single-entity ops.ofCreate/of Update are there so single ops can later ride on
batch commit, but that wiring is follow-up.
`TransactionWorkspaceMetaStoreManager` remains in `IcebergCatalogHandler` as
a temporary bridge to run Iceberg table ops and collect pending mutations
before assembling the final change-set.
Does this `MetaStoreChangeSet `pivot provide a good enough architectural
stepping stone to merge this and unblock the Iceberg `commitTransaction`
endpoint today? Or do you strongly feel we need to rip out the
`TransactionWorkspaceMetaStoreManager` and build the full `Session` API within
this PR itself?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
