ayushtkn commented on code in PR #4934:
URL: https://github.com/apache/polaris/pull/4934#discussion_r3497986380
##########
runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/IcebergCatalogHandler.java:
##########
@@ -1272,19 +1275,35 @@ public void commitTransaction(CommitTransactionRequest
commitTransactionRequest)
// Commit all accumulated changes for this table in a single atomic
operation
if (!currentMetadata.changes().isEmpty()) {
tableOps.commit(baseMetadata, currentMetadata);
+ tableFileIOs.put(tableIdentifier, tableOps.io());
}
tableMetadataObjs.add(currentMetadata);
});
- // Commit the collected updates in a single atomic operation
+ // Extract newly written metadata locations from the buffered entity
updates.
+ // We cannot use tableOps.current().metadataFileLocation() because
requestRefresh()
+ // causes doRefresh() to read from the store where the entity hasn't been
persisted yet.
+ // The pendingUpdates entities have the correct new location set by
doCommit().
List<EntityWithPath> pendingUpdates =
transactionMetaStoreManager.getPendingUpdates();
+ List<Map.Entry<FileIO, String>> writtenMetadataFiles =
Review Comment:
The list needs to be populated during the transaction loop (before we know
the outcome), because that's when `tableOps.commit()` writes the metadata
files. We can't discover which files were written after the fact. The list is
only consumed inside the failure block, but it must be built before the
`commitTransactionBatch` call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
