yuqi1129 commented on code in PR #11892:
URL: https://github.com/apache/gravitino/pull/11892#discussion_r3536405985
##########
catalogs/catalog-lakehouse-generic/src/main/java/org/apache/gravitino/catalog/lakehouse/lance/LanceTableOperations.java:
##########
@@ -547,13 +560,76 @@ private Table repairTableMetadata(NameIdentifier ident,
Column[] columns, long d
}
}
+ /**
+ * Applies an idempotent update to the stored table, retrying when the
optimistic-lock CAS is lost
+ * to a concurrent update. The repair-on-load path runs on every {@code
loadTable}, so concurrent
+ * loads of the same table race on the version CAS; {@code store.update}
surfaces the lost race as
+ * an {@link IOException} whose message starts with {@link
+ * TableMetaService#UPDATE_ENTITY_CONFLICT_MESSAGE_PREFIX}. Because the
updater is idempotent, the
+ * loser sleeps a short randomized backoff (to avoid re-colliding), re-reads
the latest (already
+ * repaired) entity, and retries instead of failing the whole load with a
fatal error. Other IO
+ * failures (DB outage, serialization errors, etc.) are not conflicts and
fail fast.
+ */
+ private TableEntity updateTableWithCasRetry(
+ NameIdentifier ident, Function<TableEntity, TableEntity> updater) throws
IOException {
+ IOException lastConflict = null;
+ for (int attempt = 1; attempt <= REPAIR_UPDATE_MAX_ATTEMPTS; attempt++) {
+ try {
+ return store.update(ident, TableEntity.class, Entity.EntityType.TABLE,
updater);
+ } catch (IOException e) {
+ // Only retry when the update matched 0 rows (lost optimistic-lock
CAS). Other IO failures
+ // (DB outage, serialization errors, etc.) should fail fast.
+ String message = e.getMessage();
+ if (message == null
+ ||
!message.startsWith(TableMetaService.UPDATE_ENTITY_CONFLICT_MESSAGE_PREFIX)) {
+ throw e;
+ }
+
+ lastConflict = e;
+ LOG.debug(
+ "Optimistic-lock conflict updating table {} metadata (attempt
{}/{}), {}",
+ ident,
+ attempt,
+ REPAIR_UPDATE_MAX_ATTEMPTS,
+ attempt < REPAIR_UPDATE_MAX_ATTEMPTS ? "retrying" : "retries
exhausted",
+ e);
+
+ if (attempt < REPAIR_UPDATE_MAX_ATTEMPTS) {
Review Comment:
Yeah, assuming `REPAIR_UPDATE_MAX_ATTEMPTS` is 3, we would only sleep after
the first and second failure, in the third attempt, we would not retry if it
fails again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
