yuqi1129 opened a new issue, #11891:
URL: https://github.com/apache/gravitino/issues/11891
### Version
main (Lance catalog; also affects branch-1.3)
### Describe what's wrong
`LanceTableOperations.loadTable` repairs a table's stored schema from the
underlying Lance dataset on every load via `repairTableMetadata` /
`recordCheckedEmptyVersion`, which do an optimistic-locked `EntityStore.update`
(CAS on the entity's `currentVersion`). When two loads repair the same table
concurrently (e.g. Spark issues parallel `LOAD` during planning + execution),
the slower CAS matches zero rows and `TableMetaService.updateTable` throws
`IOException("Failed to update the entity")`. `repairTableMetadata` rethrows it
as a fatal `RuntimeException("Failed to repair table ...")` → HTTP 500, even
though the concurrent update already left the table consistent.
### Error message and/or stacktrace
```
java.lang.RuntimeException: Failed to repair table ...orders...
at
...LanceTableOperations.repairTableMetadata(LanceTableOperations.java:546)
at ...LanceTableOperations.loadTable(LanceTableOperations.java:222)
Caused by: java.io.IOException: Failed to update the entity: ...orders...
at ...TableMetaService.updateTable(TableMetaService.java:239) //
updateResult == 0 (lost CAS)
at ...RelationalEntityStore.update(RelationalEntityStore.java:160)
at
...LanceTableOperations.repairTableMetadata(LanceTableOperations.java:530)
```
### How to reproduce
Concurrently load / `SELECT` the same Lance table so two repair-on-load
paths race. Deterministic unit reproduction: stub `EntityStore.update` to throw
`IOException("Failed to update the entity")` (exactly what a lost CAS produces)
and call `loadTable` — it throws `RuntimeException("Failed to repair table")`.
Seen in CI: `LanceSparkRESTServiceIT.testSelectFromEmptyTableViaSpark` (and the
drop-table case) fail intermittently.
### Proposed fix
Wrap the repair `store.update` (in both `repairTableMetadata` and
`recordCheckedEmptyVersion`) in a bounded CAS-retry loop: on the conflict
`IOException`, re-read and re-apply the idempotent updater against the latest
version, returning a usable table instead of failing.
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
