mengxianwen1 opened a new pull request, #11240: URL: https://github.com/apache/gravitino/pull/11240
### What changes were proposed in this pull request?
Add a `loadTable` self-healing mechanism in `LanceTableOperations` to fix
empty schema for Lance tables created via Spark's `StagedTableCatalog` path.
Changes:
- Override `loadTable` in `LanceTableOperations` to detect
`lance.create-empty=true`, read actual schema from the underlying Lance
dataset, and update Gravitino entity store with the real columns.
- Change `lance.create-empty` property from immutable to mutable in
`LanceTableDelegator` to allow the heal process to clear the flag.
- Change `toGenericTable` and `toColumnEntities` visibility from `private`
to `protected` in `ManagedTableOperations` for subclass access.
### Why are the changes needed?
When Spark creates Lance tables through `lance-rest-server` using the
`StagedTableCatalog` path (`stageCreate` → `declareTable(createEmpty=true)` →
`StagedCommit`), Gravitino only stores empty column metadata. After
`StagedCommit` writes
actual data to storage, the schema is never synced back to Gravitino. This
results in Gravitino Web UI and API showing empty columns while the actual
Lance dataset has the real schema.
Fix: #11027
### Does this PR introduce _any_ user-facing change?
No user-facing API changes. After this fix, users will see correct column
information in Gravitino Web UI and API for Lance tables created via Spark's
`StagedTableCatalog` path.
### How was this patch tested?
1. Updated existing integration test `testCrateEmptyTable` in
`CatalogGenericCatalogLanceIT` to verify the `lance.create-empty` property is
now mutable.
2. All unit tests pass:
- `./gradlew :core:test -PskipITs`
- `./gradlew :catalogs:catalog-lakehouse-generic:test -PskipITs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
