youngyjd opened a new pull request, #9196:
URL: https://github.com/apache/gravitino/pull/9196
### What changes were proposed in this pull request?
This PR refactored and polished the generic lakehouse table implementation
by:
- Consolidated table entity model: Deleted GenericTableEntity and merged its
functionality into TableEntity, adding fields for generic lakehouse table
properties (format, properties, partitions, sortOrder, distribution, indexes,
comment).
- Implemented table CRUD operations: Added complete implementation for
loadTable(), createTable(), alterTable(), and dropTable() operations in
GenericLakehouseCatalogOperations that were previously marked as TODOs.
- Improved storage layer:
- Enhanced POConverters to properly serialize/deserialize TableEntity
fields
- Updated TableMetaService to handle table version metadata during alter
operations
- Modified database schema to make format column nullable and add unique
constraint uk_table_id_version_deleted_at in table_version_info table
- Added utility classes: Created EntityConverter to handle conversion
between entity types and catalog types, reducing code duplication.
- Polished Lance catalog operations: Refactored LanceCatalogOperations with
better index handling, data type conversion, and table metadata management.
- Added comprehensive tests: Created TestPropertiesMetadata,
TestEntityConverter, and enhanced existing tests in TestJDBCBackend and
TestEntityStorage to validate the new table operation
### Why are the changes needed?
This PR addresses unresolved review comments from PR
https://github.com/apache/gravitino/pull/9181 that added Lance table
operations. The refactoring:
- Eliminates code duplication between different table entity types
- Simplifies the table entity model by consolidating into a single
TableEntity class
- Improves maintainability and extensibility for supporting additional
lakehouse formats
- Enables proper storage and retrieval of generic lakehouse table metadata
(format, partitions, distribution, indexes, etc.)
Fix: #8915
### Does this PR introduce _any_ user-facing change?
No. This is an internal refactoring that improves code structure and
completes the Lance table operation implementation. The API remains unchanged.
### How was this patch tested?
Unit test and IT test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
