Gabriel39 opened a new issue, #66516: URL: https://github.com/apache/doris/issues/66516
## Parent Tracked by #66340. The baseline Lance integration in #65730 is read-only: it supports namespace/table discovery and table description, but it does not expose Lance metadata mutation operations. ## Current status Doris currently maps the Lance Namespace SDK's read APIs as follows: - `SHOW DATABASES` uses `ListNamespaces`. - `SHOW TABLES` uses `ListTables`. - `DESC` and table loading use `DescribeTable`. `LanceExternalCatalog` does not currently provide `ExternalMetadataOps`, so namespace/table creation, schema changes, rename, and drop operations are not supported. ## Scope for Doris 4.2 - [ ] Support `CREATE DATABASE` by creating a Lance Namespace. - [ ] Support `DROP DATABASE` by dropping a Lance Namespace with well-defined non-empty and `IF EXISTS` behavior. - [ ] Support `CREATE TABLE` by creating a Lance dataset/table and registering the catalog metadata. - [ ] Support `SHOW CREATE TABLE` with an accurate Doris representation of the Lance schema and relevant table properties. - [ ] Support `ALTER TABLE` schema evolution, including add, alter/backfill, rename, and drop column operations supported by Lance. - [ ] Support `RENAME TABLE`. - [ ] Support `DROP TABLE`, with explicit semantics for deleting the underlying dataset versus deregistering only the catalog entry. - [ ] Refresh Doris metadata caches after successful mutations. - [ ] Define behavior for filesystem and REST Namespace catalogs, authorization, concurrent updates, failures, and version consistency. Row-level DML (`INSERT`, `UPDATE`, and `DELETE`) is not part of this issue. ## Completion criteria - [ ] A Lance implementation of Doris external metadata operations is integrated with FE DDL statements. - [ ] Filesystem and REST Namespace behavior is covered where the corresponding Lance implementation supports the operation. - [ ] FE unit tests and regression tests cover success, `IF EXISTS`/`IF NOT EXISTS`, conflicts, unsupported operations, and cache refresh. - [ ] User documentation describes supported syntax, catalog-specific behavior, data-deletion semantics, and limitations. - [ ] The implementation is merged into the Doris 4.2 branch and verified with representative Lance catalogs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
