wangzhaobo957-cloud opened a new pull request, #68349:
URL: https://github.com/apache/doris/pull/68349
### What problem does this PR solve?
Issue Number: close #66516
Related PR: N/A
Problem Summary:
Lance catalogs are currently read-only from Doris. This PR adds DDL support
backed by the Lance Namespace API for both filesystem and REST catalogs.
- Add create/drop namespace operations, including FORCE semantics and
root-namespace protection.
- Add create/drop/rename table operations and ENGINE=LANCE / SHOW CREATE
TABLE support.
- Convert Doris schemas to Arrow and create empty Lance datasets through
Arrow IPC.
- Add top-level ADD, MODIFY, RENAME, and DROP COLUMN schema evolution.
- Refresh Doris metadata, schema caches, and the upstream Lance table-access
cache after successful mutations.
- Reject unsupported defaults, positions, comments, complex type changes,
nested paths, and multi-clause schema changes explicitly.
The current Lance Java SDK does not implement explicit
backfill/virtual-column operations for DirectoryNamespace or RestNamespace, so
this PR exposes only operations that the pinned SDK can execute reliably and
documents that limitation.
### Release note
Support namespace, table, and top-level column DDL for Lance external
catalogs.
### Test Plan
- Compile the FE reactor against JDK 17.
- Run focused Lance DDL, type conversion, namespace mutation, metadata
cache, and ALTER TABLE command tests.
- Run Checkstyle and Markdownlint.
- Add a filesystem regression suite covering successful DDL and unsupported
operations.
### Test Result
- FE reactor compilation: passed.
- 65 focused unit tests: passed.
- Checkstyle: passed with 0 violations.
- Markdownlint for docs/lance-ddl.md: passed.
- The Doris + MinIO regression suite was added but not executed locally
because this environment does not provide that cluster.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. Lance external catalogs can now execute supported DDL
statements.
- Does this need documentation?
- [ ] No.
- [x] Yes. Added docs/lance-ddl.md in this PR.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
