Gabriel39 opened a new pull request, #67545:
URL: https://github.com/apache/doris/pull/67545
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
The connector metastore SPI refactor removed DLF provider registration and
the plugin-local DLF client path, so existing Iceberg and Paimon catalogs using
`dlf` can no longer be created or replayed on master.
This PR restores DLF support for both connectors:
- Register engine-specific DLF metastore providers and restore legacy
property aliases, endpoint derivation, catalog-id fallback, and sensitive
credential handling.
- Restore the Iceberg DLF catalog and client pool, with catalog-scoped
client caching and OSS-backed `S3FileIO`.
- Route Paimon DLF through its Hive catalog using `ProxyMetaStoreClient`,
while preserving legacy alias-only OSS and OSS-HDFS storage configurations.
- Bundle the matching Hive 3 and Hive 2 DLF clients into the existing
Iceberg/HMS and Paimon private shades, including their Thrift and Tea namespace
isolation.
### Release note
Restore Aliyun DLF metastore support for Iceberg and Paimon catalogs.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- Targeted Iceberg, Paimon, metastore provider, property-binding, and
DLF catalog tests passed.
- The 19-module connector package build passed with tests skipped; FE
Checkstyle reported zero violations.
- Verified both generated private shade jars contain the DLF proxy
client and relocated Thrift and Tea classes.
- The full Iceberg connector suite was also attempted; 1 of 1,376
tests failed in the unrelated
`IcebergWritePlanProviderTest.planMergePreservesExplicitlyEmptyReadAcrossConcurrentFirstAppend`
case. The focused rerun reproduces the same failure, and this PR does not
modify that code path.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. Iceberg and Paimon catalogs can use the DLF metastore backend
again.
- Does this need documentation?
- [x] No. Existing DLF catalog properties and behavior are restored.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
