wenzhenghu opened a new issue, #66233: URL: https://github.com/apache/doris/issues/66233
## Search before asking I searched existing Apache Doris issues for duplicate external metadata local names, `meta_names_mapping` conflicts, and ambiguous remote-to-local name resolution. I did not find an issue covering this exact gap. ## Version Apache Doris `master` at `2215dc476a1f9a1f6edba8c7a056e450165f7001`. This problem was found while reviewing #65126, but the ambiguous mapping state is already possible on the master baseline and was not introduced by that PR. ## What is wrong? External metadata name loading does not unconditionally enforce that one exact local database/table name identifies only one remote object. For example, a custom naming hook or mapping can produce: ```text RemoteA -> LocalX RemoteB -> LocalX ``` This state is not resolvable: SQL can reference only `LocalX`, while the object cache key and generated Doris metadata ID are also derived from `LocalX`. The generic conflict checks currently run only under selected case-insensitive/lower-case modes. In ordinary mode 0, a custom connector naming hook or an uncovered mapping collision can still produce the same exact local name for multiple remote objects. ## Master baseline behavior The legacy `MetaCache` keeps the complete remote/local pairs in a mutable list. - `MetaCache.listNames()` returns both local entries, so database enumeration can expose duplicate `LocalX` values. - `MetaCache.getRemoteName(LocalX)` uses `findFirst()`, so only `RemoteA` is selected. - The metadata object cache is keyed only by `LocalX`, so `RemoteB` cannot be addressed independently. - Incremental `updateCache()` appends another pair without replacing the existing local identity, making the result event-order dependent. Relevant code: - https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java#L83-L92 - https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java#L128-L138 - https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java#L240-L267 - https://github.com/apache/doris/blob/2215dc476a1f9a1f6edba8c7a056e450165f7001/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java#L593-L618 ## Impact The trigger is narrow because normal remote metadata normally has unique names and standard mapping implementations reject many direct conflicts. The remaining gap mainly affects custom connector naming hooks, general `ExternalCatalog` subclasses, or uncovered mode-0 mapping collisions. Once triggered, the impact is a correctness problem: - one remote database/table becomes inaccessible or is resolved through the wrong remote identity - cached and cache-bypass paths may select different remote objects - object-cache keys and deterministic IDs collide - database enumeration may contain duplicate local names - incremental events may change the selected remote object according to event order ## Expected behavior Within one catalog scope for databases, and within one database scope for tables: - an exact local name must identify at most one remote object - two different remote names mapping to the same exact local name must fail with a clear conflict error - case-sensitive mode 0 must continue to allow distinct local names such as `Foo` and `foo` - mode-specific case-insensitive conflict checks must remain in effect - duplicate/replayed incremental events must remain idempotent ## Suggested fix 1. Validate the exact local-name uniqueness invariant when building the complete external metadata names snapshot. 2. Report both conflicting remote names and the shared local name. 3. Add a defensive invariant in the shared names value abstraction so custom loaders cannot publish an ambiguous snapshot. 4. Keep mode-aware case-insensitive collision validation in the catalog/database loader. 5. Add database and table tests, including a custom naming hook in mode 0. PR #65126 plans to fix this by validating the invariant in its immutable `NameCacheValue` snapshot before publication. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
