924060929 opened a new pull request, #69458:
URL: https://github.com/apache/doris/pull/69458
### What problem does this PR solve?
Issue Number: None
Problem Summary:
External catalog database/table names, lookup objects, and schemas
previously had separate FE cache owners. After another engine drops or
recreates a table, refreshing its schema could leave the old table in the
cached membership list. Independent ownership also made it difficult to fence
old loads and clean up discarded database instances consistently.
Share one catalog-owned FE cache registry across names, lookup objects, and
schemas. Explicit table/database refresh atomically invalidates the target
scope and its parent membership snapshot. Write-triggered refresh keeps the
metadata-only scope, preserving table objects and cached names. Persist the
explicit table refresh scope for hot and cold replay paths.
Database eviction retires only that instance's entries. An accepted cache
operation can finish without blocking eviction on remote I/O; subsequent reads
bypass the retired entry and cannot publish cache/ID state. Catalog retirement
also retires naming adapters, and cold database-refresh replay neither accesses
closed entries nor initializes a new owner. Schema runtimes borrow the FE owner
and remove only their own entries when reconfigured.
Lookup handles retain their existing entry-count/TTL retention policy
through an explicit cache-definition budget policy. Names and schema payloads
continue to use configured byte budgets. This avoids recursively estimating
mutable catalog/database back-references and rejecting every lookup object when
a byte budget is enabled. Unified statistics also include naming-only owners
before the first schema load.
### Release note
Explicit external table/database refresh now invalidates cached lookup
objects and parent membership consistently. Ordinary write-triggered metadata
refresh keeps cached table names.
### Check List (For Author)
- Test
- [x] Regression test: Hive metadata refresh suite against an isolated
local FE and a real HMS, with two independent catalogs performing
drop/refresh/recreate operations. Expected output generated by the regression
runner.
- [x] Unit Test: **290 tests passed** (88 connector-cache + 202 FE),
including deterministic blocked-load, same-name recreation,
eviction/retirement, budget-enabled ID lookup, and ALTER followed by cold
refresh replay cases.
- [x] Manual test: FE build and Checkstyle. Runtime validation covers
FE/HMS metadata; no multi-FE cluster, actual Trino process, or BE data
execution was exercised. The local FE-only JDBC setup disables the distribute
planner; the regression suite does not change this variable.
- Behavior changed:
- [x] Yes. Explicit refresh also invalidates lookup and membership;
write-triggered refresh remains metadata-only. No new user configuration.
- Does this need documentation?
- [x] No. Existing refresh commands and lookup retention policy; the
cache-definition API documents the payload-byte accounting boundary.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
