924060929 opened a new pull request, #67417:
URL: https://github.com/apache/doris/pull/67417
### What problem does this PR solve?
Issue Number: None
Related PR: #65126
Problem Summary:
On branch-4.1, a slow external database or table-name load can start before
`REFRESH CATALOG` invalidates the legacy `MetaCache`. The invalidation itself
returns without waiting for remote I/O, but the pre-refresh Caffeine miss load
can complete afterward and publish its old names into the now-invalidated
cache. A later lookup then reads metadata collected before the refresh.
This is the branch-4.1 equivalent of the table-names non-blocking refresh
sequence covered by #65126. It is separate from the already-fixed
bounded-executor/removal-listener deadlock: the existing
`MetaCacheDeadlockTest` passes on the current branch and fails only when the
synchronous removal listener fix is reverted.
The fix is deliberately limited to the legacy `MetaCache` used by
branch-4.1. It does not backport the `MetaCacheEntry` catalog/database refactor
from #65126. Name misses are loaded outside Caffeine synchronized miss
publication and deduplicated with one load lock. A generation and a short
mutation/publication lock prevent a load that started before invalidation,
update, or per-name removal from being cached afterward. Remote connector I/O
remains outside the mutation lock, so refresh does not wait for the slow load.
Before the fix, the deterministic test fails with:
```text
expected:<[local-2]> but was:<[local-1]>
```
After the fix, the first caller still receives its completed load, while the
next lookup reloads and receives `local-2`.
### Release note
Prevent stale external database or table names from being republished after
concurrent metadata cache invalidation.
### Check List (For Author)
- Test:
- [x] Unit Test
- `MetaCacheTest`: 13 tests passed
- `MetaCacheDeadlockTest`: passed
- [x] FE Checkstyle: 0 violations
- [ ] Regression test
- [ ] Manual test
- Behavior changed:
- [x] Yes. An in-flight external name load still returns to its original
caller but is not cached after a concurrent invalidation or explicit name
mutation.
- Does this need documentation:
- [x] No
`./build.sh --fe` was attempted, but the isolated worktree does not contain
the complete `thirdparty/installed` tree and the script started
downloading/building the full third-party toolchain. It was stopped as an
environment setup operation. The focused FE unit-test runs compiled all FE main
sources and test sources successfully.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
