wenzhenghu opened a new pull request, #64705:
URL: https://github.com/apache/doris/pull/64705
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
This PR avoids blocking external meta cache invalidation on slow miss loads.
Previously, when an external metadata miss was loaded through Caffeine's
synchronous loading path, operations such as refresh catalog could wait on slow
external metadata access and block the FE replay-related invalidation path.
This change moves the slow miss load out of the synchronous Caffeine load path,
prevents stale write-back after invalidation with an invalidate generation
check, and adds regression coverage for the refresh catalog blocking scenario.
It also handles null results from manual miss loaders without writing null
values into the cache and adds a focused FE unit test for that path.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
Manual test:
1. Reproduced the blocking path with refresh catalog against a JDBC external
catalog and a debug point that sleeps in PluginDrivenExternalTable.initSchema.
2. Repeated the baseline scenario 5 times with
enable_external_meta_cache_manual_miss_load=false and observed refresh catalog
blocked for about 14s while DESC stayed slow.
3. Repeated the optimized scenario 5 times with
enable_external_meta_cache_manual_miss_load=true and observed refresh catalog
return within about 1s while DESC remained slow.
4. Added a regression case as a manual-test reference because its execution
depends on JDBC regression environment and FE debug-point availability.
Unit test:
- FE_UT_PARALLEL=1 ./run-fe-ut.sh --run
org.apache.doris.datasource.metacache.MetaCacheEntryTest
- Behavior changed:
- [x] Yes.
Behavior change:
- refresh catalog and the corresponding FE invalidation path are no longer
blocked by slow external metadata miss loads in this meta cache entry
implementation.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
