uchenily opened a new pull request, #66929:
URL: https://github.com/apache/doris/pull/66929
### What problem does this PR solve?
In cloud mode, the old topn deferred materialization path
(DeferMaterializeTopNResult, enabled when
topn_lazy_materialization_threshold=0) broadcasts a multiget_data RPC to all
BEs in the compute group. Each BE uses force_use_only_cached=true to only read
the local tablet cache, avoiding meta-service RPCs on BEs that do not hold the
tablet (optimization introduced by #50803, parameter-passing bug fixed by
#58044).
This optimization assumes the tablet cache is enabled and that the BE which
performed the scan phase still has the tablet cached by the time the
deferred-materialization RPC arrives. When this assumption breaks, every BE
returns 0 rows and the coordinator reports:
Miss matched return row loc count 0, expected 0, input N
Two scenarios break the assumption:
1. tablet_cache_capacity=0: CloudTabletMgr falls back to DummyLRUCache
(every insert and lookup is a no-op), so force_use_only_cached always misses --
even on the BE that performed the scan. This is a silent misconfiguration with
no visible warning.
2. Cache eviction under a small tablet_cache_capacity: in a multi-BE cluster
the scan phase loads the tablet into the scanning BE's cache, but between the
scan and the deferred- materialization RPC other concurrent queries may load
enough tablets to evict it. The scanning BE then misses just like the
non-scanning BEs, and since it is the only BE that holds the rowset data, the
coordinator cannot recover the rows.
(In a normal multi-BE cluster, BEs that did not perform the scan are
*expected* to miss the cache and skip -- the coordinator merges results from
the scanning BE. The bug only manifests when the scanning BE itself misses,
which happens with cache disabled or evicted.)
This patch makes the old path resilient without giving up the #50803
optimization:
rowid_fetcher.cpp: when force_use_only_cached=true misses, fall back to a
normal get_tablet() load (force_use_only_cached=false). The cache-only fast
path is still attempted first, so the broadcast-remote-read avoidance is
preserved when the cache hits. The fallback only pays a meta-service RPC on
cache miss, which is the same behavior as the new multiget_data_v2 path
(#55870).
Reproduction
------------
Environment: single-node cloud-mode cluster
```
CREATE TABLE t (k int, v int) DUPLICATE KEY(k)
DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES("replication_num"="1");
INSERT INTO t VALUES (1, 1), (2, 2);
```
Set tablet_cache_capacity=0 in be_custom.conf and restart BE.
Before fix:
```
mysql> SET enable_sql_cache=false;
mysql> SET topn_lazy_materialization_threshold=0;
mysql> SELECT k,v FROM t ORDER BY k LIMIT 10;
ERROR 1105 (HY000): errCode = 2, detailMessage =
(127.0.0.1)[INTERNAL_ERROR]
Miss matched return row loc count 0, expected 0, input 2
```
After fix:
```
mysql> SET enable_sql_cache=false;
mysql> SET topn_lazy_materialization_threshold=0;
mysql> SELECT k,v FROM t ORDER BY k LIMIT 10;
+------+------+
| k | v |
+------+------+
| 1 | 1 |
| 2 | 2 |
+------+------+
```
Issue Number: close #xxx
Related PR: #50803
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
