mrhhsg opened a new pull request, #70211:
URL: https://github.com/apache/doris/pull/70211
### What problem does this PR solve?
Issue Number: None
Problem Summary:
`VBloomPredicate::get_digest()` hashes the entire Bloom runtime-filter
payload. Scan initialization and scanner/file-local expression clones can
repeatedly request the same seeded digest. With large Bloom filters, this
redundant hashing becomes a significant scan CPU cost.
This change:
- Shares a digest cache within each predicate clone lineage.
- Keys entries by the actual shared payload owner token, completed-mutation
generation, and child-derived seed. In-place assign/merge and writes through
light-copy aliases invalidate old entries; a newly allocated payload cannot
reuse an old owner identity. The existing hash and zero-child-digest fallback
are preserved.
- Initializes each entry once per stable payload version/seed across
concurrent clones, outside the map mutex. Version increments are per
operation/batch, not per inserted row; retained tokens do not keep old large
Bloom directories alive.
- Starts a fresh cache on `set_filter()` while existing clones retain their
original payload/cache pair.
It does not disable condition cache, change runtime-filter sizing, or change
query results.
### Release note
Reduce repeated Bloom runtime-filter hashing during scan initialization
while preserving condition-cache digest values.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test (build and static checks below)
- ASAN BE and FE builds passed using `build.sh`.
- 49 focused ASAN BE tests passed, repeated 20 times:
```bash
DORIS_TOOLCHAIN=clang GTEST_REPEAT=20 ./run-be-ut.sh --run \
--filter='VBloomPredicateDigestTest.*:BloomFilterFuncTest.*:RuntimeFilterWrapperTest.*:RuntimeFilterMergerTest.*:FileScannerV2Test.ConditionCacheDigestIncludesRuntimeFilterPayload'
-j24
```
The unmodified `vparquet_writer_test.cpp` currently fails to compile
because its `WriteInfo` initializer no longer matches the writer API. It was
temporarily excluded from the focused UT build and restored afterward; this is
not a full-UT-suite pass.
- A 64 MiB payload requested 128 times performs one full hash per seed.
Eight concurrent clones with two distinct seeds perform exactly two hashes.
Eight digest tests also cover child rebinding, filter replacement, an in-place
old-superset-to-new-subset assignment after the first digest, batch/set/merge
mutation through light-copy aliases, equal-generation owner replacement,
unchanged legacy digests, and an undigestible child.
- The new SQL regression was generated with `-forceGenOut` and rerun
without it using an isolated worktree cluster. These three suites passed:
```bash
./run-regression-test.sh --conf
output/local-regression/regression-conf-12836.groovy \
--run -d query_p0/cache \
-s
test_bloom_predicate_digest_cache,condition_cache_eligibility,condition_cache
```
- clang-format 16/check-format, changed-line clang-tidy, and build hygiene
passed. Full-file clang-tidy also reported one pre-existing signed/unsigned
loop-comparison warning outside this patch; it was not changed. Whitespace
checks allow the regression runner's generated final blank separator.
- No end-to-end large-dataset latency improvement is claimed without an
A/B benchmark.
- Behavior changed:
- [x] No. Digest values and query results remain unchanged.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
