HappenLee commented on PR #65837: URL: https://github.com/apache/doris/pull/65837#issuecomment-5263880035
## Performance validation Tested commit `b716b37c76e` with the local `$run_path/doris` deployment and the TPCH SF100 database. ### A/B setup - Query: equality join with an IN runtime filter on the probe table's hash-distribution key. - Probe table: 16 range partitions × 96 hash buckets = 1,536 probe tablets, with data distributed across every tablet. - Build table: one matching key. - 10 interleaved disabled/enabled pairs after warm-up. - SQL/result cache disabled; one fragment instance; global IN runtime filter. - Partition runtime-filter pruning disabled to isolate bucket/tablet pruning. - Cost-based runtime-filter removal disabled so both sides retain the same runtime filter. - The only A/B change was `enable_runtime_filter_bucket_prune`. | Metric | Disabled | Enabled | Improvement | |---|---:|---:|---:| | Client wall-time median | 141.5 ms | 75.0 ms | 47.0% lower / 1.89× faster | | Doris profile latency median | 117 ms | 51 ms | 56.4% lower / 2.29× faster | | Probe tablets pruned | 0 / 1,536 | 1,520 / 1,536 | 99.0% | | Total scanners (probe + build) | 1,537 | 17 | 98.9% fewer | The result was identical in both modes (`120`), and both profiles read 33 rows. The enabled profile reduced the probe side from 1,536 scanners to 16 scanners. ### TPCH SF100 sanity check I also ran the runtime-filter join against the existing SF100 `lineitem` table (600,037,902 rows, 96 hash buckets). Both modes returned the same result (`6`), while the enabled mode pruned 95 of the 96 `lineitem` buckets. This validates that the optimization preserves correctness and materially reduces tablet/scanner setup overhead when runtime-filter selectivity identifies a small subset of hash buckets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
