matanper opened a new issue, #70576: URL: https://github.com/apache/doris/issues/70576
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description A scan learns that a segment cannot match only after it has loaded and opened it. `LazyInitSegmentIterator::init` (`be/src/storage/segment/lazy_init_segment_iterator.cpp:35-50`) loads the segment through the segment cache. Then `Segment::new_iterator` (`be/src/storage/segment/segment.cpp:425-458`) checks the segment-level zone maps and returns an `EmptySegmentIterator` when no row can match. Before that check, the scan has already paid for: - a rowset reader per rowset and a lazy iterator per segment; - a scanner, tablet reader and scheduler slot for every split, including splits whose segments all turn out empty; - the segment load. In cloud mode a segment-cache miss is a remote read of the footer and column metadata. Selective queries that run repeatedly over the same recent data (dashboards, alerting rules, polling) open the same segments over and over, only to rule them out by the same zone maps each time. Segments are immutable, so their segment-level zone maps never change. The result of the first open can be kept and reused. ### Solution A small BE-local **segment zone-map index**: an LRU keyed by segment, holding the segment-level zone maps that `Segment::new_iterator` already evaluated for the top-level columns it read. 1. **Fill on open.** `new_iterator` stores the zone maps it read, at no extra I/O. A string bound cut at the zone-map size limit is not stored, and a zone map whose type differs from the column's read type is not used. 2. **Rule out before open.** Before loading a segment, the scan evaluates the kept zone maps with the same `zone_map_may_match` logic as the open path, for its column predicates and zone-map-capable expressions. When no row can match, it skips the load and counts the segment exactly as the open would: `filtered_segment_number`, filtered rows. A miss, or anything the index cannot decide, falls through to a normal open, so results never change. 3. **Rule out earlier when every segment of a unit is ruled out.** - Leave a rowset out before its reader is initialized. - When splitting a tablet among scanners, first split exactly as today, then drop splits whose segments are all ruled out. Build no scanner or tablet reader for a tablet left with nothing to read. - A ruled-out segment inside a split that is still read is skipped when the scanner reaches it. 4. **Keep the probing cheap for every scan shape.** Do not gate the index on query shape: skipping it for LIMIT or key-order Top-N scans gave up most of the gain on heavy queries. Instead, share one read schema per scan per BE across pipeline instances, and precompute per-column types and slot mappings once per scan, so a decision allocates nothing. Config: `enable_segment_zone_map_index` (mutable, default false) and `segment_zone_map_index_capacity_mb` (mutable, default 256). Profile counters: segments pruned before open, rowsets pruned before init, tablets pruned before scan, scanners avoided, index hit and miss. bvars for the same, plus index bytes and entries. Only queries use it. Compaction and other readers open every segment as today. Scans that collect BM25 statistics, or that run ANN Top-N, build no probes, so their statistics still cover every rowset. Correctness tests we have: - randomized parity sweeps showing that ruling out before open, before rowset init and before scanner building removes exactly what the open path would rule out (same rows, same filtered counters); - a flag-off build producing identical scanners; - delete-predicate rowsets always being read; - merge-on-read value predicates only ruling out base rowsets; - a pruned segment whose file was deleted not being loaded. ### Results In production on cloud-mode clusters (arm64, query compute groups with two BEs, index capped at 256 MiB): | | cluster A, 8 h | cluster B, 48 h | |---|---|---| | segment opens skipped by the index, per query BE | ~3.8–3.9 billion (~130 K/s) | ~3.1–3.2 billion (~18 K/s) | | share of the segments these queries would have opened | ~95% | ~95% | | index hit rate | 99.99% | 99.99% | | index memory | 240–268 MB (at the cap) | 268 MB (at the cap) | | rowsets left out before reader init | ~3.6 billion | ~3.1 billion | In an A/B on cluster A (interleaved off/on phases, frozen queries, identical rows in every run), filtered and heavy queries were 19–87% faster. Short queries that stop early on a LIMIT were 24–36 ms slower, from the up-front probing. Limits: the first scan of a segment after a BE starts, or after eviction, gets no benefit, because the index fills on open. A cold Top-N read is covered separately by #70135: ruling a segment out at open by the current Top-N threshold. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! We have a patch in four parts: the index and pruning before open, rowset pruning before init, scanner building, and cheap probing. We can open them as separate PRs. ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
