yy782 opened a new pull request, #70379: URL: https://github.com/apache/doris/pull/70379
### Background Doris currently does not support predicate pushdown on struct columns (issue #70376). When a query filters on a struct field, for example `SELECT * FROM t WHERE s.a < 2`, the field predicate does not participate in zone-map pruning. The struct column is materialized in full and the field predicate is evaluated row by row, so segments and pages that could never satisfy the predicate are still read. This PR reuses Doris's existing expression zone-map pruning path instead of introducing a struct-specific mechanism, in the spirit of DuckDB's struct filter pushdown (duckdb#10314). ### What problem does this PR solve? Issue Number: close #70376 Related PR: None PR Type: Enhancement Problem Summary: Extend the expression zone-map pruning path to recognize `struct.field OP literal` predicates and evaluate them against each scalar subfield's own min/max bounds, so segments, pages and external-table row groups can be pruned instead of evaluating the field predicate only row by row. ### Details - **Change map:** The change lives in the storage zone-map path: the struct column writer now keeps per-scalar-subfield zone maps, `ZoneMapEvalContext` keys zone maps by a slot path, `SegmentIterator` prunes struct subfields, and the comparison zonemap evaluator recognizes struct field predicates. A handful of call sites were updated for the `slot_index` signature change. - **Design:** The existing zone-map pruning path is reused rather than given a struct-specific branch. Scalar STRUCT subfields now carry their own per-page and segment-level zone maps, and zone maps are looked up by a slot path (column id followed by field ordinals) so top-level columns and nested subfields share one dispatch. The segment iterator recurses into STRUCT columns and prunes each scalar subfield at page granularity, while the comparison evaluator recognizes `struct.field OP literal` as a zone-map-pushable predicate against the subfield's own bounds. - **Scope:** STRUCT scalar subfields at any nesting depth; `s.a`, `s.a.b`, etc. ARRAY/MAP element access is intentionally excluded because its elements are not individually zone-mapped. Dictionary and bloom-filter pruning remain top-level-only for nested subfields. - **Backward compatibility:** Segments written before this change have no subfield zone map and simply get no struct-field pruning on those subfields, falling back to normal evaluation. The file format is unchanged. - **Behavior change:** Pruning only drops pages and segments that provably cannot match; the field predicate is still applied on the surviving rows, so query results are identical with pruning enabled versus disabled. ### Testing - `sh run-be-ut.sh --run --filter='ExprZonemapFilterTest.StructField*'`: BE unit test `be/test/exprs/expr_zonemap_filter_test.cpp` covering struct field extraction/evaluation, null-literal rejection, nested paths and ARRAY-element exclusion; passed. - `sh run-regression-test.sh --run -f regression-test/suites/query_p0/expr_zonemap/test_expr_zonemap_pruning.groovy`: regression suite `test_expr_zonemap_pruning` with the added `test_expr_zonemap_pruning_struct` (`STRUCT<a:INT, b:INT>`) using `assertSameWithAndWithoutPruning` and exact-count checks for `<`, `=`, `>`, `<=`, `>=`, `!=` and `AND` on `s.a` / `s.b`; passed. - Correctness holds by construction: pruning is conservative and the predicate is still applied row-by-row on surviving rows. ### Notes for Reviewers ARRAY/MAP element access (`s.arr[1] OP x`) is intentionally not pruned — there is no per-element zone map, only STRUCT field access participates. The `slot_index` change from `int` to `std::vector<int>` at `runtime_filter_expr`, `vbloom_predicate`, `vectorized_fn_call`, `vtopn_pred` and `parquet_scan` is a mechanical signature update. ### Release note Support predicate pushdown (zone-map pruning) on STRUCT scalar subfields (`struct.field OP literal`). ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: Claude Code (Claude Opus 5.5) - Generated: code, tests, this description - Human review: The author reviewed the AI-generated code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
