LuciferYang opened a new pull request, #68521: URL: https://github.com/apache/doris/pull/68521
## Proposed changes Issue Number: close #67995 ### The problem Three hidden columns store a placeholder on disk and get their real value substituted at read time, so their on-disk zone map describes the placeholder rather than the value rows come back with: | column | on disk | effective read-time value (single-version reads) | | --- | --- | --- | | `__DORIS_VERSION_COL__` | 0 | the rowset version | | `__DORIS_COMMIT_TSO_COL__` | 0 | the rowset commit_tso, once published | | `__DORIS_BINLOG_TSO__` | NULL | the commit_tso on a binlog read | Zone-map pruning that trusts the on-disk summary drops matching rows. On a single-version unique-MoW rowset, `WHERE __DORIS_VERSION_COL__ = <real version>` is pruned against `[0, 0]` and wrongly returns nothing. ### The fix A new `Segment::placeholder_effective_value(cid, schema, read_options)`, keyed on the read-schema ordinal (`version_ordinal()` / `commit_tso_ordinal()` / `tso_ordinal()`), returns the effective value on single-version reads and `nullopt` otherwise. Multi-version reads carry real per-row values, and an unpublished commit-tso (`end_tso() == -1`) leaves the on-disk 0 that is already correct. Every zone-map consumer routes through it: - the expression segment-zonemap builder synthesizes a `min == max == v`, non-null summary in place of the on-disk one; - the ColumnPredicate segment path prunes against the effective value; - the expression page path and the ColumnPredicate page path skip the column, since a placeholder is constant across pages; - the pushed-down aggregate gate bails when a placeholder column is referenced, so a min/max/count is not answered from the placeholder summary. ### How was this patch tested? - Unit tests for `placeholder_effective_value`: each of the three ordinals substitutes on a single-version read; a multi-version read, an unpublished commit-tso, and a plain column return `nullopt`. - Regression `test_expr_zonemap_hidden_placeholder` on a single-version unique-MoW rowset: `WHERE __DORIS_VERSION_COL__ = <real version>` returns every row, a non-existent version prunes to zero, a pushed `MIN/MAX/COUNT` reads real data, and the MoR path stays correct. ### Notes for reviewers - Based on `dea1b99e`; may need a rebase onto current master. - Merge before #67774, whose native slot-vs-slot pruning inherits this fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
