Baymine opened a new pull request, #66294:
URL: https://github.com/apache/doris/pull/66294
### What problem does this PR solve?
Issue Number: close #66293
Problem Summary:
A predicate over a monotonic function of a column, e.g. `date_trunc(dt_col,
'day') = '2026-03-26'`, `year(dt_col) >= 2024`, `*_ceil` / `*_floor`,
`from_unixtime(epoch_col) >= '...'`, `to_date(dt_col)`, `to_monday(dt_col)`, or
a binary-collation prefix such as `substring(s_col, 1, 3) = 'abc'` /
`left(s_col, 3) >= 'abc'`, cannot be consumed by any storage-layer min/max
statistics today, because the statistics are kept on the bare column, not on
the wrapped expression. As a result ORC/Parquet row-group pruning, OLAP zonemap
pruning, and partition pruning all miss these predicates and scan far more data
than necessary.
This adds a new Nereids expression-rewrite rule
`InferPredicateFromMonotonicFunction`, registered in
`ExpressionOptimization.ADD_RANGE` alongside `AddMinMax`. For a predicate over
a monotonic (or piecewise floor/ceil) function of a column it derives an
ADDITIONAL bare-column range predicate that any storage layer's min/max can
consume, while always keeping the original predicate (the derived one is a
one-way / relaxed implication, so the original still does the exact filtering).
The rule is gated by a new session variable
`enable_infer_predicate_from_monotonic_function` (default `true`).
The derivation is built on the existing `Monotonic` trait framework:
- `Monotonic` gains four default methods (`isFloor`/`floorUpperBound`,
`isCeil`/`ceilLowerBound`) so floor/ceil-shaped functions can expose a
reversible bound; `date_trunc`, `to_date`, `to_monday`, `date`, and the
`*_ceil`/`*_floor` families implement them, reusing the `*Add` scalar functions
so carry-over (leap year / month end) is handled by the kernel rather than by
hand.
- `DateUtils.hasZoneOffsetTransition` is added as the soundness gate for
`from_unixtime`: the epoch->local map is only strictly monotonic on a stretch
with no UTC-offset transition, so the rule refuses to derive a bound whenever a
DST/base-offset transition falls in the guarded interval.
Soundness safeguards:
- from_unixtime: a DST-aware guard band just below the constant's epoch; if
any zone-offset transition (fall-back or gap) lies in the guarded interval the
rule derives nothing.
- prefix functions (substring/left): the derived range bound is only emitted
under binary collation, where lexicographic byte order matches the compared
string order.
- date_trunc / floor units: the `[c, c + 1 unit)` upper bound is built from
the exact trunc unit via the reused `*Add` functions, so end-of-month /
leap-year clamp reversibility is preserved.
Note: this ports the monotonic-inference rule only. Compound-unit `+/-
INTERVAL` operator binding (e.g. `DAY_HOUR`, `HOUR_SECOND`) from the same
internal change is intentionally out of scope here (it needs separate
parser/registry work) and will be proposed as a follow-up.
### Release note
Add session variable `enable_infer_predicate_from_monotonic_function`
(default true) that lets the optimizer derive an extra bare-column range
predicate from predicates over monotonic functions (date_trunc, year,
*_ceil/*_floor, from_unixtime, to_date, to_monday, substring/left prefix), so
storage-layer min/max statistics (ORC/Parquet row-group, OLAP zonemap,
partition pruning) can prune more data. The original predicate is always kept;
set the variable to false to disable.
### Check List (For Author)
- Test
- [x] Unit Test
- [ ] Regression test
- [x] No need to manual test. Explain why:
- The change is a pure FE optimizer rewrite guarded by a session
variable; correctness is fully covered by JUnit tests that assert the derived
predicates and the soundness gates (DST transition guard, floor/ceil bounds,
prefix binary-collation) directly. New/updated tests:
InferPredicateFromMonotonicFunctionTest (68), DateFloorTraitTest (12),
DateUtilsTest (10); regression checks on SimplifyArithmeticComparisonRuleTest,
DatetimeFunctionBinderTest, BindExpressionTest all pass.
- Behavior changed:
- [x] Yes. Adds a new session variable and an additional (relaxed)
derived predicate; the original predicate is always preserved, so query results
are unchanged.
- Does this need documentation?
- [ ] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
