OIiveirra opened a new pull request, #69576:
URL: https://github.com/apache/doris/pull/69576
### What problem does this PR solve?
Issue Number: N/A (no associated GitHub issue)
Related PR: #69113
Problem Summary:
Backport #69113 to `branch-4.2`. An outer predicate on a projected date
expression can be expanded and evaluated before an inner condition rejects the
row. A filtered row containing `9999-12-31 23:59:59` can therefore cause
`WEEKS_ADD(dt, 5)` to overflow instead of returning the expected empty result.
Preserve potentially overflowing date arithmetic aliases above Filter, Join,
Limit, EXCEPT, INTERSECT, Generate and CTE consumer boundaries in logical and
physical project predicate pushdown. The branch adaptation uses its existing
logical/physical EXCEPT and INTERSECT classes and retains its Project-level
NoneMovableFunction and volatile-expression guards. Safe integer-zero offsets
and ordinary UNION retain the behavior of the master fix.
### Release note
Fix date arithmetic overflow errors caused by predicate pushdown evaluating
expressions on rows excluded by an inner query.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
<details>
<summary>Local validation on branch-4.2</summary>
- FE/BE build and strict Checkstyle passed with JDK 17 and clang21/ASAN. The
service build used the repository's `DISABLE_BUILD_UI=ON` option.
- **17 FE unit tests passed**, with zero failures, errors or skips:
`DateArithmeticFilterPushDownTest` (11), `PushDowFilterThroughProjectTest` (4),
and `org.apache.doris.nereids.postprocess.PushDownFilterThroughProjectTest` (2).
- **Three native P0 suites passed**, with zero failed/fatal/skipped suites:
`nereids_rules_p0/filter_push_down/test_date_arithmetic_rejecting_boundary`,
`nereids_rules_p0/date_function_rewrite/date_function_rewrite`, and
`nereids_rules_p0/cte/test_cte_filter_pushdown`.
- Tests ran on this branch's verified binaries in a fresh isolated 1 FE + 3
BE instance. Source, configuration, framework and process identities remained
stable. The epoch-based date-rewrite suite was run with the private instance's
default time zone set to `Asia/Shanghai`.
- Source whitespace checks passed; regression outputs retain the framework's
standard final separator. The five added test/output files are identical to the
merged master fix; expected outputs were not changed for this backport.
Manual validation passed **18 checks covering ORC and Parquet**, in each of
UTC and Asia/Shanghai:
1. Generate 200 rows: one row has `dt='9999-12-31 23:59:59'` and
`filter_dt='2025-01-01'`; the other 199 have `dt='2020-01-01'` and
`filter_dt='2020-01-01'`. Export the same input to ORC and Parquet on a fresh
private HDFS fixture.
2. For WEEKS_ADD, HOURS_ADD, SECOND_CEIL, YEAR_CEIL and DAYS_ADD, run the
filtered CTE/outer-alias predicate shape from the original issue. All return
zero rows without overflow. A surviving-row control returns a count of 199; a
shared CTE control returns 398 rows.
3. Verify that arithmetic on a participating maximum-date row still raises
the expected out-of-range error, and that a NULL date returns NULL. Input row
hashes and HDFS/runtime identities remained unchanged during each run.
These are targeted local results, not a full TeamCity or External-group
verdict. The five historical RQG executions have not been replayed because
their actual external-table DDL and immutable ORC inputs were unavailable; the
manual fixtures above were reconstructed.
</details>
- Behavior changed:
- [ ] No.
- [x] Yes. Preserve date arithmetic evaluation boundaries while
retaining safe zero-offset predicate pushdown and partition pruning. Nonzero
date predicates can remain above rejecting boundaries.
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
