xylaaaaa opened a new pull request, #65562:
URL: https://github.com/apache/doris/pull/65562
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
FileScannerV2 ORC had three correctness and reliability gaps in metadata and
predicate pushdown:
1. MIN/MAX aggregate pushdown interpreted timezone-less ORC TIMESTAMP
statistics in the session timezone, shifting wall-clock values in non-UTC
sessions.
2. A partially pushdownable AND below an odd number of NOT operators could
produce a stronger SARG and incorrectly prune matching stripes.
3. Recoverable SARG construction or stripe-statistics evaluation exceptions
failed reader open even though residual scanning could return correct results.
This PR restores wall-clock TIMESTAMP statistics with UTC coordinates,
tracks negation parity and requires all AND children to be pushdownable under
NOT, and falls back to scanning without SARG for recoverable SARG failures.
Split ranges and residual predicates are preserved. Cancellation and memory
allocation or limit failures remain fatal.
### Release note
Fix ORC V2 metadata and predicate pushdown correctness, and fall back to
residual scanning when optional SARG processing fails.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
-
`NewOrcReaderTest.AggregatePushdownTimestampMinMaxMatchesScanInSessionTimezone`
- `NewOrcReaderTest.SargPartialAndRespectsNegationContext`
- `NewOrcReaderTest.SargExceptionsFallBackToResidualScan`
- `NewOrcReaderTest.*`
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. Unsafe or failed ORC pushdown now falls back conservatively,
and TIMESTAMP MIN/MAX pushdown matches normal scan results.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
