PavithranRick opened a new pull request, #17489:
URL: https://github.com/apache/hudi/pull/17489
### Describe the issue this Pull Request addresses
This PR fixes incorrect matching behavior in expression index pruning.
Currently, queries that reference the raw data column (e.g., `ts`) are
incorrectly
matched against expression indexes (e.g., `from_unixtime(ts)`), causing
casting
exceptions during pruning. Although the engine swallows the errors and
continues,
it still performs unnecessary lookups.
This PR ensures expression indexes are matched **only when the query
expression
uses the same function** as the index definition. Direct column references
skip
expression-index matching (unless the index function is `identity`). This
prevents
invalid matches and avoids inefficient or erroneous pruning attempts.
### Summary and Changelog
- Added function-type validation in `extractQueryAndLiterals` to ensure query
expressions match the index's expression function.
- Direct column references no longer match expression indexes.
- Updated logic around line 443 of `ExpressionIndexSupport.scala`.
- Eliminates silent casting failures caused by mismatched expression types.
- No public API changes; logic fix only.
### Impact
- Improved correctness: non-matching predicates (e.g., `ts = value`) will no
longer attempt expression index pruning such as `from_unixtime(ts)`.
- Reduced overhead: avoids unnecessary failed lookups and swallowed
exceptions.
- More predictable and safe index pruning behavior.
### Risk Level
Low
Change is isolated to expression-index match logic.
Verification included:
- Tests for matching-with-function predicates.
- Tests for non-matching direct column predicates.
- Tests for mismatched expression types.
### Documentation Update
None
No new configs or user-facing features. Existing documentation remains valid.
### Contributor's checklist
- [ ] Read through contributor's guide
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
