anshulsingh-py opened a new pull request, #19853: URL: https://github.com/apache/hudi/pull/19853
### Describe the issue this Pull Request addresses Closes #19852. Follow-up to #19638 / #19850, discussed with @voonhous [here](https://github.com/apache/hudi/issues/19638#issuecomment-5559550232). #19850 fixed procedure filters so functions outside the hardcoded resolution table are rejected with a clear error instead of silently dropping rows. This PR builds on that: when a function isn't in the table, fall back to Spark's own `FunctionRegistry` to resolve it before giving up. So builtins like `concat`, `instr`, `if` etc. actually evaluate correctly instead of being rejected — while anything the registry also can't resolve (a genuine typo) still falls through to the existing rejection path from #19850, unchanged. ### Summary and Changelog - Add `resolveViaFunctionRegistry` fallback in the shared `bindAndResolveExpression`, used by both `evaluateFilter` and `validateFilterExpression`. - Any Spark builtin not already in the hardcoded table now resolves and evaluates correctly. - Updated the pinned regression test that asserted `concat`/`instr`/`if` were rejected, since they now resolve; added coverage confirming they evaluate and validate correctly. ### Impact Procedure filters can now use any Spark builtin function, not just the ones explicitly enumerated in the resolution table. No change in behavior for functions genuinely unsupported by Spark itself. ### Risk Level Low — additive fallback only triggers for functions not already handled by the existing table, and preserves the existing rejection behavior for anything that still can't resolve. ### Documentation Update None; internal filtering utility. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
