raushanprabhakar1 opened a new pull request, #3196:
URL: https://github.com/apache/datafusion-comet/pull/3196
This commit fixes the `array_contains` implementation to properly handle
Spark's three-valued null logic:
1. Returns `null` if the search value is `null`
2. Returns `null` if the array itself is `null`
3. Returns `true` if the value is found in the array
4. Returns `false` if no match is found **and** no `null` elements exist
5. Returns `null` (indeterminate) if no match is found **but** `null`
elements exist
### Implementation Details
The implementation uses nested `CaseWhen` expressions to:
- First check if the array is `null` (return `null` if so)
- Then check if the search key is `null` (return `null` if so)
- Use `array_has` to check if the value exists (return `true` if found)
- If not found, check whether the array contains `null` elements using
`array_has` with a `null` literal:
- Return `null` if `null` elements exist
- Return `false` otherwise
This ensures compatibility with Spark's behavior, where `array_contains`
returns `null` when the result is indeterminate due to `null` elements
in the array.
### References
- Fixes #3177
### Tests
- Added comprehensive test cases covering all three-valued logic scenarios,
including:
- `null` arrays
- `null` search values
- Arrays containing `null` elements
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
