shoemoney opened a new issue, #51127: URL: https://github.com/apache/arrow/issues/51127
### Describe the bug, including details regarding any error messages, version, or platform. `is_null(arr, nan_is_null=True)` does not mark NaN values as null when `arr` is dictionary-encoded with floating-point dictionary values. `IsNullExec` decides whether to take the NaN path by looking at the top-level type id. For a dictionary-encoded array that id is `DICTIONARY`, so the floating-point check is skipped entirely and NaN entries in the dictionary are never marked null. The same call on the decoded (non-dictionary) array returns the expected result. Related: null *values* inside the dictionary (as opposed to null indices) are also not reflected, so a valid index pointing at a null dictionary slot is reported as non-null. Discussion and a first attempt at a fix are in #51000; @pitrou suggested computing `is_null` on the dictionary values, `take`-ing by the indices, and OR-ing the result with the index validity bitmap, which covers both the NaN case and null dictionary values in one pass. ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
