[
https://issues.apache.org/jira/browse/ARROW-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382437#comment-17382437
]
Eduardo Ponce commented on ARROW-13364:
---------------------------------------
*EqualOptions* has a *nans_equal* member to control the behavior of comparisons
between NaNs. I assume this was included to satisfy the behavior of different
tools.
IEEE 754 states that all logical operations with a NaN should always result in
false, except for *NaN != x*. For a discussion on this topic, refer to the
first answer of [this stack overflow
question|https://stackoverflow.com/questions/1565164/what-is-the-rationale-for-all-comparisons-returning-false-for-ieee754-nan-values]
and [wikipedia NaN page|https://en.wikipedia.org/wiki/NaN].
My opinion for Arrow is that all logical comparisons with a NaN value should
return false except for:
* NaN != x
* NaN == NaN
this would eliminate the use of the *nans_equal* option.
> [C++] Should NaN comparison return false or NaN/NA?
> ---------------------------------------------------
>
> Key: ARROW-13364
> URL: https://issues.apache.org/jira/browse/ARROW-13364
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Jonathan Keane
> Priority: Major
>
> In working on ARROW-12964 we ran into some corner behaviors with {{NaN}} that
> don't match our (and R's) expectations. It appears that (any?) comparison
> with `NaN` results in false:
> {code:r}
> > Scalar$create(NaN) > 5
> Scalar
> false
> {code}
> though at least in R this would result in an NA value:
> {code:r}
> > NaN > 5
> [1] NA
> {code}
> The current behavior _does_ match numpy's behavior:
> {code:python}
> >>> np.nan > 5
> False
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
