nchammas commented on code in PR #45036:
URL: https://github.com/apache/spark/pull/45036#discussion_r1491158277
##########
core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala:
##########
@@ -110,6 +110,18 @@ class OpenHashSet[@specialized(Long, Int, Double, Float)
T: ClassTag](
this
}
+ /**
+ * Check if a key exists at the provided position using object equality
rather than
+ * cooperative equality. Otherwise, hash sets will mishandle values for
which `==`
+ * and `equals` return different results, like 0.0/-0.0 and NaN/NaN.
Review Comment:
Yes, the differences are subtle:
```scala
scala> 0.0 == -0.0
val res0: Boolean = true
scala> 0.0 equals -0.0
val res1: Boolean = false
scala> Double.NaN == Double.NaN
val res2: Boolean = false
scala> Double.NaN equals Double.NaN
val res3: Boolean = true
```
There is a long discussion on the Scala forums from 2017 about this
difference and some of the problems it causes:
[Can we get rid of cooperative
equality?](https://contributors.scala-lang.org/t/can-we-get-rid-of-cooperative-equality/1131)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
