JoshRosen commented on a change in pull request #34351:
URL: https://github.com/apache/spark/pull/34351#discussion_r733921709
##########
File path:
core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala
##########
@@ -149,17 +149,12 @@ class OpenHashMap[K : ClassTag, @specialized(Long, Int,
Double) V: ClassTag](
}
}
- // The following member variables are declared as protected instead of
private for the
- // specialization to work (specialized class extends the non-specialized one
and needs access
- // to the "private" variables).
- // They also should have been val's. We use var's because there is a Scala
compiler bug that
Review comment:
Spark was using Scala 2.9.3 at the time that this code was originally
written, so it's quite likely that this has been fixed now that Spark uses
Scala 2.12.
I searched the code and found that there's two places that duplicate this
workaround:
-
[PrimitiveKeyOpenHashMap](https://github.com/apache/spark/blob/0494dc90af48ce7da0625485a4dc6917a244d580/core/src/main/scala/org/apache/spark/util/collection/PrimitiveKeyOpenHashMap.scala#L120-L131)
-
[GraphXPrimitiveKeyOpenHashMap](https://github.com/apache/spark/blob/0494dc90af48ce7da0625485a4dc6917a244d580/graphx/src/main/scala/org/apache/spark/graphx/util/collection/GraphXPrimitiveKeyOpenHashMap.scala#L140-L152)
Once we've decided on the best fix we should make sure to apply it there as
well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
