kiszk edited a comment on issue #25404: [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 URL: https://github.com/apache/spark/pull/25404#issuecomment-532354030 I realized that [this commit](https://github.com/scala/scala/commit/846ee2b1a47014c69ebd2352d91d467be74918b5#diff-ac889f851e109fc4387cd738d52ce177) ([PR](https://github.com/scala/scala/pull/7607)) on 2019/1 affects the result of `Rand()` that calls `MurmurHash3.bytesHash()` in `XORShiftRandom`. Scala 2.12.8 was released on [2018/12](https://github.com/scala/scala/releases/tag/v2.12.8) Scala 2.12.9 was released on [2019/8](https://github.com/scala/scala/releases/tag/v2.12.9) Scala 2.12.10 was released on [2019/9](https://github.com/scala/scala/releases/tag/v2.12.10) How do we handle this? 1. Update test cases by using Scala's `MurmurHash3.bytesHash()`. Values from `Rand()` are different between Spark 3.x and Spark 2.x 2. Have own `MurmurHash3.bytesHash()`. Values from `Rand()` are the same among all of Spark versions.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
