HeartSaVioR commented on issue #24113: [MINOR][CORE] Replace Scala forkjoin package to Java bundled one URL: https://github.com/apache/spark/pull/24113#issuecomment-473773947 I just realized that the code would not compile with `-Pscala-2.11` profile, since `ForkJoinTaskSupport` in Scala 2.11 requires Scala version of `ForkJoinPool` instead of Java bundled one - so we cannot do this unless we set lower bar of Scala to `scala-2.12`. Btw, once we want to both support Scala 2.12 and Scala 2.13, current code will break since they look to simply remove the alias in Scala 2.13 (They seem to remove ForkJoinTaskSupport in scala 2.13, too). Maybe broader fix should be needed at that time then. One interesting thing is, some claims performance regression on changing ForkJoinPool from Scala one to Java bundled one (which is done Scala 2.12), and Akka team decided to keep Scala 2.11 version of ForkJoinPool in their repository. I'm not sure I can follow the discussions and changes related to this (they stated that the perf difference came from using busy-wait), but the implementation of ForkJoinPool on Scala 2.11 and Java bundled one sounds to be different, and perf. regression might happen silently between Scala 2.11 vs Scala 2.12. http://www.scala-archive.org/ForkJoinPool-ExecutionContext-Implicits-global-performance-problem-for-Scala-2-12-td4647851.html https://github.com/scala/bug/issues/10083 https://akka.io/blog/news/2017/04/13/akka-2.5.0-released https://github.com/akka/akka/blob/master/akka-actor/src/main/java/akka/dispatch/forkjoin/ForkJoinPool.java Anyway let me close the PR for now. Thanks all for spending time to review, and thanks @dongjoon-hyun to let me realize about supporting Scala versions.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
