cloud-fan edited a comment on issue #25491: [SPARK-28699][Core] Disable using radix sort for ShuffleExchangeExec in repartition case URL: https://github.com/apache/spark/pull/25491#issuecomment-522475053 cc @kiszk @dongjoon-hyun since you are the release managers. I'd like to fix this in 2.3/2.4 as it's a correctness bug. cc @jiangxb1987 @tgravescs , I think this is the right fix, but might be too expensive as normal sort is much slower than radix sort. I'm a little worried about introducing a big perf regression to 2.3/2.4. The [new fix](https://github.com/apache/spark/pull/22112) of the repartition bug was introduced at Spark 2.4 and works well so far. Shall we just disable the sort and always use the new fix?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
