tgravescs commented on issue #25491: [SPARK-28699][SQL] Disable using radix sort for ShuffleExchangeExec in repartition case URL: https://github.com/apache/spark/pull/25491#issuecomment-523007056 I'm ok either way, I guess it comes down to the performance implications, which is why I asked if anyone had tried yet. If we disable it then it should be back to performant until something fails, which in worse case is worse since your job could fail. At least with the sort it will run just might be slower. for 2.3 and 2.4 I would lean towards leaving the sort on so that users jobs don't start failing unexpectedly and we make sure to document in release notes
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
