xuanyuanking commented on issue #25491: [SPARK-28699][SQL] Disable using radix sort for ShuffleExchangeExec in repartition case URL: https://github.com/apache/spark/pull/25491#issuecomment-523727865 Thank you all for the discussion and guidance. After discussing with Wenchen, while we finish the work of SPARK-25341: support rolling back a shuffle map stage and re-generate the shuffle files, we'll come back to optimize this task. The optimization strategy is disabled sortBeforeRepartition by default. Only enable it if indeterminate stage(in this case, is the stage caused by repartition) rerun happened. Fill Jira SPARK-28845 to track this.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
