ChenjunZou commented on issue #28168: [SPARK-31395][CORE]reverse preferred location to make schedule more even URL: https://github.com/apache/spark/pull/28168#issuecomment-611629822 @Ngone51 I only want is to make the schedule more even with minor effort. I also said it may not reverse, shuffle is ok. the reverse is because hdfs write pipeline gives enough randomness. > What if others are busy except `xxx.93`? It schedules to xxx.93. It is about fairness. If the writing process is a single point spot. The scheduler may add some random condition for schedule fairness.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
