[GitHub] HyukjinKwon commented on issue #23402: [SPARK-26500][CORE] Add conf to support ingoring data locality
HyukjinKwon commented on issue #23402: [SPARK-26500][CORE] Add conf to support ingoring data locality URL: https://github.com/apache/spark/pull/23402#issuecomment-450536703 I'm closing this. @printstacktrace, please let me know if I missed how much the configuration improves the time-consuming logic. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23402: [SPARK-26500][CORE] Add conf to support ingoring data locality
HyukjinKwon commented on issue #23402: [SPARK-26500][CORE] Add conf to support ingoring data locality URL: https://github.com/apache/spark/pull/23402#issuecomment-450480604 ?? Wouldn't it work if we set zero for `spark.locality.wait`? That's what I did long time ago as well. > It's single thread and time-consuming when there are huge partitions How much does it take, and how much time this configuration saves? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
