wangshisan edited a comment on issue #25869: [SPARK-29189][SQL] Add an option to ignore block locations when listing file URL: https://github.com/apache/spark/pull/25869#issuecomment-533793656 Sorry, I didn't made myself clear. Our Spark cluster is deployed separated from HDFS cluster, all the data stored in another HDFS cluster, and these two clusters share no nodes. I mean such deploy mode, separating Spark cluster with the storage cluster(HDFS or some other distributed file systems), is pretty common. And in such Spark cluster, data locality is non sense, because it's non reachable.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
