Yes, that's how file: URLs are interpreted everywhere in Spark. (It's also explained in the link to the docs I posted earlier.)
The second interpretation below is "local:" URLs in Spark, but that doesn't work with Yarn on Spark 1.0 (so it won't work with CDH 5.1 and older either). On Mon, Sep 8, 2014 at 6:00 PM, Dimension Data, LLC. < [email protected]> wrote: > Even when using 'file:///...' nomenclature in SPARK_JAR (instead of > through the yet-to-be implemented 'spark.yarn.jar'\ > property), it's interpretation still seems to be: > > 'Tell me where the local spark jar is located so that I can upload it > (i.e. hdfs dfs -put) to a HDFS staging area for you'. > -as oppoese to- > 'Tell me where the local spark jar is located on the NMs, and I will > look for it at that UNIX path'. > -- Marcelo
