Hi Sudhir, I believe you have to use a shared file system that is accused by all nodes.
> On Jun 24, 2017, at 1:30 PM, sudhir k <k.sudhi...@gmail.com> wrote: > > > I am new to Spark and i need some guidance on how to fetch files from --files > option on Spark-Submit. > > I read on some forums that we can fetch the files from > Spark.getFiles(fileName) and can use it in our code and all nodes should read > it. > > But i am facing some issue > > Below is the command i am using > > spark-submit --deploy-mode cluster --class com.check.Driver --files > /home/sql/first.sql test.jar 20170619 > > so when i use SparkFiles.get(first.sql) , i should be able to read the file > Path but it is throwing File not Found exception. > > I tried SpackContext.addFile(/home/sql/first.sql) and then > SparkFiles.get(first.sql) but still the same error. > > Its working on the stand alone mode but not on cluster mode. Any help is > appreciated.. Using Spark 2.1.0 and Scala 2.11 > > Thanks. > > > > Regards, > Sudhir K > > > > -- > Regards, > Sudhir K
