Hi Sudhir,
I believe you have to use a shared file system that is accused by all nodes.
> On Jun 24, 2017, at 1:30 PM, sudhir k wrote:
>
>
> I am new to Spark and i need some guidance on how to fetch files from --files
> option on Spark-Submit.
>
> I read on some forums that we can fetch the files from
> Spark.getFiles(fileName) and can use it in our code and all nodes should read
> it.
>
> But i am facing some issue
>
> Below is the command i am using
>
> spark-submit --deploy-mode cluster --class com.check.Driver --files
> /home/sql/first.sql test.jar 20170619
>
> so when i use SparkFiles.get(first.sql) , i should be able to read the file
> Path but it is throwing File not Found exception.
>
> I tried SpackContext.addFile(/home/sql/first.sql) and then
> SparkFiles.get(first.sql) but still the same error.
>
> Its working on the stand alone mode but not on cluster mode. Any help is
> appreciated.. Using Spark 2.1.0 and Scala 2.11
>
> Thanks.
>
>
>
> Regards,
> Sudhir K
>
>
>
> --
> Regards,
> Sudhir K