Re: Error with --files

That fixed it! Thank you! --Ben On Thu, Apr 14, 2016 at 5:53 PM, Marcelo Vanzin wrote: > On Thu, Apr 14, 2016 at 2:14 PM, Benjamin Zaitlen > wrote: > >> spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files > >>

Re: Error with --files

On Thu, Apr 14, 2016 at 2:14 PM, Benjamin Zaitlen wrote: >> spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files >> /home/ubuntu/localtest.txt#appSees.txt --files should come before the path to your python script. Otherwise it's just passed as arguments to

Re: Error with --files

bq. localtest.txt#appSees.txt Which file did you want to pass ? Thanks On Thu, Apr 14, 2016 at 2:14 PM, Benjamin Zaitlen wrote: > Hi All, > > I'm trying to use the --files option with yarn: > > spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files >>

Error with --files

Hi All, I'm trying to use the --files option with yarn: spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files > /home/ubuntu/localtest.txt#appSees.txt I never see the file in HDFS or in the yarn containers. Am I doing something incorrect ? I'm running spark 1.6.0 Thanks,