That fixed it!
Thank you!
--Ben
On Thu, Apr 14, 2016 at 5:53 PM, Marcelo Vanzin wrote:
> On Thu, Apr 14, 2016 at 2:14 PM, Benjamin Zaitlen
> wrote:
> >> spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files
> >> /home/ubuntu/localtest.txt#appSees.txt
>
> --files should come be
On Thu, Apr 14, 2016 at 2:14 PM, Benjamin Zaitlen wrote:
>> spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files
>> /home/ubuntu/localtest.txt#appSees.txt
--files should come before the path to your python script. Otherwise
it's just passed as arguments to your script when it's r
bq. localtest.txt#appSees.txt
Which file did you want to pass ?
Thanks
On Thu, Apr 14, 2016 at 2:14 PM, Benjamin Zaitlen
wrote:
> Hi All,
>
> I'm trying to use the --files option with yarn:
>
> spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files
>> /home/ubuntu/localtest.txt#
Hi All,
I'm trying to use the --files option with yarn:
spark-submit --master yarn-cluster /home/ubuntu/test_spark.py --files
> /home/ubuntu/localtest.txt#appSees.txt
I never see the file in HDFS or in the yarn containers. Am I doing
something incorrect ?
I'm running spark 1.6.0
Thanks,
--B