user@spark.apache.org<mailto:user@spark.apache.org>
Subject: RE: Cluster mode deployment from jar in S3
Date: Mon, 4 Jul 2016 09:50:50 +
Hi,
The aws CLI already has your access key aid and secret access
key when you initially configured it.
Is y
lohith.sam...@mphasis.com; user@spark.apache.org
Subject: RE: Cluster mode deployment from jar in S3
Date: Mon, 4 Jul 2016 11:30:31 +0100
Hi Lohith,Thanks for the response.
The S3 bucket does have access restrictions, but the instances in which the
Spark master and workers run have an IAM role policy t
n't put it in application code, as the issue is in
downloading the jar from S3.
-Ashic.
From: lohith.sam...@mphasis.com
To: as...@live.com; user@spark.apache.org
Subject: RE: Cluster mode deployment from jar in S3
Date: Mon, 4 Jul 2016 09:50:50 +
Hi,
The
aws CLI a
[mailto:as...@live.com]
Sent: Monday, July 04, 2016 15.06
To: Apache Spark
Subject: RE: Cluster mode deployment from jar in S3
Sorry to do this...but... *bump*
From: as...@live.com<mailto:as...@live.com>
To: user@spark.apache.org<mailto:user@spark.apache.org
Sorry to do this...but... *bump*
From: as...@live.com
To: user@spark.apache.org
Subject: Cluster mode deployment from jar in S3
Date: Fri, 1 Jul 2016 17:45:12 +0100
Hello,I've got a Spark stand-alone cluster using EC2 instances. I can submit
jobs using "--deploy-mode client"
Hello,I've got a Spark stand-alone cluster using EC2 instances. I can submit
jobs using "--deploy-mode client", however using "--deploy-mode cluster" is
proving to be a challenge. I've tries this:
spark-submit --class foo --master spark:://master-ip:7077 --deploy-mode cluster
s3://bucket/dir/foo
