Github user acvogel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5109#discussion_r27145944
--- Diff: ec2/spark_ec2.py ---
@@ -528,32 +532,53 @@ def launch_cluster(conn, opts, cluster_name):
name = '/dev/sd' + string.letters[i + 1
Github user acvogel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5109#discussion_r27149279
--- Diff: ec2/spark_ec2.py ---
@@ -567,16 +592,28 @@ def launch_cluster(conn, opts, cluster_name):
for i in my_req_ids
Github user acvogel commented on the pull request:
https://github.com/apache/spark/pull/5109#issuecomment-86156199
The use case which led to this patch is for long-running jobs with few
slaves where we require a large master instance type. For instance, training
decision trees using
Github user acvogel commented on the pull request:
https://github.com/apache/spark/pull/5109#issuecomment-85782112
I added the support for the on-demand slaves / spot master combination. I
tried out all four combinations mentioned above and encountered no problems. I
also tested
Github user acvogel commented on the pull request:
https://github.com/apache/spark/pull/5109#issuecomment-85339056
@chammas Good question! I had tested all combinations except for the case
where the master is a spot instance and the slaves are on-demand instances.
There is a bug
GitHub user acvogel opened a pull request:
https://github.com/apache/spark/pull/5109
[SPARK-6403][EC2] Add option to launch EC2 master as a spot instance
instead of on demand.
Adds option to launch EC2 master as a spot instance instead of on demand.
Introduces a new command line
