[ 
https://issues.apache.org/jira/browse/SPARK-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Chammas resolved SPARK-6220.
-------------------------------------
    Resolution: Won't Fix

Resolving this issue as won't fix since it is of low importance and can be 
replaced by a few specific options for important EC2 features that are 
currently missing.

> Allow extended EC2 options to be passed through spark-ec2
> ---------------------------------------------------------
>
>                 Key: SPARK-6220
>                 URL: https://issues.apache.org/jira/browse/SPARK-6220
>             Project: Spark
>          Issue Type: Improvement
>          Components: EC2
>            Reporter: Nicholas Chammas
>            Priority: Minor
>
> There are many EC2 options exposed by the boto library that spark-ec2 uses. 
> Over time, many of these EC2 options have been bubbled up here and there to 
> become spark-ec2 options.
> Examples:
> * spot prices
> * placement groups
> * VPC, subnet, and security group assignments
> It's likely that more and more EC2 options will trickle up like this to 
> become spark-ec2 options.
> While major options are well suited to this type of promotion, we should 
> probably allow users to pass through EC2 options they want to use through 
> spark-ec2 in some generic way.
> Let's add two options:
> * {{--ec2-instance-option}} -> 
> [{{boto::run}}|http://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.image.Image.run]
> * {{--ec2-spot-instance-option}} -> 
> [{{boto::request_spot_instances}}|http://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.request_spot_instances]
> Each option can be specified multiple times and is simply passed directly to 
> the underlying boto call.
> For example:
> {code}
> spark-ec2 \
>     ...
>     --ec2-instance-option "instance_initiated_shutdown_behavior=terminate" \
>     --ec2-instance-option "ebs_optimized=True"
> {code}
> I'm not sure about the exact syntax of the extended options, but something 
> like this will do the trick as long as it can be made to pass the options 
> correctly to boto in most cases.
> I followed the example of {{ssh}}, which supports multiple extended options 
> similarly.
> {code}
> ssh -o LogLevel=ERROR -o UserKnowHostsFile=/dev/null ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to