[
https://issues.apache.org/jira/browse/SPARK-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen resolved SPARK-3405.
-------------------------------
Resolution: Fixed
Fix Version/s: 1.3.0
Issue resolved by pull request 2872
[https://github.com/apache/spark/pull/2872]
> EC2 cluster creation on VPC
> ---------------------------
>
> Key: SPARK-3405
> URL: https://issues.apache.org/jira/browse/SPARK-3405
> Project: Spark
> Issue Type: New Feature
> Components: EC2
> Affects Versions: 1.0.2
> Environment: Ubuntu 12.04
> Reporter: Dawson Reid
> Priority: Minor
> Fix For: 1.3.0
>
>
> It would be very useful to be able to specify the EC2 VPC in which the Spark
> cluster should be created.
> When creating a Spark cluster on AWS via the spark-ec2 script there is no way
> to specify a VPC id of the VPC you would like the cluster to be created in.
> The script always creates the cluster in the default VPC.
> In my case I have deleted the default VPC and the spark-ec2 script errors out
> with the following :
> Setting up security groups...
> Creating security group test-master
> ERROR:boto:400 Bad Request
> ERROR:boto:<?xml version="1.0" encoding="UTF-8"?>
> <Response><Errors><Error><Code>VPCIdNotSpecified</Code><Message>No default
> VPC for this
> user</Message></Error></Errors><RequestID>312a2281-81a1-4d3c-ba10-0593a886779d</RequestID></Response>
> Traceback (most recent call last):
> File "./spark_ec2.py", line 860, in <module>
> main()
> File "./spark_ec2.py", line 852, in main
> real_main()
> File "./spark_ec2.py", line 735, in real_main
> conn, opts, cluster_name)
> File "./spark_ec2.py", line 247, in launch_cluster
> master_group = get_or_make_group(conn, cluster_name + "-master")
> File "./spark_ec2.py", line 143, in get_or_make_group
> return conn.create_security_group(name, "Spark EC2 group")
> File
> "/home/dawson/Develop/spark-1.0.2/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/ec2/connection.py",
> line 2011, in create_security_group
> File
> "/home/dawson/Develop/spark-1.0.2/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/connection.py",
> line 925, in get_object
> boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request
> <?xml version="1.0" encoding="UTF-8"?>
> <Response><Errors><Error><Code>VPCIdNotSpecified</Code><Message>No default
> VPC for this
> user</Message></Error></Errors><RequestID>312a2281-81a1-4d3c-ba10-0593a886779d</RequestID></Response>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
