[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544136#comment-14544136 ]
Brad Willard commented on SPARK-7640: ------------------------------------- I think this might be working I disabled the amazon repos and then added a centos 6 one sudo yum-config-manager --add-repo http://dev.centos.org/centos/6/SCL/scl.repo > Private VPC with default Spark AMI breaks yum > --------------------------------------------- > > Key: SPARK-7640 > URL: https://issues.apache.org/jira/browse/SPARK-7640 > Project: Spark > Issue Type: Improvement > Components: EC2 > Affects Versions: 1.3.0, 1.3.1 > Reporter: Brad Willard > Priority: Minor > > If you create a spark cluster in a private vpc, the amazon yum repos return > 403 permission denied because Amazon cannot discern the vms are in their > datacenter. This makes it incredibly annoying to install things like python > 2.7 and different compression libs or consider updating anything. > Potential fixes: > Add fedora yum repos on the default ami to ones outside of amazon. > Change the ami to be based on a non amazon ami, like a standard red-hat one. > Switch everything to support ec2-user like most modern aws amis to make it > easier for the user to pick an ami > Petition amazon to open up their repos. > Failed Workaround: > I attempted to use a normal red-hat ami, however the current deploy scripts > assume the user and the install path are root. While the deploy script allows > you to override the user, they don't work if you set ec2-user basically > preventing you from using any current ami other than the default amazon one > which is unfortunate. > So normally this would work, but because amazon 403s you get this if you want > to use python 2.7 > $ yum install -y python27.x86_64 python27-devel.x86_64 python27-pip.noarch > Loaded plugins: priorities, security, update-motd, upgrade-helper > http://packages.us-east-1.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.us-west-1.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.us-west-2.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.eu-west-1.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.ap-southeast-1.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.ap-northeast-1.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.sa-east-1.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. > http://packages.ap-southeast-2.amazonaws.com/2015.03/main/20150301f40d/x86_64/repodata/repomd.xml: > [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 > Forbidden" > Trying other mirror. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org