GitHub user JuhongPark opened a pull request:
https://github.com/apache/spark/pull/7252
[SPARK-8863] [EC2] Check aws access key from aws credentials if there is no
boto config
'spark_ec2.py' use boto to control ec2.
And boto can support '~/.aws/credentials' which is AWS CLI default
configuration file.
We can check this information from ref of boto.
"A boto config file is a text file formatted like an .ini configuration
file that specifies values for options that control the behavior of the boto
library. In Unix/Linux systems, on startup, the boto library looks for
configuration files in the following locations and in the following order:
/etc/boto.cfg - for site-wide settings that all users on this machine will
use
(if profile is given) ~/.aws/credentials - for credentials shared between
SDKs
(if profile is given) ~/.boto - for user-specific settings
~/.aws/credentials - for credentials shared between SDKs
~/.boto - for user-specific settings"
* ref of boto: http://boto.readthedocs.org/en/latest/boto_config_tut.html
* ref of aws cli :
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
However 'spark_ec2.py' only check boto config & environment variable even
if there is '~/.aws/credentials', and 'spark_ec2.py' is terminated.
So I changed to check '~/.aws/credentials'.
cc @rxin
Jira : https://issues.apache.org/jira/browse/SPARK-8863
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JuhongPark/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7252.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7252
----
commit 23c57928aed6ae0dfeb00d7fad05128f15287b7d
Author: JPark <[email protected]>
Date: 2015-07-06T13:36:07Z
Check aws access key from aws credentials if there is no boto config
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
