No problems, I think you have the picture right, but might have missed my
earlier question: do I need to install Boto on the target server?
So, yes:
- All playbooks are run on my laptop
- A playbook aws-start.yml first creates the EC2 instance. It operates
on the localhost
- A playbook provision.yml then attempts to connect to the S3 bucket.
It operates on the EC2 instance
Andrew
On Friday, 19 June 2015 17:30:23 UTC+10, benno joy wrote:
>
> Hi Andrew,
>
> Sorry if i am understanding this wrong, i assume you already have an ec2
> instance which has an iam role attached which gives it access to download
> buckets/files from S3 right ? and in your playbook you have an s3 task
> which runs on this target server which has boto and python installed , so i
> am a bit confused as to why you would need to reinstall ansible boto etc..
> on your local macbook
> Probalby if you can attach your playbook it might make things clear.
>
>
> - Benno
>
>
> On Fri, Jun 19, 2015 at 12:53 PM, Andrew Burrow <
> [email protected] <javascript:>> wrote:
>
>> Just a follow up. I tried two more scenarios, the second being the
>> boil-the-ocean approach :-)
>>
>> 1. I deactivated the virtual environment, and reinstalled Ansible and
>> Boto to /usr/local using Homebrew and Pip as follows:
>>
>> brew install ansible
>> pip install boto==2.38.0
>>
>> I then reran the playbook, and got the same error message, but again I
>> was able to execute the s3 and cloudformation tasks locally.
>>
>> 2. I set my path to a minimum, uninstalled all the Homebrew python, and
>> reinstalled Ansible and Boto using pip into the system as follows:
>>
>> PATH="/usr/local/bin:/usr/bin:/bin"
>> pip uninstall boto
>> brew uninstall ansible
>> brew uninstall python
>> curl -O https://bootstrap.pypa.io/get-pip.py
>> sudo python2.7 get-pip.py
>> sudo pip install six
>> sudo pip install boto
>> sudo pip install ansible
>>
>> I then reran the playbook, and got the same error message, but again I
>> was able to execute the s3 and cloudformation tasks locally.
>>
>>
>> Thanks
>>
>> Andrew
>>
>>
>> On Friday, 19 June 2015 16:52:53 UTC+10, Andrew Burrow wrote:
>>>
>>> Thanks Benno,
>>>
>>> I install Ansible and Boto in a virtualenv using pip, and then add the
>>> following to group_vars/localhosts.yml, which is enough to ensure that
>>> the cloudformation, s3, and ec2 modules run on the localhost. Do I
>>> need to also install Boto on the remote?
>>>
>>> # Do not use the system installed Python when running locally
>>> ansible_python_interpreter: python
>>>
>>> The exact set of packages is:
>>>
>>> Jinja2==2.7.3
>>> MarkupSafe==0.23
>>> PyYAML==3.11
>>> ansible==1.9.1
>>> boto==2.38.0
>>> ecdsa==0.13
>>> paramiko==1.15.2
>>> pycrypto==2.6.1
>>> six==1.9.0
>>> wsgiref==0.1.2
>>>
>>> regards
>>>
>>> Andrew
>>>
>>> On Friday, 19 June 2015 15:44:43 UTC+10, benno joy wrote:
>>>>
>>>> Hi Andrew,
>>>>
>>>> instance profiles do work without any issues, from the error msg:
>>>> Failed to connect to S3: 'module' object has no attribute
>>>> 'connect_to_region'
>>>>
>>>> seems like boto is not installed properly, how did you install boto ?
>>>> can you please try reinstalling boto and check.
>>>>
>>>> - Benno
>>>>
>>>>
>>>> On Fri, Jun 19, 2015 at 9:51 AM, Andrew Burrow <
>>>> [email protected]> wrote:
>>>>
>>>>> I am unable to make use of IAM roles in my Ansible playbooks.
>>>>> Specifically, I have authorised an EC2 instance to get from an S3 bucket,
>>>>> but I cannot work out how to make use of this authorisation from within
>>>>> Ansible.
>>>>>
>>>>>
>>>>> *The question*
>>>>>
>>>>> How do I write Ansible task(s) that satisfies all the following :
>>>>>
>>>>> 1. Runs on an EC2 instance
>>>>> 2. Uses the IAM role defined on the EC2 instance to obtain
>>>>> authorisation to access an S3 bucket
>>>>> 3. Gets a file from the S3 bucket
>>>>>
>>>>>
>>>>> *A work around*
>>>>>
>>>>> I can get the EC2 instance to download from S3, only by passing in my
>>>>> credentials as follows:
>>>>>
>>>>> - name: Download the part archive from S3
>>>>> s3:
>>>>> aws_access_key: "{{ lookup('env','aws_key') }}"
>>>>> aws_secret_key: "{{ lookup('env','aws_secret') }}"
>>>>> region: "{{ aws_packages_region }}"
>>>>> bucket: "{{ aws_packages_bucket }}"
>>>>> object: "/JI79IML/my_part_X86_64_c7.15.tar.gz"
>>>>> dest: "/data/parts/JI79IML/my_part_X86_64_c7.15.tar.gz"
>>>>> mode: get
>>>>> overwrite: no
>>>>>
>>>>> However, I would rather not send my AWS credentials to the instance.
>>>>> Instead I have defined a role with the appropriate permissions to get
>>>>> files
>>>>> from the S3 bucket.
>>>>>
>>>>>
>>>>> *What I've tried*
>>>>>
>>>>> The top answer in the stack overflow question linked below, suggests
>>>>> that it is simple matter of leaving the secret access key parameters out,
>>>>> and letting the Boto library take care of assuming the role.
>>>>>
>>>>> - http://stackoverflow.com/questions/28997757/ansible-and-s3-module
>>>>>
>>>>> However, when I try this with Ansible 1.8.4 and Boto 2.36.0 I get
>>>>>
>>>>> msg: No handler was ready to authenticate. 1 handlers were checked. [
>>>>> 'HmacAuthV1Handler'] Check your credentials
>>>>>
>>>>> and with Ansible 1.9.1 and Boto 2.38.0 I get:
>>>>>
>>>>> msg: Failed to connect to S3: 'module' object has no attribute
>>>>> 'connect_to_region'
>>>>>
>>>>>
>>>>> *How I've confirmed the IAM role*
>>>>>
>>>>> To confirm that the IAM role is *sufficient*, I installed awscli on
>>>>> the EC2 instance and performed the download directly. First, I assumed
>>>>> the
>>>>> role
>>>>>
>>>>> aws sts assume-role --role-arn "${ROLE_ARN}" --role-session-name
>>>>> "GettingMyPart"
>>>>>
>>>>> which returns an absolutely baffling error message that the user with
>>>>> the assumed role cannot assume the role?!? But seems to do the trick,
>>>>> because I can then download the part
>>>>>
>>>>> aws s3api get-object --bucket "${BUCKET_NAME}" --key JI79IML/
>>>>> my_part_X86_64_c7.15.tar.gz my_part_X86_64_c7.15.tar.gz
>>>>>
>>>>> To confirm that the IAM role is *required*, I created another
>>>>> instance that does not enjoy a role and installed awscli on this
>>>>> second EC2 instance and followed the above steps. In each case, I got
>>>>> the
>>>>> message "Unable to locate credentials" as expected
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Ansible Project" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To post to this group, send email to [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/ansible-project/550cc437-c0b2-4999-8710-cf87e28f45e6%40googlegroups.com
>>>>>
>>>>> <https://groups.google.com/d/msgid/ansible-project/550cc437-c0b2-4999-8710-cf87e28f45e6%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Ansible Project" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/ansible-project/ee82581e-c91d-4f22-8f3a-02ecfea51cd5%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/ansible-project/ee82581e-c91d-4f22-8f3a-02ecfea51cd5%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/0e19302d-9955-4194-a145-e9f891b991d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
