Hi Andrew,
If the s3 task is running on the target server, then the provisioned instance needs to have boto installed. but if boto is not installed you should have got a message like "boto is not installed on this machine etc.." maybe an incomplete boto installation on target server ? - Benno On Fri, Jun 19, 2015 at 1:11 PM, Andrew Burrow < [email protected]> wrote: > No problems, I think you have the picture right, but might have missed my > earlier question: do I need to install Boto on the target server? > > So, yes: > > - All playbooks are run on my laptop > - A playbook aws-start.yml first creates the EC2 instance. It > operates on the localhost > - A playbook provision.yml then attempts to connect to the S3 bucket. > It operates on the EC2 instance > > Andrew > > On Friday, 19 June 2015 17:30:23 UTC+10, benno joy wrote: >> >> Hi Andrew, >> >> Sorry if i am understanding this wrong, i assume you already have an ec2 >> instance which has an iam role attached which gives it access to download >> buckets/files from S3 right ? and in your playbook you have an s3 task >> which runs on this target server which has boto and python installed , so i >> am a bit confused as to why you would need to reinstall ansible boto etc.. >> on your local macbook >> Probalby if you can attach your playbook it might make things clear. >> >> >> - Benno >> >> >> On Fri, Jun 19, 2015 at 12:53 PM, Andrew Burrow < >> [email protected]> wrote: >> >>> Just a follow up. I tried two more scenarios, the second being the >>> boil-the-ocean approach :-) >>> >>> 1. I deactivated the virtual environment, and reinstalled Ansible and >>> Boto to /usr/local using Homebrew and Pip as follows: >>> >>> brew install ansible >>> pip install boto==2.38.0 >>> >>> I then reran the playbook, and got the same error message, but again I >>> was able to execute the s3 and cloudformation tasks locally. >>> >>> 2. I set my path to a minimum, uninstalled all the Homebrew python, and >>> reinstalled Ansible and Boto using pip into the system as follows: >>> >>> PATH="/usr/local/bin:/usr/bin:/bin" >>> pip uninstall boto >>> brew uninstall ansible >>> brew uninstall python >>> curl -O https://bootstrap.pypa.io/get-pip.py >>> sudo python2.7 get-pip.py >>> sudo pip install six >>> sudo pip install boto >>> sudo pip install ansible >>> >>> I then reran the playbook, and got the same error message, but again I >>> was able to execute the s3 and cloudformation tasks locally. >>> >>> >>> Thanks >>> >>> Andrew >>> >>> >>> On Friday, 19 June 2015 16:52:53 UTC+10, Andrew Burrow wrote: >>>> >>>> Thanks Benno, >>>> >>>> I install Ansible and Boto in a virtualenv using pip, and then add the >>>> following to group_vars/localhosts.yml, which is enough to ensure that >>>> the cloudformation, s3, and ec2 modules run on the localhost. Do I >>>> need to also install Boto on the remote? >>>> >>>> # Do not use the system installed Python when running locally >>>> ansible_python_interpreter: python >>>> >>>> The exact set of packages is: >>>> >>>> Jinja2==2.7.3 >>>> MarkupSafe==0.23 >>>> PyYAML==3.11 >>>> ansible==1.9.1 >>>> boto==2.38.0 >>>> ecdsa==0.13 >>>> paramiko==1.15.2 >>>> pycrypto==2.6.1 >>>> six==1.9.0 >>>> wsgiref==0.1.2 >>>> >>>> regards >>>> >>>> Andrew >>>> >>>> On Friday, 19 June 2015 15:44:43 UTC+10, benno joy wrote: >>>>> >>>>> Hi Andrew, >>>>> >>>>> instance profiles do work without any issues, from the error msg: >>>>> Failed to connect to S3: 'module' object has no attribute >>>>> 'connect_to_region' >>>>> >>>>> seems like boto is not installed properly, how did you install boto ? >>>>> can you please try reinstalling boto and check. >>>>> >>>>> - Benno >>>>> >>>>> >>>>> On Fri, Jun 19, 2015 at 9:51 AM, Andrew Burrow < >>>>> [email protected]> wrote: >>>>> >>>>>> I am unable to make use of IAM roles in my Ansible playbooks. >>>>>> Specifically, I have authorised an EC2 instance to get from an S3 bucket, >>>>>> but I cannot work out how to make use of this authorisation from within >>>>>> Ansible. >>>>>> >>>>>> >>>>>> *The question* >>>>>> >>>>>> How do I write Ansible task(s) that satisfies all the following : >>>>>> >>>>>> 1. Runs on an EC2 instance >>>>>> 2. Uses the IAM role defined on the EC2 instance to obtain >>>>>> authorisation to access an S3 bucket >>>>>> 3. Gets a file from the S3 bucket >>>>>> >>>>>> >>>>>> *A work around* >>>>>> >>>>>> I can get the EC2 instance to download from S3, only by passing in my >>>>>> credentials as follows: >>>>>> >>>>>> - name: Download the part archive from S3 >>>>>> s3: >>>>>> aws_access_key: "{{ lookup('env','aws_key') }}" >>>>>> aws_secret_key: "{{ lookup('env','aws_secret') }}" >>>>>> region: "{{ aws_packages_region }}" >>>>>> bucket: "{{ aws_packages_bucket }}" >>>>>> object: "/JI79IML/my_part_X86_64_c7.15.tar.gz" >>>>>> dest: "/data/parts/JI79IML/my_part_X86_64_c7.15.tar.gz" >>>>>> mode: get >>>>>> overwrite: no >>>>>> >>>>>> However, I would rather not send my AWS credentials to the >>>>>> instance. Instead I have defined a role with the appropriate permissions >>>>>> to get files from the S3 bucket. >>>>>> >>>>>> >>>>>> *What I've tried* >>>>>> >>>>>> The top answer in the stack overflow question linked below, suggests >>>>>> that it is simple matter of leaving the secret access key parameters out, >>>>>> and letting the Boto library take care of assuming the role. >>>>>> >>>>>> - >>>>>> http://stackoverflow.com/questions/28997757/ansible-and-s3-module >>>>>> >>>>>> However, when I try this with Ansible 1.8.4 and Boto 2.36.0 I get >>>>>> >>>>>> msg: No handler was ready to authenticate. 1 handlers were checked. [ >>>>>> 'HmacAuthV1Handler'] Check your credentials >>>>>> >>>>>> and with Ansible 1.9.1 and Boto 2.38.0 I get: >>>>>> >>>>>> msg: Failed to connect to S3: 'module' object has no attribute >>>>>> 'connect_to_region' >>>>>> >>>>>> >>>>>> *How I've confirmed the IAM role* >>>>>> >>>>>> To confirm that the IAM role is *sufficient*, I installed awscli on >>>>>> the EC2 instance and performed the download directly. First, I assumed >>>>>> the >>>>>> role >>>>>> >>>>>> aws sts assume-role --role-arn "${ROLE_ARN}" --role-session-name >>>>>> "GettingMyPart" >>>>>> >>>>>> which returns an absolutely baffling error message that the user with >>>>>> the assumed role cannot assume the role?!? But seems to do the trick, >>>>>> because I can then download the part >>>>>> >>>>>> aws s3api get-object --bucket "${BUCKET_NAME}" --key JI79IML/ >>>>>> my_part_X86_64_c7.15.tar.gz my_part_X86_64_c7.15.tar.gz >>>>>> >>>>>> To confirm that the IAM role is *required*, I created another >>>>>> instance that does not enjoy a role and installed awscli on this >>>>>> second EC2 instance and followed the above steps. In each case, I got >>>>>> the >>>>>> message "Unable to locate credentials" as expected >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Ansible Project" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/ansible-project/550cc437-c0b2-4999-8710-cf87e28f45e6%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/ansible-project/550cc437-c0b2-4999-8710-cf87e28f45e6%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Ansible Project" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/ansible-project/ee82581e-c91d-4f22-8f3a-02ecfea51cd5%40googlegroups.com >>> <https://groups.google.com/d/msgid/ansible-project/ee82581e-c91d-4f22-8f3a-02ecfea51cd5%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Ansible Project" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/ansible-project/0e19302d-9955-4194-a145-e9f891b991d6%40googlegroups.com > <https://groups.google.com/d/msgid/ansible-project/0e19302d-9955-4194-a145-e9f891b991d6%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAFUV_d4WcxYg%3DhnYSEEUMCf9PSPS1Nry6SkRYyusYSSinHKYUQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
