You should be able to set aws_default_acl to null (ie "aws_default_acl: ~"
or "aws_default_acl: null"). Sounds like that's not working though. What
error did you get? Would you mind filing an issue?

https://pulp.plan.io/projects/pulp/issues/new

Thanks.

David


On Fri, Nov 8, 2019 at 10:20 PM Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca>
wrote:

> I encounted a new issue that escaped my notice by some weird coincidence.
>
>
> It turns that you use the pulp_settings section of the playbook,
> aws_default_acl: None gets converted to AWS_DEFAULT_ACL = "None", instead
> of AWS_DEFAULT_ACL = None, which results in the cryptic
>
>
> "An error occurred (InvalidArgument) when calling the PutObject operation:
> None"
>
> Is there a way to ensure that Ansible converts aws_default_acl: None to
> AWS_DEFAULT_ACL = None?
>
> I tried setting it to null but I got an error.
>
> Cheers,
> ​
>
>
> Joey Dumont
>
> Technical Advisor, Knowledge, Information, and Technology Services
> National Research Council Canada / Governement of Canada
> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436
>
> Conseiller technique, Services du savoir, de l'information et de la
> technologie
> Conseil national de recherches Canada / Gouvernement du Canada
> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436
> ------------------------------
> *From:* pulp-list-boun...@redhat.com <pulp-list-boun...@redhat.com> on
> behalf of Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca>
> *Sent:* 06 November 2019 11:36
> *To:* Mike DePaulo
> *Cc:* pulp-list@redhat.com
> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running pulp
> on Amazon EC2 (pulp3)
>
>
> PR, as suggested: https://github.com/pulp/pulpcore/pull/366​.
>
>
> Cheers,
>
>
> Joey Dumont
>
> Technical Advisor, Knowledge, Information, and Technology Services
> National Research Council Canada / Governement of Canada
> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436
>
> Conseiller technique, Services du savoir, de l'information et de la
> technologie
> Conseil national de recherches Canada / Gouvernement du Canada
> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436
> ------------------------------
> *From:* Mike DePaulo <mikedep...@redhat.com>
> *Sent:* 06 November 2019 10:52
> *To:* Dumont, Joey
> *Cc:* David Davis; pulp-list@redhat.com
> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running pulp
> on Amazon EC2 (pulp3)
>
> Hi Joey,
>
> It sounds like aws_default_acl should be documented here then:
>
> https://docs.pulpproject.org/en/3.0/nightly/installation/storage.html?highlight=aws
>
> Care to submit a documentation PR?
> https://github.com/pulp/pulpcore/blob/master/docs/installation/storage.rst
>
> Thanks,
> -Mike
>
> On Wed, Nov 6, 2019 at 9:07 AM Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca>
> wrote:
>
>> Turns out the issue was on my end. I had to add aws_default_acl: None to
>> the pulp_settings section of the playbook. The public-read ACL was
>> incompatible with the BlockPublicAccess settings that I had set on my S3
>> bucket. ​
>>
>> I'm now encountering a different issue, but I'll start another thread for
>> that one.
>>
>> Thanks for the pointers, they were very helpful!
>>
>> Joey Dumont
>>
>> Technical Advisor, Knowledge, Information, and Technology Services
>> National Research Council Canada / Governement of Canada
>> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436
>>
>> Conseiller technique, Services du savoir, de l'information et de la
>> technologie
>> Conseil national de recherches Canada / Gouvernement du Canada
>> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.:
>> 438-340-7436
>> ------------------------------
>> *From:* David Davis <davidda...@redhat.com>
>> *Sent:* 01 November 2019 15:51
>> *To:* Dumont, Joey
>> *Cc:* pulp-list@redhat.com
>> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running
>> pulp on Amazon EC2 (pulp3)
>>
>> Unfortunately I don't know of a good way to debug the problem other than
>> to dig into the code. If you want to debug from the Pulp code, you could
>> stick a debugger in the artifact saver stage:
>>
>>
>> https://github.com/pulp/pulpcore/blob/2203fee1407738a4ddd8e644fcbc741aab0bca63/pulpcore/plugin/stages/artifact_stages.py#L179-L200
>>
>> What I would probably do though is stick a debug statement here in
>> django-storages to see what params it's passing to boto3:
>>
>>
>> https://github.com/jschneier/django-storages/blob/0ab2b1e3efd2bcaf0f24540a718993acc7742d9b/storages/backends/s3boto3.py#L511
>>
>> You can see the location of django-storages with `pip show
>> django-storages`.
>>
>> Sorry I don't have a better answer for you. Perhaps this is something we
>> can improve in the future. Also, I'd be curious as to what the issue is as
>> it sounds like everything should work in theory.
>>
>> David
>>
>>
>> On Fri, Nov 1, 2019 at 2:26 PM Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca>
>> wrote:
>>
>>> I've installed the latest pulp3 using the Ansible installer using the
>>> following playbook:
>>>
>>>
>>> ---
>>> - hosts: mirrors
>>>   vars:
>>>     prereq_pip_packages:
>>>       - django-storages
>>>       - boto3
>>>     pulp_use_system_wide_pkgs: True
>>>     pulp_default_admin_password: !vault |
>>>           $ANSIBLE_VAULT;1.1;AES256
>>>           ...
>>>     pulp_settings:
>>>       secret_key: !vault |
>>>           $ANSIBLE_VAULT;1.1;AES256
>>>           ...
>>>       default_file_storage: 'storages.backends.s3boto3.S3Boto3Storage'
>>>       aws_storage_bucket_name: 'xxx-pulp-storage'
>>>       aws_s3_region_name: 'ca-central-1'
>>>       aws_s3_addressing_style: "path"
>>>       media_root: '/pulp3/'
>>>     pulp_install_plugins:
>>>       pulp-file: {}
>>>       pulp-rpm:
>>>         prereq_role: "pulp.pulp_rpm_prerequisites"
>>>         #      pulp-docker: {}
>>>   roles:
>>>     - pulp-database
>>>     - pulp-workers
>>>     - pulp-resource-manager
>>>     - pulp-webserver
>>>     - pulp-content
>>>   environment:
>>>     DJANGO_SETTINGS_MODULE: pulpcore.app.settings
>>>
>>> I also set up an RPM repo that uses S3 for storage. However, when I try
>>> to sync, I get an AccessDenied error. I know the instance profile is
>>> correct, as I can upload objects from that instance using both the AWS CLI
>>> and Boto3 without specifying credentials.
>>>
>>> How can I debug this further? Is there a way for me know what parameters
>>> are passed to the put_object boto3 call by the sync task?
>>>
>>> Cheers,
>>>
>>>
>>>
>>> Joey Dumont
>>>
>>> Technical Advisor, Knowledge, Information, and Technology Services
>>> National Research Council Canada / Governement of Canada
>>> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436
>>>
>>> Conseiller technique, Services du savoir, de l'information et de la
>>> technologie
>>> Conseil national de recherches Canada / Gouvernement du Canada
>>> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.:
>>> 438-340-7436
>>> _______________________________________________
>>> Pulp-list mailing list
>>> Pulp-list@redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
>
> --
>
> Mike DePaulo
>
> He / Him / His
>
> Service Reliability Engineer, Pulp
>
> Red Hat <https://www.redhat.com/>
>
> IM: mikedep333
>
> GPG: 51745404
> <https://www.redhat.com/>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list@redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
_______________________________________________
Pulp-list mailing list
Pulp-list@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

Reply via email to