I think that with Ansible syntax, you need to specify: AWS_DEFAULT_ACL: I am fairly certain the lack of any value at the end would get interpreted as Python None.
- Mike On Sat, Nov 9, 2019 at 6:57 AM David Davis <davidda...@redhat.com> wrote: > You should be able to set aws_default_acl to null (ie "aws_default_acl: ~" > or "aws_default_acl: null"). Sounds like that's not working though. What > error did you get? Would you mind filing an issue? > > https://pulp.plan.io/projects/pulp/issues/new > > Thanks. > > David > > > On Fri, Nov 8, 2019 at 10:20 PM Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca> > wrote: > >> I encounted a new issue that escaped my notice by some weird coincidence. >> >> >> It turns that you use the pulp_settings section of the playbook, >> aws_default_acl: None gets converted to AWS_DEFAULT_ACL = "None", instead >> of AWS_DEFAULT_ACL = None, which results in the cryptic >> >> >> "An error occurred (InvalidArgument) when calling the PutObject >> operation: None" >> >> Is there a way to ensure that Ansible converts aws_default_acl: None to >> AWS_DEFAULT_ACL = None? >> >> I tried setting it to null but I got an error. >> >> Cheers, >> >> >> >> Joey Dumont >> >> Technical Advisor, Knowledge, Information, and Technology Services >> National Research Council Canada / Governement of Canada >> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436 >> >> Conseiller technique, Services du savoir, de l'information et de la >> technologie >> Conseil national de recherches Canada / Gouvernement du Canada >> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.: >> 438-340-7436 >> ------------------------------ >> *From:* pulp-list-boun...@redhat.com <pulp-list-boun...@redhat.com> on >> behalf of Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca> >> *Sent:* 06 November 2019 11:36 >> *To:* Mike DePaulo >> *Cc:* pulp-list@redhat.com >> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running >> pulp on Amazon EC2 (pulp3) >> >> >> PR, as suggested: https://github.com/pulp/pulpcore/pull/366. >> >> >> Cheers, >> >> >> Joey Dumont >> >> Technical Advisor, Knowledge, Information, and Technology Services >> National Research Council Canada / Governement of Canada >> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436 >> >> Conseiller technique, Services du savoir, de l'information et de la >> technologie >> Conseil national de recherches Canada / Gouvernement du Canada >> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.: >> 438-340-7436 >> ------------------------------ >> *From:* Mike DePaulo <mikedep...@redhat.com> >> *Sent:* 06 November 2019 10:52 >> *To:* Dumont, Joey >> *Cc:* David Davis; pulp-list@redhat.com >> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running >> pulp on Amazon EC2 (pulp3) >> >> Hi Joey, >> >> It sounds like aws_default_acl should be documented here then: >> >> https://docs.pulpproject.org/en/3.0/nightly/installation/storage.html?highlight=aws >> >> Care to submit a documentation PR? >> https://github.com/pulp/pulpcore/blob/master/docs/installation/storage.rst >> >> Thanks, >> -Mike >> >> On Wed, Nov 6, 2019 at 9:07 AM Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca> >> wrote: >> >>> Turns out the issue was on my end. I had to add aws_default_acl: None to >>> the pulp_settings section of the playbook. The public-read ACL was >>> incompatible with the BlockPublicAccess settings that I had set on my S3 >>> bucket. >>> >>> I'm now encountering a different issue, but I'll start another thread >>> for that one. >>> >>> Thanks for the pointers, they were very helpful! >>> >>> Joey Dumont >>> >>> Technical Advisor, Knowledge, Information, and Technology Services >>> National Research Council Canada / Governement of Canada >>> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436 >>> >>> Conseiller technique, Services du savoir, de l'information et de la >>> technologie >>> Conseil national de recherches Canada / Gouvernement du Canada >>> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.: >>> 438-340-7436 >>> ------------------------------ >>> *From:* David Davis <davidda...@redhat.com> >>> *Sent:* 01 November 2019 15:51 >>> *To:* Dumont, Joey >>> *Cc:* pulp-list@redhat.com >>> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running >>> pulp on Amazon EC2 (pulp3) >>> >>> Unfortunately I don't know of a good way to debug the problem other than >>> to dig into the code. If you want to debug from the Pulp code, you could >>> stick a debugger in the artifact saver stage: >>> >>> >>> https://github.com/pulp/pulpcore/blob/2203fee1407738a4ddd8e644fcbc741aab0bca63/pulpcore/plugin/stages/artifact_stages.py#L179-L200 >>> >>> What I would probably do though is stick a debug statement here in >>> django-storages to see what params it's passing to boto3: >>> >>> >>> https://github.com/jschneier/django-storages/blob/0ab2b1e3efd2bcaf0f24540a718993acc7742d9b/storages/backends/s3boto3.py#L511 >>> >>> You can see the location of django-storages with `pip show >>> django-storages`. >>> >>> Sorry I don't have a better answer for you. Perhaps this is something we >>> can improve in the future. Also, I'd be curious as to what the issue is as >>> it sounds like everything should work in theory. >>> >>> David >>> >>> >>> On Fri, Nov 1, 2019 at 2:26 PM Dumont, Joey <joey.dum...@nrc-cnrc.gc.ca> >>> wrote: >>> >>>> I've installed the latest pulp3 using the Ansible installer using the >>>> following playbook: >>>> >>>> >>>> --- >>>> - hosts: mirrors >>>> vars: >>>> prereq_pip_packages: >>>> - django-storages >>>> - boto3 >>>> pulp_use_system_wide_pkgs: True >>>> pulp_default_admin_password: !vault | >>>> $ANSIBLE_VAULT;1.1;AES256 >>>> ... >>>> pulp_settings: >>>> secret_key: !vault | >>>> $ANSIBLE_VAULT;1.1;AES256 >>>> ... >>>> default_file_storage: 'storages.backends.s3boto3.S3Boto3Storage' >>>> aws_storage_bucket_name: 'xxx-pulp-storage' >>>> aws_s3_region_name: 'ca-central-1' >>>> aws_s3_addressing_style: "path" >>>> media_root: '/pulp3/' >>>> pulp_install_plugins: >>>> pulp-file: {} >>>> pulp-rpm: >>>> prereq_role: "pulp.pulp_rpm_prerequisites" >>>> # pulp-docker: {} >>>> roles: >>>> - pulp-database >>>> - pulp-workers >>>> - pulp-resource-manager >>>> - pulp-webserver >>>> - pulp-content >>>> environment: >>>> DJANGO_SETTINGS_MODULE: pulpcore.app.settings >>>> >>>> I also set up an RPM repo that uses S3 for storage. However, when I try >>>> to sync, I get an AccessDenied error. I know the instance profile is >>>> correct, as I can upload objects from that instance using both the AWS CLI >>>> and Boto3 without specifying credentials. >>>> >>>> How can I debug this further? Is there a way for me know what >>>> parameters are passed to the put_object boto3 call by the sync task? >>>> >>>> Cheers, >>>> >>>> >>>> >>>> Joey Dumont >>>> >>>> Technical Advisor, Knowledge, Information, and Technology Services >>>> National Research Council Canada / Governement of Canada >>>> joey.dum...@nrc-cnrc.gc.ca / Tel: 613-990-8152 / Cell: 438-340-7436 >>>> >>>> Conseiller technique, Services du savoir, de l'information et de la >>>> technologie >>>> Conseil national de recherches Canada / Gouvernement du Canada >>>> joey.dum...@nrc-cnrc.gc.ca / Tél.: 613-990-8152 / Tél. cell.: >>>> 438-340-7436 >>>> _______________________________________________ >>>> Pulp-list mailing list >>>> Pulp-list@redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-list >>> >>> _______________________________________________ >>> Pulp-list mailing list >>> Pulp-list@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> >> >> -- >> >> Mike DePaulo >> >> He / Him / His >> >> Service Reliability Engineer, Pulp >> >> Red Hat <https://www.redhat.com/> >> >> IM: mikedep333 >> >> GPG: 51745404 >> <https://www.redhat.com/> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > > -- Mike DePaulo He / Him / His Service Reliability Engineer, Pulp Red Hat <https://www.redhat.com/> IM: mikedep333 GPG: 51745404 <https://www.redhat.com/>
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list