You should be able to set aws_default_acl to null (ie "aws_default_acl: ~" or "aws_default_acl: null"). Sounds like that's not working though. What error did you get? Would you mind filing an issue?
https://pulp.plan.io/projects/pulp/issues/new Thanks. David On Fri, Nov 8, 2019 at 10:20 PM Dumont, Joey <[email protected]> wrote: > I encounted a new issue that escaped my notice by some weird coincidence. > > > It turns that you use the pulp_settings section of the playbook, > aws_default_acl: None gets converted to AWS_DEFAULT_ACL = "None", instead > of AWS_DEFAULT_ACL = None, which results in the cryptic > > > "An error occurred (InvalidArgument) when calling the PutObject operation: > None" > > Is there a way to ensure that Ansible converts aws_default_acl: None to > AWS_DEFAULT_ACL = None? > > I tried setting it to null but I got an error. > > Cheers, > > > > Joey Dumont > > Technical Advisor, Knowledge, Information, and Technology Services > National Research Council Canada / Governement of Canada > [email protected] / Tel: 613-990-8152 / Cell: 438-340-7436 > > Conseiller technique, Services du savoir, de l'information et de la > technologie > Conseil national de recherches Canada / Gouvernement du Canada > [email protected] / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436 > ------------------------------ > *From:* [email protected] <[email protected]> on > behalf of Dumont, Joey <[email protected]> > *Sent:* 06 November 2019 11:36 > *To:* Mike DePaulo > *Cc:* [email protected] > *Subject:* Re: [Pulp-list] Issues with using S3 storage when running pulp > on Amazon EC2 (pulp3) > > > PR, as suggested: https://github.com/pulp/pulpcore/pull/366. > > > Cheers, > > > Joey Dumont > > Technical Advisor, Knowledge, Information, and Technology Services > National Research Council Canada / Governement of Canada > [email protected] / Tel: 613-990-8152 / Cell: 438-340-7436 > > Conseiller technique, Services du savoir, de l'information et de la > technologie > Conseil national de recherches Canada / Gouvernement du Canada > [email protected] / Tél.: 613-990-8152 / Tél. cell.: 438-340-7436 > ------------------------------ > *From:* Mike DePaulo <[email protected]> > *Sent:* 06 November 2019 10:52 > *To:* Dumont, Joey > *Cc:* David Davis; [email protected] > *Subject:* Re: [Pulp-list] Issues with using S3 storage when running pulp > on Amazon EC2 (pulp3) > > Hi Joey, > > It sounds like aws_default_acl should be documented here then: > > https://docs.pulpproject.org/en/3.0/nightly/installation/storage.html?highlight=aws > > Care to submit a documentation PR? > https://github.com/pulp/pulpcore/blob/master/docs/installation/storage.rst > > Thanks, > -Mike > > On Wed, Nov 6, 2019 at 9:07 AM Dumont, Joey <[email protected]> > wrote: > >> Turns out the issue was on my end. I had to add aws_default_acl: None to >> the pulp_settings section of the playbook. The public-read ACL was >> incompatible with the BlockPublicAccess settings that I had set on my S3 >> bucket. >> >> I'm now encountering a different issue, but I'll start another thread for >> that one. >> >> Thanks for the pointers, they were very helpful! >> >> Joey Dumont >> >> Technical Advisor, Knowledge, Information, and Technology Services >> National Research Council Canada / Governement of Canada >> [email protected] / Tel: 613-990-8152 / Cell: 438-340-7436 >> >> Conseiller technique, Services du savoir, de l'information et de la >> technologie >> Conseil national de recherches Canada / Gouvernement du Canada >> [email protected] / Tél.: 613-990-8152 / Tél. cell.: >> 438-340-7436 >> ------------------------------ >> *From:* David Davis <[email protected]> >> *Sent:* 01 November 2019 15:51 >> *To:* Dumont, Joey >> *Cc:* [email protected] >> *Subject:* Re: [Pulp-list] Issues with using S3 storage when running >> pulp on Amazon EC2 (pulp3) >> >> Unfortunately I don't know of a good way to debug the problem other than >> to dig into the code. If you want to debug from the Pulp code, you could >> stick a debugger in the artifact saver stage: >> >> >> https://github.com/pulp/pulpcore/blob/2203fee1407738a4ddd8e644fcbc741aab0bca63/pulpcore/plugin/stages/artifact_stages.py#L179-L200 >> >> What I would probably do though is stick a debug statement here in >> django-storages to see what params it's passing to boto3: >> >> >> https://github.com/jschneier/django-storages/blob/0ab2b1e3efd2bcaf0f24540a718993acc7742d9b/storages/backends/s3boto3.py#L511 >> >> You can see the location of django-storages with `pip show >> django-storages`. >> >> Sorry I don't have a better answer for you. Perhaps this is something we >> can improve in the future. Also, I'd be curious as to what the issue is as >> it sounds like everything should work in theory. >> >> David >> >> >> On Fri, Nov 1, 2019 at 2:26 PM Dumont, Joey <[email protected]> >> wrote: >> >>> I've installed the latest pulp3 using the Ansible installer using the >>> following playbook: >>> >>> >>> --- >>> - hosts: mirrors >>> vars: >>> prereq_pip_packages: >>> - django-storages >>> - boto3 >>> pulp_use_system_wide_pkgs: True >>> pulp_default_admin_password: !vault | >>> $ANSIBLE_VAULT;1.1;AES256 >>> ... >>> pulp_settings: >>> secret_key: !vault | >>> $ANSIBLE_VAULT;1.1;AES256 >>> ... >>> default_file_storage: 'storages.backends.s3boto3.S3Boto3Storage' >>> aws_storage_bucket_name: 'xxx-pulp-storage' >>> aws_s3_region_name: 'ca-central-1' >>> aws_s3_addressing_style: "path" >>> media_root: '/pulp3/' >>> pulp_install_plugins: >>> pulp-file: {} >>> pulp-rpm: >>> prereq_role: "pulp.pulp_rpm_prerequisites" >>> # pulp-docker: {} >>> roles: >>> - pulp-database >>> - pulp-workers >>> - pulp-resource-manager >>> - pulp-webserver >>> - pulp-content >>> environment: >>> DJANGO_SETTINGS_MODULE: pulpcore.app.settings >>> >>> I also set up an RPM repo that uses S3 for storage. However, when I try >>> to sync, I get an AccessDenied error. I know the instance profile is >>> correct, as I can upload objects from that instance using both the AWS CLI >>> and Boto3 without specifying credentials. >>> >>> How can I debug this further? Is there a way for me know what parameters >>> are passed to the put_object boto3 call by the sync task? >>> >>> Cheers, >>> >>> >>> >>> Joey Dumont >>> >>> Technical Advisor, Knowledge, Information, and Technology Services >>> National Research Council Canada / Governement of Canada >>> [email protected] / Tel: 613-990-8152 / Cell: 438-340-7436 >>> >>> Conseiller technique, Services du savoir, de l'information et de la >>> technologie >>> Conseil national de recherches Canada / Gouvernement du Canada >>> [email protected] / Tél.: 613-990-8152 / Tél. cell.: >>> 438-340-7436 >>> _______________________________________________ >>> Pulp-list mailing list >>> [email protected] >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >> _______________________________________________ >> Pulp-list mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/pulp-list > > > > -- > > Mike DePaulo > > He / Him / His > > Service Reliability Engineer, Pulp > > Red Hat <https://www.redhat.com/> > > IM: mikedep333 > > GPG: 51745404 > <https://www.redhat.com/> > _______________________________________________ > Pulp-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
