I can confirm that this works for me as well. Thanks Gavin, Reece
From: Gavin Jones Date: Sunday, May 17, 2015 at 8:54 PM To: Reece Webb Cc: "Baird, Josh", "pulp-list@redhat.com<mailto:pulp-list@redhat.com>" Subject: Re: [Pulp-list] Pulp RHEL Repo Download Forbidden?? Ok if it helps anyone I have deleted the repos in Pulp and created them again it's all fine now. After checking on my servers which were directly connected to RHN there certificates had been updated, hence the break in the PULP sync. Thanks On Wed, May 6, 2015 at 9:26 AM, Gavin Jones <gavin...@gmail.com<mailto:gavin...@gmail.com>> wrote: hey Josh / Reece, I hate to say I am glad, I am not the only one with this issue. Did anyone on #Pulp speak about the issue? Let us know how you go with troubleshooting this. Thanks On Wed, May 6, 2015 at 3:22 AM, Webb, Reece <reece.w...@ucsf.edu<mailto:reece.w...@ucsf.edu>> wrote: I have seen this issue for months, a sync fails 9 times out of 10. It appears to be an issue (for me at least) on the Redhat side of things. I use curl to get more info. I’ll run it one time and get a failure: # curl -v —key ./Workstation-Entitlement.pem --cert ./Workstation-Entitlement.pem -k https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo * About to connect() to cdn.redhat.com<http://cdn.redhat.com> port 443 (#0) * Trying 184.84.192.251... * Connected to cdn.redhat.com<http://cdn.redhat.com> (184.84.192.251) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * skipping SSL peer certificate verification * NSS: client certificate from file * subject: CN=8a85f9894bd9c252014be203f1a6096f * start date: Aug 01 04:00:00 2014 GMT * expire date: Aug 01 03:59:59 2015 GMT * common name: 8a85f9894bd9c252014be203f1a6096f * issuer: E=ca-supp...@redhat.com<mailto:ca-supp...@redhat.com>,CN=Red Hat Candlepin Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=cdn.redhat.com<http://cdn.redhat.com>,OU=Red Hat Network,O=Red Hat,L=Raleigh,ST=North Carolina,C=US * start date: May 14 19:48:02 2014 GMT * expire date: May 11 19:48:02 2024 GMT * common name: cdn.redhat.com<http://cdn.redhat.com> * issuer: E=ca-supp...@redhat.com<mailto:ca-supp...@redhat.com>,CN=Red Hat Entitlement Operations Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US > GET /content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo > HTTP/1.1 > User-Agent: curl/7.29.0 > Host: cdn.redhat.com<http://cdn.redhat.com> > Accept: */* > < HTTP/1.1 403 Forbidden < Server: AkamaiGHost < Mime-Version: 1.0 < Content-Type: text/html < Content-Length: 369 < Expires: Tue, 05 May 2015 17:13:05 GMT < Date: Tue, 05 May 2015 17:13:05 GMT < X-Cache: TCP_DENIED from a128-241-218-165.deploy.akamaitechnologies.com<http://a128-241-218-165.deploy.akamaitechnologies.com> (AkamaiGHost/7.2.0-15182023) (-) < Connection: keep-alive < EJ-HOST: edgejavaapp2.prod.a4.vary.redhat.com<http://edgejavaapp2.prod.a4.vary.redhat.com> < X-Akamai-Request-ID: 4a217f0 < <HTML><HEAD> <TITLE>Access Denied</TITLE> </HEAD><BODY> <H1>Access Denied</H1> You don't have permission to access "http://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo" on this server.<P> Reference #18.a5daf180.1430845985.4a217f0 And then I’ll re-run the command seconds later with a successful response: # curl -v --key ./Workstation-Entitlement.pem --cert ./Workstation-Entitlement.pem -k https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo * About to connect() to cdn.redhat.com<http://cdn.redhat.com> port 443 (#0) * Trying 184.84.192.251... * Connected to cdn.redhat.com<http://cdn.redhat.com> (184.84.192.251) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * skipping SSL peer certificate verification * NSS: client certificate from file * subject: CN=8a85f9894bd9c252014be203f1a6096f * start date: Aug 01 04:00:00 2014 GMT * expire date: Aug 01 03:59:59 2015 GMT * common name: 8a85f9894bd9c252014be203f1a6096f * issuer: E=ca-supp...@redhat.com<mailto:ca-supp...@redhat.com>,CN=Red Hat Candlepin Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=cdn.redhat.com<http://cdn.redhat.com>,OU=Red Hat Network,O=Red Hat,L=Raleigh,ST=North Carolina,C=US * start date: May 14 19:48:02 2014 GMT * expire date: May 11 19:48:02 2024 GMT * common name: cdn.redhat.com<http://cdn.redhat.com> * issuer: E=ca-supp...@redhat.com<mailto:ca-supp...@redhat.com>,CN=Red Hat Entitlement Operations Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US > GET /content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo > HTTP/1.1 > User-Agent: curl/7.29.0 > Host: cdn.redhat.com<http://cdn.redhat.com> > Accept: */* > < HTTP/1.1 200 OK < Server: Apache < ETag: "11f6fa6eaa857d424b630447ab5334de:1424446169" < Last-Modified: Fri, 20 Feb 2015 08:29:44 GMT < Accept-Ranges: bytes < Content-Length: 1471 < Content-Type: text/plain < Date: Tue, 05 May 2015 17:16:10 GMT < X-Cache: TCP_HIT from a128-241-218-165.deploy.akamaitechnologies.com<http://a128-241-218-165.deploy.akamaitechnologies.com> (AkamaiGHost/7.2.0-15182023) (-) < Connection: keep-alive < EJ-HOST: rhej03.web.prod.ext.phx2.redhat.com<http://rhej03.web.prod.ext.phx2.redhat.com> < X-Akamai-Request-ID: 4a57fb3 < [checksums] LiveOS/squashfs.img = sha256:198ef91d868e76c994680645964ef3873ec66fddb84be450370b051facaec8aa images/pxeboot/initrd.img = sha256:101b3b5630b7032557be95aa8dcef50b01d8bfcdfa33429cea30fe09eaae9426 images/pxeboot/upgrade.img = sha256:03453b1f504e548ab9a933daa2f1fd440e48638f5deb9fac50be7dad929c1907 images/pxeboot/vmlinuz = sha256:67421a4877919ff0c16c27a53cba229e5f0771ae9cd32f3918caae2124a5a710 repodata/repomd.xml = sha256:014184dc5e503979a5577a97423e4340e5f71ac2746250bbdce91e0301b8c93f … I never have this issue syncing the Server repositories, only Workstation (and RHEL5 Client). Reece From: "Baird, Josh" Date: Tuesday, May 5, 2015 at 4:23 AM To: Gavin Jones, "pulp-list@redhat.com<mailto:pulp-list@redhat.com>" Subject: Re: [Pulp-list] Pulp RHEL Repo Download Forbidden?? Hi Gavin, I am having the same problem. I just noticed that it was occurring yesterday. I re-issued new entitlement certificates with valid expiration dates from RHN and the problem is still occurring. I have verified that my certificates contain path/entitlements for the channels that I am trying to sync (via rct cat-cert). Occasionally, Pulp will be able to download the metadata for certain channels, but then get 'Forbidden' when downloading individual packages. Other times, it will throw a 'Forbidden' before being able to download the metadata as you pasted below. I am going to hopefully spend some time working with the developers in #pulp today to get this figured out. I have a feeling it is CDN related, but I'm not exactly sure at this point. Thanks, Josh From:pulp-list-boun...@redhat.com<mailto:pulp-list-boun...@redhat.com> [mailto:pulp-list-boun...@redhat.com] On Behalf Of Gavin Jones Sent: Tuesday, May 05, 2015 12:13 AM To: pulp-list@redhat.com<mailto:pulp-list@redhat.com> Subject: [Pulp-list] Pulp RHEL Repo Download Forbidden?? Hi Everyone, I seem to be getting an error when downloading from the Redhat Repos. This has only just stopped working and has been working fine for months. It looks to be certificate related I believe from the logs. * Firstly I have not changed anything on the pulp side * I have checked my subscriptions are still active and the hosts that are connected to RHEL are still connected. - Pulp Version: rpm -qa | grep -i pulp python-pulp-client-lib-2.6.0-1.el7.noarch pulp-rpm-plugins-2.6.0-1.el7.noarch python-pulp-bindings-2.6.0-1.el7.noarch python-kombu-3.0.24-5.pulp.el7.noarch python-isodate-0.5.0-4.pulp.el7.noarch pulp-admin-client-2.6.0-1.el7.noarch pulp-rpm-admin-extensions-2.6.0-1.el7.noarch python-pulp-common-2.6.0-1.el7.noarch pulp-server-2.6.0-1.el7.noarch pulp-selinux-2.6.0-1.el7.noarch python-pulp-rpm-common-2.6.0-1.el7.noarch - Attempting to download the repo. Please see below: pulp-admin rpm repo sync run --repo-id=rhel-7-server-rhn-tools-rpms +----------------------------------------------------------------------+ Synchronizing Repository [rhel-7-server-rhn-tools-rpms] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [\] ... failed Forbidden Task Failed Importer indicated a failed response - Error Log journalctl -f ay 05 13:33:05 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhn-tools/os/. May 05 13:33:05 pulp01.rap.local pulp[2741]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com<http://cdn.redhat.com> May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) sync failed May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) Traceback (most recent call last): May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",...e 104, in run May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) metadata_files = self.get_metadata() May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",... get_metadata May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) raise FailedException(str(e)) May 05 13:33:06 pulp01.rap.local pulp[2741]: pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) FailedException: Forbidden May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) Task pulp.server.managers.repo.sync.sync[81644b21-6bec-47dd-a31b-552baa2a27a8] raised unexpected: P...d response',) May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) Traceback (most recent call last): May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) R = retval = fun(*args, **kwargs) May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in __call__ May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) return super(Task, self).__call__(*args, **kwargs) May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__ May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) return self.run(*args, **kwargs) May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) raise PulpExecutionException(_('Importer indicated a failed response')) May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: (2554-28000) PulpExecutionException: Importer indicated a failed response May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[e8f32211-ccc5-4918-b4d5-ada23e15ecf4] succeeded in 0.010533269s: None is there a clean way to fix this issue without Deleting the entire repo and going through the process of setting this up again? Thanks for your time.
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
