hey Josh / Reece, I hate to say I am glad, I am not the only one with this issue. Did anyone on #Pulp speak about the issue?
Let us know how you go with troubleshooting this. Thanks On Wed, May 6, 2015 at 3:22 AM, Webb, Reece <reece.w...@ucsf.edu> wrote: > I have seen this issue for months, a sync fails 9 times out of 10. It > appears to be an issue (for me at least) on the Redhat side of things. I > use curl to get more info. > > I’ll run it one time and get a failure: > > # curl -v —key ./Workstation-Entitlement.pem --cert > ./Workstation-Entitlement.pem -k > https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo > > - About to connect() to cdn.redhat.com port 443 (#0) > * Trying 184.84.192.251... > * Connected to cdn.redhat.com (184.84.192.251) port 443 (#0) > * Initializing NSS with certpath: sql:/etc/pki/nssdb > * skipping SSL peer certificate verification > * NSS: client certificate from file > * subject: CN=8a85f9894bd9c252014be203f1a6096f > * start date: Aug 01 04:00:00 2014 GMT > * expire date: Aug 01 03:59:59 2015 GMT > * common name: 8a85f9894bd9c252014be203f1a6096f > * issuer: E=ca-supp...@redhat.com,CN=Red Hat Candlepin > Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US > * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA > * Server certificate: > * subject: CN=cdn.redhat.com,OU=Red Hat Network,O=Red > Hat,L=Raleigh,ST=North Carolina,C=US > * start date: May 14 19:48:02 2014 GMT > * expire date: May 11 19:48:02 2024 GMT > * common name: cdn.redhat.com > * issuer: E=ca-supp...@redhat.com,CN=Red Hat Entitlement Operations > Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US > > GET > /content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo > HTTP/1.1 > > User-Agent: curl/7.29.0 > > Host: cdn.redhat.com > > Accept: */* > > > < HTTP/1.1 403 Forbidden > < Server: AkamaiGHost > < Mime-Version: 1.0 > < Content-Type: text/html > < Content-Length: 369 > < Expires: Tue, 05 May 2015 17:13:05 GMT > < Date: Tue, 05 May 2015 17:13:05 GMT > < X-Cache: TCP_DENIED from > a128-241-218-165.deploy.akamaitechnologies.com > (AkamaiGHost/7.2.0-15182023) (-) > < Connection: keep-alive > < EJ-HOST: edgejavaapp2.prod.a4.vary.redhat.com > < X-Akamai-Request-ID: 4a217f0 > < > <HTML><HEAD> > <TITLE>Access Denied</TITLE> > </HEAD><BODY> > <H1>Access Denied</H1> > > You don't have permission to access > > "http://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo" > on this server.<P> > Reference #18.a5daf180.1430845985.4a217f0 > > > > And then I’ll re-run the command seconds later with a successful > response: > > # curl -v --key ./Workstation-Entitlement.pem --cert > ./Workstation-Entitlement.pem -k > https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo > * About to connect() to cdn.redhat.com port 443 (#0) > * Trying 184.84.192.251... > * Connected to cdn.redhat.com (184.84.192.251) port 443 (#0) > * Initializing NSS with certpath: sql:/etc/pki/nssdb > * skipping SSL peer certificate verification > * NSS: client certificate from file > * subject: CN=8a85f9894bd9c252014be203f1a6096f > * start date: Aug 01 04:00:00 2014 GMT > * expire date: Aug 01 03:59:59 2015 GMT > * common name: 8a85f9894bd9c252014be203f1a6096f > * issuer: E=ca-supp...@redhat.com,CN=Red Hat Candlepin Authority,OU=Red > Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US > * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA > * Server certificate: > * subject: CN=cdn.redhat.com,OU=Red Hat Network,O=Red > Hat,L=Raleigh,ST=North Carolina,C=US > * start date: May 14 19:48:02 2014 GMT > * expire date: May 11 19:48:02 2024 GMT > * common name: cdn.redhat.com > * issuer: E=ca-supp...@redhat.com,CN=Red Hat Entitlement Operations > Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US > > GET > /content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo > HTTP/1.1 > > User-Agent: curl/7.29.0 > > Host: cdn.redhat.com > > Accept: */* > > > < HTTP/1.1 200 OK > < Server: Apache > < ETag: "11f6fa6eaa857d424b630447ab5334de:1424446169" > < Last-Modified: Fri, 20 Feb 2015 08:29:44 GMT > < Accept-Ranges: bytes > < Content-Length: 1471 > < Content-Type: text/plain > < Date: Tue, 05 May 2015 17:16:10 GMT > < X-Cache: TCP_HIT from a128-241-218-165.deploy.akamaitechnologies.com > (AkamaiGHost/7.2.0-15182023) (-) > < Connection: keep-alive > < EJ-HOST: rhej03.web.prod.ext.phx2.redhat.com > < X-Akamai-Request-ID: 4a57fb3 > < > [checksums] > LiveOS/squashfs.img = > sha256:198ef91d868e76c994680645964ef3873ec66fddb84be450370b051facaec8aa > images/pxeboot/initrd.img = > sha256:101b3b5630b7032557be95aa8dcef50b01d8bfcdfa33429cea30fe09eaae9426 > images/pxeboot/upgrade.img = > sha256:03453b1f504e548ab9a933daa2f1fd440e48638f5deb9fac50be7dad929c1907 > images/pxeboot/vmlinuz = > sha256:67421a4877919ff0c16c27a53cba229e5f0771ae9cd32f3918caae2124a5a710 > repodata/repomd.xml = > sha256:014184dc5e503979a5577a97423e4340e5f71ac2746250bbdce91e0301b8c93f > > … > > > I never have this issue syncing the Server repositories, only > Workstation (and RHEL5 Client). > > Reece > > > > From: "Baird, Josh" > Date: Tuesday, May 5, 2015 at 4:23 AM > To: Gavin Jones, "pulp-list@redhat.com" > Subject: Re: [Pulp-list] Pulp RHEL Repo Download Forbidden?? > > Hi Gavin, > > > > I am having the same problem. I just noticed that it was occurring > yesterday. I re-issued new entitlement certificates with valid expiration > dates from RHN and the problem is still occurring. I have verified that my > certificates contain path/entitlements for the channels that I am trying to > sync (via rct cat-cert). Occasionally, Pulp will be able to download the > metadata for certain channels, but then get 'Forbidden' when downloading > individual packages. Other times, it will throw a 'Forbidden' before being > able to download the metadata as you pasted below. > > > > I am going to hopefully spend some time working with the developers in > #pulp today to get this figured out. I have a feeling it is CDN related, > but I'm not exactly sure at this point. > > > > Thanks, > > > > Josh > > > > *From:* pulp-list-boun...@redhat.com [mailto:pulp-list-boun...@redhat.com > <pulp-list-boun...@redhat.com>] *On Behalf Of *Gavin Jones > *Sent:* Tuesday, May 05, 2015 12:13 AM > *To:* pulp-list@redhat.com > *Subject:* [Pulp-list] Pulp RHEL Repo Download Forbidden?? > > > > > > > > Hi Everyone, I seem to be getting an error when downloading from the > Redhat Repos. This has only just stopped working and has been working fine > for months. > > > > It looks to be certificate related I believe from the logs. > > > > > > * Firstly I have not changed anything on the pulp side > > * I have checked my subscriptions are still active and the hosts that are > connected to RHEL are still connected. > > > > > > - Pulp Version: > > > > rpm -qa | grep -i pulp > > > > python-pulp-client-lib-2.6.0-1.el7.noarch > > pulp-rpm-plugins-2.6.0-1.el7.noarch > > python-pulp-bindings-2.6.0-1.el7.noarch > > python-kombu-3.0.24-5.pulp.el7.noarch > > python-isodate-0.5.0-4.pulp.el7.noarch > > pulp-admin-client-2.6.0-1.el7.noarch > > pulp-rpm-admin-extensions-2.6.0-1.el7.noarch > > python-pulp-common-2.6.0-1.el7.noarch > > pulp-server-2.6.0-1.el7.noarch > > pulp-selinux-2.6.0-1.el7.noarch > > python-pulp-rpm-common-2.6.0-1.el7.noarch > > > > > > - Attempting to download the repo. > > > > Please see below: > > > > pulp-admin rpm repo sync run --repo-id=rhel-7-server-rhn-tools-rpms > > +----------------------------------------------------------------------+ > > Synchronizing Repository [rhel-7-server-rhn-tools-rpms] > > +----------------------------------------------------------------------+ > > > > This command may be exited via ctrl+c without affecting the request. > > > > > > Downloading metadata... > > [\] > > ... failed > > > > Forbidden > > > > > > Task Failed > > > > Importer indicated a failed response > > > > > > > > - Error Log > > > > journalctl -f > > > > ay 05 13:33:05 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from > https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhn-tools/os/ > . > > May 05 13:33:05 pulp01.rap.local pulp[2741]: > requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS > connection (1): cdn.redhat.com > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) sync failed > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) Traceback (most > recent call last): > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) File > "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",...e > 104, in run > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) metadata_files > = self.get_metadata() > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) File > "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",... > get_metadata > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) raise > FailedException(str(e)) > > May 05 13:33:06 pulp01.rap.local pulp[2741]: > pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) FailedException: > Forbidden > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) Task > pulp.server.managers.repo.sync.sync[81644b21-6bec-47dd-a31b-552baa2a27a8] > raised unexpected: P...d response',) > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) Traceback (most recent call last): > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", > line 240, in trace_task > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) R = retval = fun(*args, **kwargs) > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) File > "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in > __call__ > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) return super(Task, self).__call__(*args, **kwargs) > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", > line 437, in __protected_call__ > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) return self.run(*args, **kwargs) > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) File > "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line > 114, in sync > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) raise PulpExecutionException(_('Importer indicated a > failed response')) > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR: > (2554-28000) PulpExecutionException: Importer indicated a failed response > > May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:INFO: Task > pulp.server.async.tasks._release_resource[e8f32211-ccc5-4918-b4d5-ada23e15ecf4] > succeeded in 0.010533269s: None > > > > is there a clean way to fix this issue without Deleting the entire repo > and going through the process of setting this up again? > > > > Thanks for your time. > > > > > > > > >
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list
