Re: [Pulp-list] RHEL 8 rpm repo sync errors
Hi Dennis , Thank you very much for all these details . I will try to create repo as shown on that url " Sync a Protected Repo". Yes , our other repositories are able to sync Red Hat content . To compare their configuration I did not know exactly where to look for that . Sincerely,Ramana Bora - Original message -From: Dennis Kliban To: Venkataramana Bora Cc: pulp-list Subject: [EXTERNAL] Re: [Pulp-list] RHEL 8 rpm repo sync errorsDate: Thu, Mar 19, 2020 1:24 AM I was referring to the certificate and key configured on the importer associated with the repository. Our docs explain how to configure a new repository to sync from the Red Hat servers[0]. If your other repositories are able to sync Red Hat content, I would compare the configuration of those repositories with the broken one. [0] https://docs.pulpproject.org/en/2.21/plugins/pulp_rpm/user-guide/recipes.html#sync-a-protected-repo On Wed, Mar 18, 2020 at 3:43 PM Venkataramana Borawrote: Hi Dennis , Thanks a lot for your reply. Sorry, for this question . I'm not well versed with Puppet configuration , some one else did the config part here, no longer with company. You said , we should have the correct client certificate and client key configured. Is that SSLCertificateFile and SSLCertificateKeyFile in /etc/httpd/conf.d/ssl.conf ? Can you please let me know patch to check? Sincerely,Ramana Bora - Original message -From: Dennis Kliban To: Venkataramana Bora Cc: pulp-list Subject: [EXTERNAL] Re: [Pulp-list] RHEL 8 rpm repo sync errorsDate: Wed, Mar 18, 2020 7:25 PM This looks like a problem with the client certificate. The very first request for a file that pulp tries to download is receiving a 403 response from the CDN. Please make sure you have the correct client certificate and client key configured. On Tue, Mar 17, 2020 at 5:53 PM Venkataramana Bora wrote: Hi Teamn ,Getting errors as shown here when we are trying to sync RHEL 8 repos (Base OS and Appstream) creatd on Pulp master. The same Pulp master we already using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 repos creation,1.Could you please let us know whethere RHEL 8 rpm repos sync works on Pulp master version 2.16 or not ?2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are there . We have no issues with RHEL 6 and 7 repo syncs, those created some years ago and working well. Is there any thing specifically need to do for RHEL 8 repos in terms of ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log messages. 3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..." For this added these 3 lines in /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here https://pulp.plan.io/issues/6327 { "validate":true} After adding that json , I see that RHEL 8 repodata xml.gz in rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , checked that thru pulp web access .Please help resolving this issue . Feed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/osFeed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/osRepoid for these urls are :rhel-8-server-baseos-x86_64rhel-8-server-appstream-x86_64 Note: Tried with "8" in place "$releasever" in feed but same error . Our Pulp master ver.2.16 is on CentOS 7.7. -- Tasks performed: [root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full --repo-id rhel-8-server-baseos-x86_64+--+ Synchronizing Repository [rhel-8-server-baseos-x86_64]+--+ The following sync configuration options will be used: Force Full: True This command may be exited via ctrl+c without affecting the request. Downloading metadata...[\]... completed Downloading repository content...[-][==] 100%RPMs: 0/0 itemsDelta RPMs: 0/0 items ... completed Downloading distribution files...[==] 100%Distributions: 0/0 items Task Failed Error retrieving metadata: Forbidden -From /var/log/messages: Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc]Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] Task pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] succeeded in 0.115055091001s: NoneMar 17 17:11:23 swy01
Re: [Pulp-list] RHEL 8 rpm repo sync errors
I was referring to the certificate and key configured on the importer associated with the repository. Our docs explain how to configure a new repository to sync from the Red Hat servers[0]. If your other repositories are able to sync Red Hat content, I would compare the configuration of those repositories with the broken one. [0] https://docs.pulpproject.org/en/2.21/plugins/pulp_rpm/user-guide/recipes.html#sync-a-protected-repo On Wed, Mar 18, 2020 at 3:43 PM Venkataramana Bora wrote: > Hi Dennis , Thanks a lot for your reply. > Sorry, for this question . I'm not well versed with Puppet configuration , > some one else did the config part here, no longer with company. > You said , we should have the correct client certificate and client key > configured. > Is that SSLCertificateFile and SSLCertificateKeyFile in > /etc/httpd/conf.d/ssl.conf ? > Can you please let me know patch to check? > > > > Sincerely, > Ramana Bora > > > > - Original message - > From: Dennis Kliban > To: Venkataramana Bora > Cc: pulp-list > Subject: [EXTERNAL] Re: [Pulp-list] RHEL 8 rpm repo sync errors > Date: Wed, Mar 18, 2020 7:25 PM > > This looks like a problem with the client certificate. The very first > request for a file that pulp tries to download is receiving a 403 response > from the CDN. Please make sure you have the correct client certificate and > client key configured. > > On Tue, Mar 17, 2020 at 5:53 PM Venkataramana Bora > wrote: > > > > Hi Teamn , > Getting errors as shown here when we are trying to sync RHEL 8 repos (Base > OS and Appstream) creatd on Pulp master. The same Pulp master we already > using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 > repos creation, > 1.Could you please let us know whethere RHEL 8 rpm repos sync works on > Pulp master version 2.16 or not ? > 2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are > there . We have no issues with RHEL 6 and 7 repo syncs, those created some > years ago and working well. > Is there any thing specifically need to do for RHEL 8 repos in terms of > ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log > messages. > 3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..." > For this added these 3 lines in > /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here > https://pulp.plan.io/issues/6327 > { >"validate":true > } > After adding that json , I see that RHEL 8 repodata xml.gz in > rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , > checked that thru pulp web access . > Please help resolving this issue . > > Feed url: > https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/os > Feed url: > https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/os > Repoid for these urls are : > rhel-8-server-baseos-x86_64 > rhel-8-server-appstream-x86_64 > Note: Tried with "8" in place "$releasever" in feed but same error . Our > Pulp master ver.2.16 is on CentOS 7.7. > > -- > Tasks performed: > > [root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full > --repo-id rhel-8-server-baseos-x86_64 > +--+ > Synchronizing Repository [rhel-8-server-baseos-x86_64] > +--+ > The following sync configuration options will be used: > Force Full: True > > This command may be exited via ctrl+c without affecting the request. > > Downloading metadata... > [\] > ... completed > Downloading repository content... > [-] > [==] 100% > RPMs: 0/0 items > Delta RPMs: 0/0 items > ... completed > Downloading distribution files... > [==] 100% > Distributions: 0/0 items > Task Failed > Error retrieving metadata: Forbidden > > > - > From /var/log/messages: > > Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received > task: > pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc] > Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] > Task > pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] > succ
Re: [Pulp-list] RHEL 8 rpm repo sync errors
Hi Dennis , Thanks a lot for your reply. Sorry, for this question . I'm not well versed with Puppet configuration , some one else did the config part here, no longer with company. You said , we should have the correct client certificate and client key configured. Is that SSLCertificateFile and SSLCertificateKeyFile in /etc/httpd/conf.d/ssl.conf ? Can you please let me know patch to check? Sincerely,Ramana Bora - Original message -From: Dennis Kliban To: Venkataramana Bora Cc: pulp-list Subject: [EXTERNAL] Re: [Pulp-list] RHEL 8 rpm repo sync errorsDate: Wed, Mar 18, 2020 7:25 PM This looks like a problem with the client certificate. The very first request for a file that pulp tries to download is receiving a 403 response from the CDN. Please make sure you have the correct client certificate and client key configured. On Tue, Mar 17, 2020 at 5:53 PM Venkataramana Borawrote: Hi Teamn ,Getting errors as shown here when we are trying to sync RHEL 8 repos (Base OS and Appstream) creatd on Pulp master. The same Pulp master we already using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 repos creation,1.Could you please let us know whethere RHEL 8 rpm repos sync works on Pulp master version 2.16 or not ?2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are there . We have no issues with RHEL 6 and 7 repo syncs, those created some years ago and working well. Is there any thing specifically need to do for RHEL 8 repos in terms of ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log messages. 3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..." For this added these 3 lines in /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here https://pulp.plan.io/issues/6327 { "validate":true} After adding that json , I see that RHEL 8 repodata xml.gz in rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , checked that thru pulp web access .Please help resolving this issue . Feed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/osFeed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/osRepoid for these urls are :rhel-8-server-baseos-x86_64rhel-8-server-appstream-x86_64 Note: Tried with "8" in place "$releasever" in feed but same error . Our Pulp master ver.2.16 is on CentOS 7.7. -- Tasks performed: [root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full --repo-id rhel-8-server-baseos-x86_64+--+ Synchronizing Repository [rhel-8-server-baseos-x86_64]+--+ The following sync configuration options will be used: Force Full: True This command may be exited via ctrl+c without affecting the request. Downloading metadata...[\]... completed Downloading repository content...[-][==] 100%RPMs: 0/0 itemsDelta RPMs: 0/0 items ... completed Downloading distribution files...[==] 100%Distributions: 0/0 items Task Failed Error retrieving metadata: Forbidden -From /var/log/messages: Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc]Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] Task pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] succeeded in 0.115055091001s: NoneMar 17 17:11:23 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.Mar 17 17:11:23 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:23 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: ForbiddenMar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with code 403: ForbiddenMar 17 17:11:24 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new
Re: [Pulp-list] RHEL 8 rpm repo sync errors
This looks like a problem with the client certificate. The very first request for a file that pulp tries to download is receiving a 403 response from the CDN. Please make sure you have the correct client certificate and client key configured. On Tue, Mar 17, 2020 at 5:53 PM Venkataramana Bora wrote: > > > Hi Teamn , > Getting errors as shown here when we are trying to sync RHEL 8 repos (Base > OS and Appstream) creatd on Pulp master. The same Pulp master we already > using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 > repos creation, > 1.Could you please let us know whethere RHEL 8 rpm repos sync works on > Pulp master version 2.16 or not ? > 2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are > there . We have no issues with RHEL 6 and 7 repo syncs, those created some > years ago and working well. > Is there any thing specifically need to do for RHEL 8 repos in terms of > ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log > messages. > 3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..." > For this added these 3 lines in > /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here > https://pulp.plan.io/issues/6327 > { >"validate":true > } > After adding that json , I see that RHEL 8 repodata xml.gz in > rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , > checked that thru pulp web access . > Please help resolving this issue . > > Feed url: > https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/os > Feed url: > https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/os > Repoid for these urls are : > rhel-8-server-baseos-x86_64 > rhel-8-server-appstream-x86_64 > Note: Tried with "8" in place "$releasever" in feed but same error . Our > Pulp master ver.2.16 is on CentOS 7.7. > > -- > Tasks performed: > > [root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full > --repo-id rhel-8-server-baseos-x86_64 > +--+ > Synchronizing Repository [rhel-8-server-baseos-x86_64] > +--+ > The following sync configuration options will be used: > Force Full: True > > This command may be exited via ctrl+c without affecting the request. > > Downloading metadata... > [\] > ... completed > Downloading repository content... > [-] > [==] 100% > RPMs: 0/0 items > Delta RPMs: 0/0 items > ... completed > Downloading distribution files... > [==] 100% > Distributions: 0/0 items > Task Failed > Error retrieving metadata: Forbidden > > > - > From /var/log/messages: > > Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received > task: > pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc] > Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] > Task > pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] > succeeded in 0.115055091001s: None > Mar 17 17:11:23 swy01opplppr01 pulp: > pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata > from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/. > Mar 17 17:11:23 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting > new HTTPS connection (1): cdn.redhat.com > Mar 17 17:11:23 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: > Download failed: Download of > https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml > failed with code 403: Forbidden > Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: > [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.com > Mar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: > [fbbc36d3] Download failed: Download of > https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with > code 403: Forbidden > Mar 17 17:11:24 swy01opplppr01 pulp: > pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata > from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/. > Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting > new HTTPS connection (1): cdn.redhat.com > Mar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: > Download failed: Download of > https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml > failed with code 403: Forbidden > Mar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: > [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.com > Mar 17 17:11:25 swy0
[Pulp-list] RHEL 8 rpm repo sync errors
Hi Teamn ,Getting errors as shown here when we are trying to sync RHEL 8 repos (Base OS and Appstream) creatd on Pulp master. The same Pulp master we already using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 repos creation,1.Could you please let us know whethere RHEL 8 rpm repos sync works on Pulp master version 2.16 or not ?2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are there . We have no issues with RHEL 6 and 7 repo syncs, those created some years ago and working well. Is there any thing specifically need to do for RHEL 8 repos in terms of ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log messages. 3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..." For this added these 3 lines in /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here https://pulp.plan.io/issues/6327 { "validate":true} After adding that json , I see that RHEL 8 repodata xml.gz in rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , checked that thru pulp web access .Please help resolving this issue . Feed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/osFeed url: https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/osRepoid for these urls are :rhel-8-server-baseos-x86_64rhel-8-server-appstream-x86_64 Note: Tried with "8" in place "$releasever" in feed but same error . Our Pulp master ver.2.16 is on CentOS 7.7. -- Tasks performed: [root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full --repo-id rhel-8-server-baseos-x86_64+--+ Synchronizing Repository [rhel-8-server-baseos-x86_64]+--+ The following sync configuration options will be used: Force Full: True This command may be exited via ctrl+c without affecting the request. Downloading metadata...[\]... completed Downloading repository content...[-][==] 100%RPMs: 0/0 itemsDelta RPMs: 0/0 items ... completed Downloading distribution files...[==] 100%Distributions: 0/0 items Task Failed Error retrieving metadata: Forbidden -From /var/log/messages: Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc]Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] Task pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] succeeded in 0.115055091001s: NoneMar 17 17:11:23 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.Mar 17 17:11:23 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:23 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: ForbiddenMar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with code 403: ForbiddenMar 17 17:11:24 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml failed with code 403: ForbiddenMar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:25 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os failed with code 403: ForbiddenMar 17 17:11:25 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.Mar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.comMar 17 17:11:25 swy01opplppr01 pulp: nectar.downloaders.threaded: