Katello server 3.3.1
Foreman Proxy 1.14.2
When syncing my katello content, certain repos fail to sync and therefore
never generate the repomd.xml, and then when servers try to use katello
content, they fail with a 404 and tell the user to disable the repos.
This is limited, in my environment, to CentOS 7 Extras, Updates, and Plus
repos. However, subscription-manager behaves in such a way that all
install/update tasks fail on these errors, instead of skipping the failing
repo.
>From what I can tell, DRPMs are not correctly linked on the Katello server,
and so the sync process fails with 404 errors and "[foreman-tasks/action]
[E] PLP0000: Importer indicated a failed response
(Katello::Errors::PulpError)" reported on the Katello server.
The Smart Proxy logs the following error:
Mar 28 13:14:33 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container
reported: {'downloads': {}, 'total_sources': 0} for base URL:
https://foreman-01.prod.mcs.som.mob.nuance.com/pulp/repos/nuance_mobility/Development/MCS_Mesos/custom/CentOS_7/Plus_x86_64/
Mar 28 13:14:33 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 9 DRPMs.
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (1): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (2): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (3): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (4): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (5): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (6): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (7): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (8): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:33 smart-proxy-01 pulp:
requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
connection (9): foreman-01.prod.mcs.som.mob.nuance.com
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632) 'error_code'
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632) Traceback (most
recent call last):
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632) File
"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",
line 232, in run
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632)
self.update_content(metadata_files, url)
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632) File
"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",
line 501, in update_content
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632) if
error[constants.ERROR_CODE] == constants.ERROR_KEY_ID_FILTER:
Mar 28 13:14:34 smart-proxy-01 pulp:
pulp_rpm.plugins.importers.yum.sync:ERROR: (4409-85632) KeyError:
'error_code'
Mar 28 13:14:34 smart-proxy-01 pulp: pulp.server.async.tasks:INFO: Task
failed : [4a88b023-5779-4b03-b1d1-b6c832d89d2b]
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
Task
pulp.server.managers.repo.sync.sync[4a88b023-5779-4b03-b1d1-b6c832d89d2b]
raised unexpected: PulpExecutionException('Importer indicated a failed
response',)
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
Traceback (most recent call last):
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in
trace_task
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
R = retval = fun(*args, **kwargs)
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line
488, in __call__
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
return super(Task, self).__call__(*args, **kwargs)
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line
103, in __call__
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
return super(PulpTask, self).__call__(*args, **kwargs)
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in
__protected_call__
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
return self.run(*args, **kwargs)
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
File
"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py",
line 810, in sync
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
raise pulp_exceptions.PulpExecutionException(_('Importer indicated a
failed response'))
Mar 28 13:14:34 smart-proxy-01 pulp: celery.worker.job:ERROR: (4283-85632)
PulpExecutionException: Importer indicated a failed response
Corresponding 404s:
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/d5/29a4bfd392fd8b6a2bdeb0f85e403a4303cef05e521a8ad05065604473cd88/drpms/kernel-plus-doc-3.10.0-327.36.2.el7.centos.plus_3.10.0-514.6.2.el7.centos.plus.noarch.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/d5/29a4bfd392fd8b6a2bdeb0f85e403a4303cef05e521a8ad05065604473cd88/drpms/kernel-plus-doc-3.10.0-327.36.2.el7.centos.plus_3.10.0-514.6.2.el7.centos.plus.noarch.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/0c/28310fe548a3368fc89b0c7127f8701a5ebc1bda6d0e4dc973ceb1ecb49059/drpms/python-perf-3.10.0-327.36.2.el7.centos.plus_3.10.0-514.el7.centos.plus.x86_64.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/0c/28310fe548a3368fc89b0c7127f8701a5ebc1bda6d0e4dc973ceb1ecb49059/drpms/python-perf-3.10.0-327.36.2.el7.centos.plus_3.10.0-514.el7.centos.plus.x86_64.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/57/ea75483a4162b2f147c8a3b65a7358912b56b03b2a535614bb683be0c4654e/drpms/kernel-plus-abi-whitelists-3.10.0-327.28.3.el7.centos.plus_3.10.0-514.6.1.el7.centos.plus.noarch.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/57/ea75483a4162b2f147c8a3b65a7358912b56b03b2a535614bb683be0c4654e/drpms/kernel-plus-abi-whitelists-3.10.0-327.28.3.el7.centos.plus_3.10.0-514.6.1.el7.centos.plus.noarch.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/b1/3e2047414a7492e14cc30182a80ae499c9dd289f59d34d4379d18e2cf8a399/drpms/perf-3.10.0-327.36.3.el7.centos.plus_3.10.0-514.6.1.el7.centos.plus.x86_64.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/b1/3e2047414a7492e14cc30182a80ae499c9dd289f59d34d4379d18e2cf8a399/drpms/perf-3.10.0-327.36.3.el7.centos.plus_3.10.0-514.6.1.el7.centos.plus.x86_64.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/ed/ca7624263fa664662582eeae59b72946723675348b761ac17fc84bdf133515/drpms/kernel-plus-tools-libs-devel-3.10.0-327.36.3.el7.centos.plus_3.10.0-514.6.2.el7.centos.plus.x86_64.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/ed/ca7624263fa664662582eeae59b72946723675348b761ac17fc84bdf133515/drpms/kernel-plus-tools-libs-devel-3.10.0-327.36.3.el7.centos.plus_3.10.0-514.6.2.el7.centos.plus.x86_64.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/5f/7a640ff1d0e560b76c79959e6acf05f0990f3b618acfe1c82221cc84687e2f/drpms/kernel-plus-tools-libs-3.10.0-327.36.1.el7.centos.plus_3.10.0-514.el7.centos.plus.x86_64.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/5f/7a640ff1d0e560b76c79959e6acf05f0990f3b618acfe1c82221cc84687e2f/drpms/kernel-plus-tools-libs-3.10.0-327.36.1.el7.centos.plus_3.10.0-514.el7.centos.plus.x86_64.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/1f/62164ace61b1d3a93cb9ba6bc752f4671346051641641c208b996a5d573f9e/drpms/kernel-plus-doc-3.10.0-514.el7.centos.plus_3.10.0-514.2.2.el7.centos.plus.noarch.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/1f/62164ace61b1d3a93cb9ba6bc752f4671346051641641c208b996a5d573f9e/drpms/kernel-plus-doc-3.10.0-514.el7.centos.plus_3.10.0-514.2.2.el7.centos.plus.noarch.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/3d/2864f9ed4d3f09349896192290f7aa661b308d764da8facc7cb6deb5fd7099/drpms/kernel-plus-tools-libs-3.10.0-327.28.3.el7.centos.plus_3.10.0-514.6.1.el7.centos.plus.x86_64.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/3d/2864f9ed4d3f09349896192290f7aa661b308d764da8facc7cb6deb5fd7099/drpms/kernel-plus-tools-libs-3.10.0-327.28.3.el7.centos.plus_3.10.0-514.6.1.el7.centos.plus.x86_64.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
Mar 28 13:15:00 foreman-01 pulp_streamer: pulp.streamer.server:ERROR:
Failed to find a catalog entry with path
"/var/lib/pulp/content/units/drpm/31/7797cc56856fc0e761dcaf5e2751b94b48ef2b009a31a42b70ff09d40242c5/drpms/kernel-plus-doc-3.10.0-327.36.2.el7.centos.plus_3.10.0-514.el7.centos.plus.noarch.drpm".
Mar 28 13:15:00 foreman-01 pulp_streamer: [-] 127.0.0.1 - -
[28/Mar/2017:13:15:00 +0000] "GET
/var/lib/pulp/content/units/drpm/31/7797cc56856fc0e761dcaf5e2751b94b48ef2b009a31a42b70ff09d40242c5/drpms/kernel-plus-doc-3.10.0-327.36.2.el7.centos.plus_3.10.0-514.el7.centos.plus.noarch.drpm
HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5
Linux/3.10.0-514.6.2.el7.x86_64"
If this is a problem specific to my environment/installation, I think I
remember seeing that there is/will be a tool to fully sync and repair these
repos - is that available for use now?
--
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
