I’ve been having failures syncing with certain RH repositories as well. For me, it’s the 7Workstation and 5Client repo’s (7Server doesn’t have any problems). I get a “Forbidden”, which would seem to indicate a problem with the cert… but this sync problem is intermittent. If I manually kick off a sync, sometimes it works, often it doesn’t.
Mar 23 10:24:07 yummy pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[f8759f66-7416-4fd1-9ee4-6005ca06df8a] succeeded in 0.0339540460263s: None Mar 23 10:24:07 yummy pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/. Mar 23 10:24:07 yummy pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) sync failed Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) Traceback (most recent call last): Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 104, in run Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) metadata_files = self.get_metadata() Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 198, in get_metadata Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) raise FailedException(str(e)) Mar 23 10:24:08 yummy pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (1986-41632) FailedException: Forbidden Mar 23 10:24:08 yummy pulp: celery.worker.job:ERROR: (1595-41632) Task pulp.server.managers.repo.sync.sync[80a6866e-0ed6-456a-8b7a-8d0cadbd84aa] raised unexpected: PulpExecutionException('Importer indicated a failed response',) Here’s two back-to-back sync’s, first failed (messages above), second was fine. # pulp-admin rpm repo sync run --repo-id=base-7Workstation +----------------------------------------------------------------------+ Synchronizing Repository [base-7Workstation] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [-] ... failed Forbidden Task Failed Importer indicated a failed response # pulp-admin rpm repo sync run --repo-id=base-7Workstation +----------------------------------------------------------------------+ Synchronizing Repository [base-7Workstation] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [/] ... completed Downloading repository content... [==================================================] 100% RPMs: 0/0 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [-] ... completed Importing package groups/categories... [-] ... completed Task Succeeded Copying files [\] ... completed Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [-] ... completed Publishing Delta RPMs ... skipped Publishing Errata [-] ... completed Publishing Comps file [==================================================] 100% 86 of 86 items ... completed Publishing Metadata. [-] ... completed Closing repo metadata [-] ... completed Generating sqlite files [|] ... completed Publishing files to web [|] ... completed Writing Listings File [-] ... completed Writing Listings File [\] ... completed Task Succeeded —- The other issue I have is that I’m syncing the kickstart trees…. When I first setup Pulp, it was syncing 7.1 server and workstation. Early last week, my workstation repo looks to have fallen back to 7.0. Looking at the local workstation treeinfo file, it lists version=7.0, whereas my server treeinfo lists 7.1. I used curl to see what’s on the RH repo, treeinfo for both workstation and server list 7.0. (I disabled my server sync so that it remains at 7.1) These are the two feeds I’m syncing against: https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/ https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/kickstart/ It’s weird, if I look at the redhat-release package, it’s 7.1: # ls -al /var/lib/pulp/published/yum/http/repos/base-7Workstation/redhat-release-workstation-7.1-1.el7.x86_64.rpm But the images/pxeboot files are all from 7.0 # ls -al /var/lib/pulp/content/distribution/ks-Red\ Hat\ Enterprise\ Linux-Workstation-7.0-x86_64/images/pxeboot/ total 72456 drwxr-xr-x. 2 apache apache 55 Mar 14 14:44 . drwxr-xr-x. 3 apache apache 20 Mar 14 14:44 .. -rw-r--r--. 1 apache apache 35544564 Mar 14 14:44 initrd.img -rw-r--r--. 1 apache apache 33744152 Mar 14 14:44 upgrade.img -rw-r--r--. 1 apache apache 4902000 Mar 14 14:44 vmlinuz There is no longer a 7.1 workstation distribution directory. # ls -al /var/lib/pulp/content/distribution/ total 8 drwxr-xr-x. 5 apache apache 4096 Mar 18 08:46 . drwxr-xr-x. 6 apache apache 74 Mar 14 14:36 .. drwxrwxr-x. 5 apache apache 4096 Mar 16 14:31 ks-Red Hat Enterprise Linux Client-5.11-x86_64 drwxrwxr-x. 5 apache apache 60 Mar 14 22:30 ks-Red Hat Enterprise Linux-Server-7.1-x86_64 drwxrwxr-x. 4 apache apache 47 Mar 14 14:44 ks-Red Hat Enterprise Linux-Workstation-7.0-x86_64 I’ve only been using Pulp for 2 weeks, so I’m learning as I go. The above doesn’t seem right, but maybe I don’t have something configured correctly? Thanks for any help, Reece On 3/23/15, 7:39 AM, "Paul Jochum" <paul.joc...@alcatel-lucent.com> wrote: >Hi Barnaby: > >$ rpm -qa | grep pulp >python-pulp-rpm-common-2.5.3-1.el6.noarch >pulp-server-2.5.3-1.el6.noarch >pulp-puppet-plugins-2.5.3-1.el6.noarch >pulp-selinux-2.5.3-1.el6.noarch >python-isodate-0.5.0-4.pulp.el6.noarch >python-kombu-3.0.15-12.pulp.el6.noarch >python-pulp-puppet-common-2.5.3-1.el6.noarch >python-pulp-bindings-2.5.3-1.el6.noarch >pulp-admin-client-2.5.3-1.el6.noarch >mod_wsgi-3.4-2.pulp.el6.x86_64 >pulp-rpm-plugins-2.5.3-1.el6.noarch >pulp-puppet-admin-extensions-2.5.3-1.el6.noarch >python-pulp-common-2.5.3-1.el6.noarch >python-pulp-client-lib-2.5.3-1.el6.noarch >pulp-rpm-admin-extensions-2.5.3-1.el6.noarch > >thanks, >Paul > > >On 03/23/2015 09:25 AM, Barnaby Court wrote: >> Hi Paul, What version of Pulp are you running? "rpm -qa | grep pulp " should >> give all the pertinent information. Thanks! >> >> -Barnaby >> >> ----- Original Message ----- >> From: "Paul Jochum" <paul.joc...@alcatel-lucent.com> >> To: pulp-list@redhat.com >> Sent: Monday, March 23, 2015 8:50:48 AM >> Subject: [Pulp-list] Pulp sync failure - not well-formed (invalid token): >> line 44436, column 0 >> >> Hi All: >> >> For the last 2 weeks, one of my daily pulp sync's has been >> failing. It is always the same repo (9 other syncs on that server run >> fine). All of the sync's are to Red Hat, for different RHEL 7 >> repositories. The one that is failing is for rhel-x86_64-server-7-source. >> >> This is what I see for the failing sync: >> ***************************************************************************** >> >> $ pulp-admin -u admin -p admin rpm repo sync run --repo-id=$i >> +----------------------------------------------------------------------+ >> Synchronizing Repository [rhel-x86_64-server-7-source] >> +----------------------------------------------------------------------+ >> >> This command may be exited via ctrl+c without affecting the request. >> >> >> Downloading metadata... >> [\] >> ... failed >> >> not well-formed (invalid token): line 44436, column 0 >> >> >> Task Failed >> >> Importer indicated a failed response >> >> ***************************************************************************** >> And this is in /var/log/messages: >> Mar 23 07:43:39 lss-pulp02 pulp: celery.worker.strategy:INFO: Received >> task: >> pulp.server.async.tasks._reserve_resource[94539898-05e5-4800-aa61-b75aa05a38c5] >> Mar 23 07:43:39 lss-pulp02 pulp: celery.worker.job:INFO: Task >> pulp.server.async.tasks._reserve_resource[94539898-05e5-4800-aa61-b75aa05a38c5] >> succeeded in 0.0402820348972s: >> u'reserved_resource_worke...@lss-pulp02.ih.lucent.com.dq' >> Mar 23 07:43:39 lss-pulp02 pulp: celery.worker.strategy:INFO: Received >> task: >> pulp.server.tasks.repository.sync_with_auto_publish[e8fe2c7a-ee33-4548-bbf4-176e2018a997] >> Mar 23 07:43:39 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from >> https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/source/SRPMS/. >> Mar 23 07:43:39 lss-pulp02 pulp: >> requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS >> connection (1): cdn.redhat.com >> Mar 23 07:43:40 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:INFO: Parsing metadata. >> Mar 23 07:43:40 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata files. >> Mar 23 07:43:40 lss-pulp02 pulp: >> requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS >> connection (1): cdn.redhat.com >> Mar 23 07:43:40 lss-pulp02 pulp: >> requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS >> connection (1): cdn.redhat.com >> Mar 23 07:43:40 lss-pulp02 pulp: >> requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS >> connection (1): cdn.redhat.com >> Mar 23 07:43:40 lss-pulp02 pulp: >> requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS >> connection (1): cdn.redhat.com >> Mar 23 07:43:42 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:INFO: Generating metadata databases. >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: Traceback (most recent call >> last): >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", >> line 105, in run >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: metadata_files = >> self.get_metadata() >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", >> line 212, in get_metadata >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: metadata_files.generate_dbs() >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", >> line 275, in generate_dbs >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: for element in generator: >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", >> line 53, in package_list_generator >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: for event, element in >> xml_iterator: >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: File "<string>", line 64, >> in __iter__ >> Mar 23 07:43:45 lss-pulp02 pulp: >> pulp_rpm.plugins.importers.yum.sync:ERROR: SyntaxError: not well-formed >> (invalid token): line 44436, column 0 >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: Task >> pulp.server.tasks.repository.sync_with_auto_publish[e8fe2c7a-ee33-4548-bbf4-176e2018a997] >> raised unexpected: PulpExecutionException('Importer indicated a failed >> response',) >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: Traceback >> (most recent call last): >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: File >> "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in >> trace_task >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: R = retval >> = fun(*args, **kwargs) >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 316, >> in __call__ >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: return >> super(Task, self).__call__(*args, **kwargs) >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: File >> "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in >> __protected_call__ >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: return >> self.run(*args, **kwargs) >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp/server/tasks/repository.py", line >> 210, in sync_with_auto_publish >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: sync_result = >> managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides) >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: File >> "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", >> line 113, in sync >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: raise >> PulpExecutionException(_('Importer indicated a failed response')) >> Mar 23 07:43:45 lss-pulp02 pulp: celery.worker.job:ERROR: >> PulpExecutionException: Importer indicated a failed response >> >> ***************************************************************************** >> >> Any suggestions on how to clean it up? Would any additional information >> help? >> >> thanks, >> Paul >> >> _______________________________________________ >> Pulp-list mailing list >> Pulp-list@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-list > >_______________________________________________ >Pulp-list mailing list >Pulp-list@redhat.com >https://www.redhat.com/mailman/listinfo/pulp-list _______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://www.redhat.com/mailman/listinfo/pulp-list