No the one that the pulpcore-api service isn’t started and when trying to start manually I get that error for the jsquery not found
Kind regards On Thu, 15 Apr 2021 at 07:50 David Davis <davidda...@redhat.com> wrote: > When you say you run the playbook again and got the same error, which > error do you mean? The "No such file or directory: > '/var/lib/pulp/assets/rest_framework/js/jquery-3.4.1.min.js'" one? > > > David > > > On Thu, Apr 15, 2021 at 4:43 AM Eric VS <vs.e...@gmail.com> wrote: > >> Hey Matthias, >> >> I ran the pulpcore-manager collectstatic with success, then the playbook >> again but got the same error. This is the full list of commands for >> pulpcore-manager, any others I would need to run? >> >> Last login: Thu Apr 15 16:23:00 CST 2021 on pts/0 >> [root@tw-stg-man01 ~]# export PULP_SETTINGS=/etc/pulp/settings.py && >> /usr/local/lib/pulp/bin/pulpcore-manager >> >> Type 'pulpcore-manager help <subcommand>' for help on a specific >> subcommand. >> >> Available subcommands: >> >> [app] >> handle-artifact-checksums >> reset-admin-password >> stage-profile-summary >> >> [auth] >> changepassword >> createsuperuser >> >> [contenttypes] >> remove_stale_contenttypes >> >> [django] >> check >> compilemessages >> createcachetable >> dbshell >> diffsettings >> dumpdata >> flush >> inspectdb >> loaddata >> makemessages >> makemigrations >> migrate >> sendtestemail >> shell >> showmigrations >> sqlflush >> sqlmigrate >> sqlsequencereset >> squashmigrations >> startapp >> startproject >> test >> testserver >> >> [drf_spectacular] >> spectacular >> >> [guardian] >> clean_orphan_obj_perms >> >> [rest_framework] >> generateschema >> >> [sessions] >> clearsessions >> >> [staticfiles] >> collectstatic >> findstatic >> runserver >> >> *Eric Van Steenbergen* >> >> *E-mail: vs.e...@gmail.com <vs.e...@gmail.com>* >> *Skype: ericvs2014* >> *LinkedIn: Eric Van Steenbergen >> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>* >> >> >> On Thu, Apr 15, 2021 at 3:13 AM Matthias Dellweg <mdell...@redhat.com> >> wrote: >> >>> Yeah, i can see how that happened. There may be some triggers from the >>> previous run that aren't triggered now, because "nothing changed" according >>> to ansible. >>> This one should be: pulpcore-manager collect-static (or very similar) >>> When you just run pulpcore-manager you get a list where you should be >>> able to spot that subcommand. >>> >>> On Thu, Apr 15, 2021 at 10:06 AM Eric VS <vs.e...@gmail.com> wrote: >>> >>>> Hi Matthias, >>>> >>>> I did try to run that command but thought it was failing due to the >>>> last error. >>>> Unfortunately, when I just tried running the playbook again I ran into >>>> the next error :-) >>>> >>>> TASK [pulp.pulp_installer.pulp_health_check : Checking Pulp services] >>>> *********************************************************************************************************************************************************** >>>> skipping: [localhost] => (item={'key': 'pulpcore-worker@2.service', >>>> 'value': {'status': 'unknown', 'source': 'systemd', 'state': 'running', >>>> 'name': 'pulpcore-worker@2.service'}}) >>>> skipping: [localhost] => (item={'key': >>>> 'pulpcore-resource-manager.service', 'value': {'status': 'enabled', >>>> 'source': 'systemd', 'state': 'running', 'name': >>>> 'pulpcore-resource-manager.service'}}) >>>> skipping: [localhost] => (item={'key': 'pulpcore-worker@.service', >>>> 'value': {'status': 'enabled', 'source': 'systemd', 'state': 'unknown', >>>> 'name': 'pulpcore-worker@.service'}}) >>>> skipping: [localhost] => (item={'key': 'pulpcore-worker@1.service', >>>> 'value': {'status': 'unknown', 'source': 'systemd', 'state': 'running', >>>> 'name': 'pulpcore-worker@1.service'}}) >>>> failed: [localhost] (item={'key': 'pulpcore-api.service', 'value': >>>> {'status': 'enabled', 'source': 'systemd', 'state': 'stopped', 'name': >>>> 'pulpcore-api.service'}}) => {"ansible_loop_var": "item", "changed": false, >>>> "item": {"key": "pulpcore-api.service", "value": {"name": >>>> "pulpcore-api.service", "source": "systemd", "state": "stopped", "status": >>>> "enabled"}}, "msg": "pulpcore-api.service state: stopped"} >>>> skipping: [localhost] => (item={'key': 'pulpcore-content.service', >>>> 'value': {'status': 'enabled', 'source': 'systemd', 'state': 'running', >>>> 'name': 'pulpcore-content.service'}}) >>>> >>>> PLAY RECAP >>>> ********************************************************************************************************************************************************************************************************************** >>>> localhost : ok=115 changed=19 unreachable=0 >>>> failed=1 skipped=50 rescued=0 ignored=0 >>>> >>>> (ansible-venv) ╭─ericv@tw-stg-man01 ~ >>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) sudo systemctl start >>>> pulpcore-api >>>> >>>> 2 ↵ >>>> (ansible-venv) ╭─ericv@tw-stg-man01 ~ >>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) ^start^status >>>> (ansible-venv) ╭─ericv@tw-stg-man01 ~ >>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) sudo systemctl status >>>> pulpcore-api >>>> ● pulpcore-api.service - Pulp WSGI Server >>>> Loaded: loaded (/usr/lib/systemd/system/pulpcore-api.service; >>>> enabled; vendor preset: disabled) >>>> Active: activating (auto-restart) (Result: exit-code) since Thu >>>> 2021-04-15 16:03:12 CST; 1s ago >>>> Process: 4875 ExecStart=/usr/local/lib/pulp/bin/gunicorn >>>> pulpcore.app.wsgi:application --bind 127.0.0.1:24817 --workers 1 >>>> --timeout 90 --access-logfile - --access-logformat pulp >>>> [%({correlation-id}o)s]: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s >>>> "%(f)s" "%(a)s" (code=exited, status=3) >>>> Main PID: 4875 (code=exited, status=3) >>>> >>>> Apr 15 16:03:12 tw-stg-man01.stg01.activeinfra.net systemd[1]: Unit >>>> pulpcore-api.service entered failed state. >>>> Apr 15 16:03:12 tw-stg-man01.stg01.activeinfra.net systemd[1]: >>>> pulpcore-api.service failed. >>>> (ansible-venv) ╭─ericv@tw-stg-man01 ~ >>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) sudo journalctl -u >>>> pulpcore-api >>>> >>>> 3 ↵ >>>> -- Logs begin at Mon 2020-07-20 11:44:39 CST, end at Thu 2021-04-15 >>>> 16:03:38 CST. -- >>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Started Pulp WSGI Server. >>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 10:24:09 +0800] [815] [INFO] Starting gunicorn 20.0.4 >>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 10:24:09 +0800] [815] [INFO] Listening at: >>>> http://127.0.0.1:24817 (815) >>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 10:24:09 +0800] [815] [INFO] Using worker: sync >>>> Aug 05 10:24:09 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 10:24:09 +0800] [923] [INFO] Booting worker with pid: 923 >>>> Aug 05 10:24:27 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Stopping Pulp WSGI Server... >>>> Aug 05 10:24:27 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 10:24:27 +0800] [815] [INFO] Handling signal: term >>>> Aug 05 10:24:27 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 02:24:27 +0000] [923] [INFO] Worker exiting (pid: 923) >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[815]: >>>> [2020-08-05 10:24:28 +0800] [815] [INFO] Shutting down: Master >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Stopped Pulp WSGI Server. >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Started Pulp WSGI Server. >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 10:24:28 +0800] [3794] [INFO] Starting gunicorn 20.0.4 >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 10:24:28 +0800] [3794] [INFO] Listening at: >>>> http://127.0.0.1:24817 (3794) >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 10:24:28 +0800] [3794] [INFO] Using worker: sync >>>> Aug 05 10:24:28 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 10:24:28 +0800] [3873] [INFO] Booting worker with pid: 3873 >>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 11:16:03 +0800] [3794] [INFO] Handling signal: term >>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Stopping Pulp WSGI Server... >>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 03:16:03 +0000] [3873] [INFO] Worker exiting (pid: 3873) >>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net gunicorn[3794]: >>>> [2020-08-05 11:16:03 +0800] [3794] [INFO] Shutting down: Master >>>> Aug 05 11:16:03 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Stopped Pulp WSGI Server. >>>> -- Reboot -- >>>> Aug 05 11:19:03 tw-stg-man01.app-stg01.activeinfra.net systemd[1]: >>>> Started Pulp WSGI Server. >>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> [2020-08-05 11:19:05 +0800] [1858] [INFO] Starting gunicorn 20.0.4 >>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> [2020-08-05 11:19:05 +0800] [1858] [INFO] Listening at: >>>> http://127.0.0.1:24817 (1858) >>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> [2020-08-05 11:19:05 +0800] [1858] [INFO] Using worker: sync >>>> Aug 05 11:19:05 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> [2020-08-05 11:19:05 +0800] [2329] [INFO] Booting worker with pid: 2329 >>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> pulp: django.request:WARNING: Not Found: / >>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [05/Aug/2020:03:38:18 +0000] "GET / HTTP/1.0" 404 77 "-" >>>> "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:81.0) Gecko/20100101 >>>> Firefox/81.0 >>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> pulp: django.request:WARNING: Not Found: /favicon.ico >>>> Aug 05 11:38:18 tw-stg-man01.app-stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [05/Aug/2020:03:38:18 +0000] "GET /favicon.ico HTTP/1.0" 404 >>>> 77 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:81.0) >>>> Gecko/20100101 F >>>> Aug 07 13:40:58 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [07/Aug/2020:05:40:58 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0" >>>> Aug 07 13:41:07 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [07/Aug/2020:05:41:07 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0" >>>> Aug 07 13:41:43 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [07/Aug/2020:05:41:43 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0" >>>> Aug 07 13:44:48 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [07/Aug/2020:05:44:48 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0" >>>> Aug 07 13:44:59 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [07/Aug/2020:05:44:59 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "curl/7.29.0" >>>> Aug 07 13:48:11 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [07/Aug/2020:05:48:11 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "HTTPie/0.9.4" >>>> Aug 10 10:34:07 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [10/Aug/2020:02:34:07 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "HTTPie/0.9.4" >>>> Aug 10 12:55:34 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - - [10/Aug/2020:04:55:34 +0000] "GET /pulp/api/v3/status/ >>>> HTTP/1.0" 200 1129 "-" "HTTPie/0.9.4" >>>> Aug 10 13:01:47 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> pulp: django.request:WARNING: Not Found: / >>>> Aug 10 13:01:47 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:01:47 +0000] "GET / HTTP/1.1" 404 77 "-" >>>> "HTTPie/0.9.4" >>>> Aug 10 13:02:01 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> pulp: django.request:WARNING: Not Found: / >>>> Aug 10 13:02:01 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:02:01 +0000] "GET / HTTP/1.1" 404 77 "-" >>>> "HTTPie/0.9.4" >>>> Aug 10 13:03:16 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:16 +0000] "POST >>>> /pulp/api/v3/repositories/rpm/rpm/ HTTP/1.1" 201 441 "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:16 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:16 +0000] "GET >>>> /pulp/api/v3/repositories/rpm/rpm/46d2f3e3-904e-448c-ae36-8d0662788f8e/ >>>> HTTP/1.1" 200 441 "-" "HTTPie/0.9. >>>> Aug 10 13:03:16 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:16 +0000] "POST >>>> /pulp/api/v3/remotes/rpm/rpm/ HTTP/1.1" 201 462 "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "GET >>>> /pulp/api/v3/remotes/rpm/rpm/ HTTP/1.1" 200 514 "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "GET >>>> /pulp/api/v3/remotes/rpm/rpm/a4e0901e-5091-4a3e-ad6d-41f1f0c88d1e/ >>>> HTTP/1.1" 200 462 "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "POST >>>> /pulp/api/v3/repositories/rpm/rpm/46d2f3e3-904e-448c-ae36-8d0662788f8e/sync/ >>>> HTTP/1.1" 202 67 "-" "HTTPie >>>> Aug 10 13:03:17 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:17 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 606 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:19 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:19 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1094 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:20 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:20 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:22 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:22 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:23 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:23 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:24 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:24 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:26 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:26 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:27 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:27 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:28 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:28 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:30 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:30 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:31 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:31 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> Aug 10 13:03:33 tw-stg-man01.stg01.activeinfra.net gunicorn[1858]: >>>> 127.0.0.1 - admin [10/Aug/2020:05:03:33 +0000] "GET >>>> /pulp/api/v3/tasks/169ca438-1c83-4af6-899c-ad228fc551bc/ HTTP/1.1" 200 1201 >>>> "-" "HTTPie/0.9.4" >>>> ...skipping... >>>> Apr 15 16:03:34 tw-stg-man01.stg01.activeinfra.net systemd[1]: >>>> pulpcore-api.service: main process exited, code=exited, >>>> status=3/NOTIMPLEMENTED >>>> Apr 15 16:03:34 tw-stg-man01.stg01.activeinfra.net systemd[1]: Unit >>>> pulpcore-api.service entered failed state. >>>> Apr 15 16:03:34 tw-stg-man01.stg01.activeinfra.net systemd[1]: >>>> pulpcore-api.service failed. >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net systemd[1]: >>>> pulpcore-api.service holdoff time over, scheduling restart. >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net systemd[1]: Stopped >>>> Pulp WSGI Server. >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net systemd[1]: Started >>>> Pulp WSGI Server. >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 16:03:37 +0800] [4938] [INFO] Starting gunicorn 20.1.0 >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 16:03:37 +0800] [4938] [INFO] Listening at: >>>> http://127.0.0.1:24817 (4938) >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 16:03:37 +0800] [4938] [INFO] Using worker: sync >>>> Apr 15 16:03:37 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 16:03:37 +0800] [4941] [INFO] Booting worker with pid: 4941 >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 08:03:39 +0000] [4941] [ERROR] Exception in worker process >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> Traceback (most recent call last): >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/arbiter.py", >>>> line 589, in spawn_worker >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> worker.init_process() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", >>>> line 134, in init_process >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> self.load_wsgi() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/workers/base.py", >>>> line 146, in load_wsgi >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> self.wsgi = self.app.wsgi() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/base.py", >>>> line 67, in wsgi >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> self.callable = self.load() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", >>>> line 58, in load >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> return self.load_wsgiapp() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/app/wsgiapp.py", >>>> line 48, in load_wsgiapp >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> return util.import_app(self.app_uri) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "/usr/local/lib/pulp/lib64/python3.6/site-packages/gunicorn/util.py", >>>> line 359, in import_app >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: mod >>>> = importlib.import_module(module) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in >>>> import_module >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> return _bootstrap._gcd_import(name[level:], package, level) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "<frozen importlib._bootstrap>", line 994, in _gcd_import >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "<frozen importlib._bootstrap>", line 971, in _find_and_load >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "<frozen importlib._bootstrap>", line 665, in _load_unlocked >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "<frozen importlib._bootstrap_external>", line 678, in exec_module >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File "<frozen importlib._bootstrap>", line 219, in >>>> _call_with_frames_removed >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/wsgi.py", >>>> line 17, in <module> >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> application = get_wsgi_application() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/wsgi.py", >>>> line 13, in get_wsgi_application >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> return WSGIHandler() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/wsgi.py", >>>> line 135, in __init__ >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> self.load_middleware() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/handlers/base.py", >>>> line 37, in load_middleware >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> mw_instance = middleware(handler) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/middleware.py", >>>> line 49, in __init__ >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> self.add_files(self.static_root, prefix=self.static_prefix) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", >>>> line 113, in add_files >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> self.update_files_dictionary(root, prefix) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", >>>> line 120, in update_files_dictionary >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> stat_cache = dict(scantree(root)) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", >>>> line 271, in scantree >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> yield from scantree(entry.path) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", >>>> line 271, in scantree >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> yield from scantree(entry.path) >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> File >>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/whitenoise/base.py", >>>> line 273, in scantree >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> yield entry.path, entry.stat() >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> FileNotFoundError: [Errno 2] No such file or directory: >>>> '/var/lib/pulp/assets/rest_framework/js/jquery-3.4.1.min.js' >>>> Apr 15 16:03:39 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 08:03:39 +0000] [4941] [INFO] Worker exiting (pid: 4941) >>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 16:03:40 +0800] [4938] [INFO] Shutting down: Master >>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net gunicorn[4938]: >>>> [2021-04-15 16:03:40 +0800] [4938] [INFO] Reason: Worker failed to boot. >>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net systemd[1]: >>>> pulpcore-api.service: main process exited, code=exited, >>>> status=3/NOTIMPLEMENTED >>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net systemd[1]: Unit >>>> pulpcore-api.service entered failed state. >>>> Apr 15 16:03:40 tw-stg-man01.stg01.activeinfra.net systemd[1]: >>>> pulpcore-api.service failed. >>>> >>>> Kind regards, >>>> >>>> *Eric Van Steenbergen* >>>> >>>> *E-mail: vs.e...@gmail.com <vs.e...@gmail.com>* >>>> *Skype: ericvs2014* >>>> *LinkedIn: Eric Van Steenbergen >>>> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>* >>>> >>>> >>>> On Thu, Apr 15, 2021 at 2:56 AM Matthias Dellweg <mdell...@redhat.com> >>>> wrote: >>>> >>>>> >>>>> There is the clue: Run 'pulpcore-manager handle-artifact-checksums' >>>>> first to unset forbidden checksums. >>>>> Pulp dropped to support md5 and sha1 by default. And there is a manual >>>>> step that should bring the database in the right place. >>>>> Try rerunning the ansible playbook afterwards. >>>>> >>>>> However i thought this cleanup routine should have been called >>>>> automatically (as a migration) in the upgrade process. Can anyone spot why >>>>> it didn't? >>>>> >>>>> On Thu, Apr 15, 2021 at 1:48 AM Eric VS <vs.e...@gmail.com> wrote: >>>>> >>>>>> Hey Matthias, >>>>>> >>>>>> The only plugin I am currently using, at least until I'm familiar >>>>>> enough with Pulp, is the RPM plugin. That's our primary use case for now >>>>>> to >>>>>> make sure we are in control of what version of packages we are using >>>>>> across >>>>>> our environments and avoid discrepancies between servers. >>>>>> >>>>>> I have never added or removed any other plugin. After also upgrading >>>>>> the Postgres ansible-galaxy role I don't get the version mismatch error >>>>>> any >>>>>> more but now get the below error and 'command to run'. Running the >>>>>> command >>>>>> fails however also.... >>>>>> >>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Collect static >>>>>> content] >>>>>> ****************************************************************************************************************************************************** >>>>>> fatal: [localhost]: FAILED! => {"changed": true, "cmd": >>>>>> ["/usr/local/lib/pulp/bin/pulpcore-manager", "collectstatic", >>>>>> "--noinput", >>>>>> "--link"], "delta": "0:00:02.481870", "end": "2021-04-15 >>>>>> 07:34:39.205489", >>>>>> "msg": "non-zero return code", "rc": 1, "start": "2021-04-15 >>>>>> 07:34:36.723619", "stderr": "Traceback (most recent call last):\n File >>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>\n >>>>>> sys.exit(manage())\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\", >>>>>> line 11, in manage\n execute_from_command_line(sys.argv)\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>> line 381, in execute_from_command_line\n utility.execute()\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>> line 357, in execute\n django.setup()\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py\", >>>>>> line 24, in setup\n apps.populate(settings.INSTALLED_APPS)\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py\", >>>>>> line 114, in populate\n app_config.import_models()\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py\", >>>>>> line 211, in import_models\n self.models_module = >>>>>> import_module(models_module_name)\n File >>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in >>>>>> import_module\n return _bootstrap._gcd_import(name[level:], package, >>>>>> level)\n File \"<frozen importlib._bootstrap>\", line 994, in >>>>>> _gcd_import\n File \"<frozen importlib._bootstrap>\", line 971, in >>>>>> _find_and_load\n File \"<frozen importlib._bootstrap>\", line 955, in >>>>>> _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line >>>>>> 665, in _load_unlocked\n File \"<frozen >>>>>> importlib._bootstrap_external>\", >>>>>> line 678, in exec_module\n File \"<frozen importlib._bootstrap>\", line >>>>>> 219, in _call_with_frames_removed\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/__init__.py\", >>>>>> line 22, in <module>\n from .exporter import ( # noqa\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/exporter.py\", >>>>>> line 15, in <module>\n from .task import CreatedResource, Task\n >>>>>> >>>>>> File\"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/task.py\", >>>>>> line 14, in <module>\n from pulpcore.app.settings import WORKER_TTL\n >>>>>> File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\", >>>>>> line 366, in <module>\n raise e\n File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\", >>>>>> line 344, in <module>\n >>>>>> ).format(checksum)\ndjango.core.exceptions.ImproperlyConfigured: There >>>>>> have been identified artifacts with forbidden checksum 'sha1'. Run >>>>>> 'pulpcore-manager handle-artifact-checksums' first to unset forbidden >>>>>> checksums.", "stderr_lines": ["Traceback (most recent call last):", " >>>>>> File >>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>", " >>>>>> sys.exit(manage())", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\", >>>>>> line 11, in manage", " execute_from_command_line(sys.argv)", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>> line 381, in execute_from_command_line", " utility.execute()", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>> line 357, in execute", " django.setup()", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py\", >>>>>> line 24, in setup", " apps.populate(settings.INSTALLED_APPS)", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py\", >>>>>> line 114, in populate", " app_config.import_models()", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py\", >>>>>> line 211, in import_models", " self.models_module = >>>>>> import_module(models_module_name)", " File >>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in >>>>>> import_module", " return _bootstrap._gcd_import(name[level:], package, >>>>>> level)", " File \"<frozen importlib._bootstrap>\", line 994, in >>>>>> _gcd_import", " File \"<frozen importlib._bootstrap>\", line 971, in >>>>>> _find_and_load", " File \"<frozen importlib._bootstrap>\", line 955, in >>>>>> _find_and_load_unlocked", " File \"<frozen importlib._bootstrap>\", line >>>>>> 665, in _load_unlocked", " File \"<frozen >>>>>> importlib._bootstrap_external>\", line 678, in exec_module", " File >>>>>> \"<frozen importlib._bootstrap>\", line 219, in >>>>>> _call_with_frames_removed", >>>>>> " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/__init__.py\", >>>>>> line 22, in <module>", " from .exporter import ( # noqa", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/exporter.py\", >>>>>> line 15, in <module>", " from .task import CreatedResource, Task", " >>>>>> File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/task.py\", >>>>>> line 14, in <module>", " from pulpcore.app.settings import >>>>>> WORKER_TTL", >>>>>> " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\", >>>>>> line 366, in <module>", " raise e", " File >>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\", >>>>>> line 344, in <module>", " ).format(checksum)", >>>>>> "django.core.exceptions.ImproperlyConfigured: There have been identified >>>>>> artifacts withforbidden checksum 'sha1'. Run 'pulpcore-manager >>>>>> handle-artifact-checksums' first to unset forbidden checksums."], >>>>>> "stdout": >>>>>> "", "stdout_lines": []} >>>>>> >>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Restart all Pulp >>>>>> services] >>>>>> *************************************************************************************************************************************************** >>>>>> >>>>>> NO MORE HOSTS LEFT >>>>>> ************************************************************************************************************************************************************************************************************** >>>>>> >>>>>> PLAY RECAP >>>>>> ********************************************************************************************************************************************************************************************************************** >>>>>> localhost : ok=87 changed=3 unreachable=0 >>>>>> failed=1 skipped=48 rescued=0 ignored=0 >>>>>> >>>>>> (ansible-venv) ╭─ericv@tw-stg-man01 ~ >>>>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) export >>>>>> PULP_SETTINGS=/etc/pulp/settings.py && >>>>>> /usr/local/lib/pulp/bin/pulpcore-manager handle-artifact-checksums >>>>>> 2 ↵ >>>>>> Traceback (most recent call last): >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py", >>>>>> line 283, in <module> >>>>>> CONTENT_ORIGIN >>>>>> NameError: name 'CONTENT_ORIGIN' is not defined >>>>>> >>>>>> During handling of the above exception, another exception occurred: >>>>>> >>>>>> Traceback (most recent call last): >>>>>> File "/usr/local/lib/pulp/bin/pulpcore-manager", line 8, in <module> >>>>>> sys.exit(manage()) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py", >>>>>> line 11, in manage >>>>>> execute_from_command_line(sys.argv) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", >>>>>> line 381, in execute_from_command_line >>>>>> utility.execute() >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", >>>>>> line 357, in execute >>>>>> django.setup() >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/__init__.py", >>>>>> line 24, in setup >>>>>> apps.populate(settings.INSTALLED_APPS) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/registry.py", >>>>>> line 114, in populate >>>>>> app_config.import_models() >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/apps/config.py", >>>>>> line 211, in import_models >>>>>> self.models_module = import_module(models_module_name) >>>>>> File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in >>>>>> import_module >>>>>> return _bootstrap._gcd_import(name[level:], package, level) >>>>>> File "<frozen importlib._bootstrap>", line 994, in _gcd_import >>>>>> File "<frozen importlib._bootstrap>", line 971, in _find_and_load >>>>>> File "<frozen importlib._bootstrap>", line 955, in >>>>>> _find_and_load_unlocked >>>>>> File "<frozen importlib._bootstrap>", line 665, in _load_unlocked >>>>>> File "<frozen importlib._bootstrap_external>", line 678, in >>>>>> exec_module >>>>>> File "<frozen importlib._bootstrap>", line 219, in >>>>>> _call_with_frames_removed >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/__init__.py", >>>>>> line 22, in <module> >>>>>> from .exporter import ( # noqa >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/exporter.py", >>>>>> line 15, in <module> >>>>>> from .task import CreatedResource, Task >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/task.py", >>>>>> line 14, in <module> >>>>>> from pulpcore.app.settings import WORKER_TTL >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py", >>>>>> line 287, in <module> >>>>>> "CONTENT_ORIGIN is a required setting but it was not configured. >>>>>> This may be caused " >>>>>> django.core.exceptions.ImproperlyConfigured: CONTENT_ORIGIN is a >>>>>> required setting but it was not configured. This may be caused by invalid >>>>>> read permissions of the settings file. Note that CONTENT_ORIGIN is set by >>>>>> the installer automatically. >>>>>> (ansible-venv) ╭─ericv@tw-stg-man01 ~ >>>>>> ╰─$ (⎈ k8s01.stg01.activeinfra.net:linkerd) >>>>>> >>>>>> I finally had some time to dig deeper and played around with the >>>>>> permissions on the file to no result, until I ran the pulpcore-manager >>>>>> command as root. Then this popped up as error indicating there is some >>>>>> integrity error: >>>>>> >>>>>> [root@tw-stg-man01 ~]# export PULP_SETTINGS=/etc/pulp/settings.py && >>>>>> /usr/local/lib/pulp/bin/pulpcore-manager handle-artifact-checksums >>>>>> Removing forbidden checksum sha1 from database >>>>>> Traceback (most recent call last): >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", >>>>>> line 84, in _execute >>>>>> return self.cursor.execute(sql, params) >>>>>> psycopg2.errors.NotNullViolation: null value in column "sha1" >>>>>> violates not-null constraint >>>>>> DETAIL: Failing row contains (1883479d-9e2d-45e1-971f-37246fc3c358, >>>>>> 2020-08-13 05:50:33.543549+00, 2020-08-13 05:50:33.543572+00, >>>>>> artifact/52/e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac..., >>>>>> 286400, 51b3d1a3ff20a0f88ccb084eca86073c, null, >>>>>> bd036226130d705e95bf4ad9acaa5d8d6d4df7460943062c08154bf1, >>>>>> 52e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac0f16a27789, >>>>>> 53964efdef715dd8f8ff96588e7f1ab184a3747d39bb3c9c28fc339659014274..., >>>>>> 0b0d8a7d5d5816705016a5a59489f032fddc31afc438a482549091b730ca4f6b...). >>>>>> >>>>>> >>>>>> The above exception was the direct cause of the following exception: >>>>>> >>>>>> Traceback (most recent call last): >>>>>> File "/usr/local/lib/pulp/bin/pulpcore-manager", line 8, in <module> >>>>>> sys.exit(manage()) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py", >>>>>> line 11, in manage >>>>>> execute_from_command_line(sys.argv) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", >>>>>> line 381, in execute_from_command_line >>>>>> utility.execute() >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py", >>>>>> line 375, in execute >>>>>> self.fetch_command(subcommand).run_from_argv(self.argv) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/base.py", >>>>>> line 323, in run_from_argv >>>>>> self.execute(*args, **cmd_options) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/base.py", >>>>>> line 364, in execute >>>>>> output = self.handle(*args, **options) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/management/commands/handle-artifact-checksums.py", >>>>>> line 212, in handle >>>>>> artifacts_qs.update(**update_params) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", >>>>>> line 741, in update >>>>>> rows = query.get_compiler(self.db).execute_sql(CURSOR) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", >>>>>> line 1471, in execute_sql >>>>>> cursor = super().execute_sql(result_type) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", >>>>>> line 1142, in execute_sql >>>>>> cursor.execute(sql, params) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", >>>>>> line 67, in execute >>>>>> return self._execute_with_wrappers(sql, params, many=False, >>>>>> executor=self._execute) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", >>>>>> line 76, in _execute_with_wrappers >>>>>> return executor(sql, params, many, context) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", >>>>>> line 84, in _execute >>>>>> return self.cursor.execute(sql, params) >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", >>>>>> line 89, in __exit__ >>>>>> raise dj_exc_value.with_traceback(traceback) from exc_value >>>>>> File >>>>>> "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", >>>>>> line 84, in _execute >>>>>> return self.cursor.execute(sql, params) >>>>>> django.db.utils.IntegrityError: null value in column "sha1" violates >>>>>> not-null constraint >>>>>> DETAIL: Failing row contains (1883479d-9e2d-45e1-971f-37246fc3c358, >>>>>> 2020-08-13 05:50:33.543549+00, 2020-08-13 05:50:33.543572+00, >>>>>> artifact/52/e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac..., >>>>>> 286400, 51b3d1a3ff20a0f88ccb084eca86073c, null, >>>>>> bd036226130d705e95bf4ad9acaa5d8d6d4df7460943062c08154bf1, >>>>>> 52e229d511dca3c15b4250159cc7fd1685e8202cbb20e228bbd6ac0f16a27789, >>>>>> 53964efdef715dd8f8ff96588e7f1ab184a3747d39bb3c9c28fc339659014274..., >>>>>> 0b0d8a7d5d5816705016a5a59489f032fddc31afc438a482549091b730ca4f6b...). >>>>>> >>>>>> I could just delete the whole bunch and reinstall but that wouldn't >>>>>> teach me anything useful I guess so I'm sticking to my guns hoping you >>>>>> guys >>>>>> can help me out in understanding the root cause. >>>>>> >>>>>> Kind regards, >>>>>> >>>>>> *Eric Van Steenbergen* >>>>>> >>>>>> *E-mail: vs.e...@gmail.com <vs.e...@gmail.com>* >>>>>> *Skype: ericvs2014* >>>>>> *LinkedIn: Eric Van Steenbergen >>>>>> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>* >>>>>> >>>>>> >>>>>> On Wed, Apr 14, 2021 at 3:53 AM Matthias Dellweg <mdell...@redhat.com> >>>>>> wrote: >>>>>> >>>>>>> In your playbook, where there is the list of plugins to install, did >>>>>>> you ever remove one from that list? >>>>>>> Also can you try adding "upgrade: true" to all plugins in that list? >>>>>>> >>>>>>> On Wed, Apr 14, 2021 at 5:27 AM Eric VS <vs.e...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Me again :-). >>>>>>>> >>>>>>>> I had a Pulp installation from several weeks ago (on our staging >>>>>>>> environment) and wanted to align it today with my main Pulp >>>>>>>> installation >>>>>>>> running 3.12.0. >>>>>>>> The staging one was still running on 3.5.0, installed using the >>>>>>>> Ansible Pulp Installer so I followed the upgrade process for the >>>>>>>> collection >>>>>>>> and ran the playbook. This is the error I got, due to version mismatch >>>>>>>> but >>>>>>>> not sure on how to fix this one. >>>>>>>> >>>>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Restore SELinux >>>>>>>> contexts on Pulp dirs that may exist] >>>>>>>> ************************************************************************************************************************ >>>>>>>> changed: [localhost] >>>>>>>> >>>>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Collect static >>>>>>>> content] >>>>>>>> ****************************************************************************************************************************************************** >>>>>>>> fatal: [localhost]: FAILED! => {"changed": true, "cmd": >>>>>>>> ["/usr/local/lib/pulp/bin/pulpcore-manager", "collectstatic", >>>>>>>> "--noinput", >>>>>>>> "--link"], "delta": "0:00:00.297250", "end": "2021-04-14 >>>>>>>> 11:20:11.886134", >>>>>>>> "msg": "non-zero return code", "rc": 1, "start": "2021-04-14 >>>>>>>> 11:20:11.588884", "stderr": "Traceback (most recent call last):\n File >>>>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>\n >>>>>>>> sys.exit(manage())\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\", >>>>>>>> line 11, in manage\n execute_from_command_line(sys.argv)\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>>>> line 381, in execute_from_command_line\n utility.execute()\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>>>> line 325, in execute\n settings.INSTALLED_APPS\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\", >>>>>>>> line 79, in __getattr__\n self._setup(name)\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\", >>>>>>>> line 66, in _setup\n self._wrapped = Settings(settings_module)\n >>>>>>>> File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\", >>>>>>>> line 157, in __init__\n mod = >>>>>>>> importlib.import_module(self.SETTINGS_MODULE)\n File >>>>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in >>>>>>>> import_module\n return _bootstrap._gcd_import(name[level:], package, >>>>>>>> level)\n File \"<frozen importlib._bootstrap>\", line 994, in >>>>>>>> _gcd_import\n File \"<frozen importlib._bootstrap>\", line 971, in >>>>>>>> _find_and_load\n File \"<frozen importlib._bootstrap>\", line 955, in >>>>>>>> _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line >>>>>>>> 665, in _load_unlocked\n File \"<frozen >>>>>>>> importlib._bootstrap_external>\", >>>>>>>> line 678, in exec_module\n File \"<frozen importlib._bootstrap>\", >>>>>>>> line >>>>>>>> 219, in _call_with_frames_removed\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\", >>>>>>>> line 83, in <module>\n plugin_app_config = entry_point.load()\n >>>>>>>> File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\", >>>>>>>> line 2317, in load\n self.require(*args, **kwargs)\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\", >>>>>>>> line 2340, in require\n items = working_set.resolve(reqs, env, >>>>>>>> installer, extras=self.extras)\n File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\", >>>>>>>> line 779, in resolve\n raise VersionConflict(dist, >>>>>>>> req).with_context(dependent_req)\npkg_resources.VersionConflict: >>>>>>>> (pulpcore >>>>>>>> 3.12.0 (/usr/local/lib/pulp/lib/python3.6/site-packages), >>>>>>>> Requirement.parse('pulpcore<3.6,>=3.4'))", "stderr_lines": ["Traceback >>>>>>>> (most recent call last):", " File >>>>>>>> \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>", " >>>>>>>> sys.exit(manage())", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/manage.py\", >>>>>>>> line 11, in manage", " execute_from_command_line(sys.argv)", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>>>> line 381, in execute_from_command_line", " utility.execute()", " >>>>>>>> File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/core/management/__init__.py\", >>>>>>>> line 325, in execute", " settings.INSTALLED_APPS", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\", >>>>>>>> line 79, in __getattr__", " self._setup(name)", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\", >>>>>>>> line 66, in _setup", " self._wrapped = Settings(settings_module)", " >>>>>>>> File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/django/conf/__init__.py\", >>>>>>>> line 157, in __init__", " mod = >>>>>>>> importlib.import_module(self.SETTINGS_MODULE)", "File >>>>>>>> \"/usr/lib64/python3.6/importlib/__init__.py\", line 126, in >>>>>>>> import_module", " return _bootstrap._gcd_import(name[level:], >>>>>>>> package, >>>>>>>> level)", " File \"<frozen importlib._bootstrap>\", line 994, in >>>>>>>> _gcd_import", " File \"<frozen importlib._bootstrap>\", line 971, in >>>>>>>> _find_and_load", " File \"<frozen importlib._bootstrap>\", line 955, >>>>>>>> in >>>>>>>> _find_and_load_unlocked", " File \"<frozen importlib._bootstrap>\", >>>>>>>> line >>>>>>>> 665, in _load_unlocked", "File \"<frozen >>>>>>>> importlib._bootstrap_external>\", >>>>>>>> line 678, in exec_module", " File \"<frozen importlib._bootstrap>\", >>>>>>>> line >>>>>>>> 219, in _call_with_frames_removed", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/settings.py\", >>>>>>>> line 83, in <module>", " plugin_app_config = entry_point.load()", " >>>>>>>> File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\", >>>>>>>> line 2317, in load", " self.require(*args, **kwargs)", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\", >>>>>>>> line 2340, in require", " items = working_set.resolve(reqs, env, >>>>>>>> installer, extras=self.extras)", " File >>>>>>>> \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pkg_resources/__init__.py\", >>>>>>>> line 779, in resolve", " raise VersionConflict(dist, >>>>>>>> req).with_context(dependent_req)", "pkg_resources.VersionConflict: >>>>>>>> (pulpcore 3.12.0 (/usr/local/lib/pulp/lib/python3.6/site-packages), >>>>>>>> Requirement.parse('pulpcore<3.6,>=3.4'))"], "stdout": "", >>>>>>>> "stdout_lines": >>>>>>>> []} >>>>>>>> >>>>>>>> RUNNING HANDLER [pulp.pulp_installer.pulp_common : Restart all Pulp >>>>>>>> services] >>>>>>>> *************************************************************************************************************************************************** >>>>>>>> >>>>>>>> RUNNING HANDLER [geerlingguy.postgresql : restart postgresql] >>>>>>>> ******************************************************************************************************************************************************************* >>>>>>>> >>>>>>>> NO MORE HOSTS LEFT >>>>>>>> ************************************************************************************************************************************************************************************************************** >>>>>>>> >>>>>>>> PLAY RECAP >>>>>>>> ********************************************************************************************************************************************************************************************************************** >>>>>>>> localhost : ok=91 changed=33 unreachable=0 >>>>>>>> failed=1 skipped=46 rescued=0 ignored=0 >>>>>>>> >>>>>>>> Any help is greatly appreciated. >>>>>>>> >>>>>>>> Kind regards, >>>>>>>> >>>>>>>> *Eric Van Steenbergen* >>>>>>>> >>>>>>>> *E-mail: vs.e...@gmail.com <vs.e...@gmail.com>* >>>>>>>> *Skype: ericvs2014* >>>>>>>> *LinkedIn: Eric Van Steenbergen >>>>>>>> <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>* >>>>>>>> _______________________________________________ >>>>>>>> Pulp-list mailing list >>>>>>>> Pulp-list@redhat.com >>>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-list >>>>>>> >>>>>>> _______________________________________________ >> Pulp-list mailing list >> Pulp-list@redhat.com >> https://listman.redhat.com/mailman/listinfo/pulp-list > > -- *Eric Van Steenbergen* *E-mail: vs.e...@gmail.com <vs.e...@gmail.com>* *Skype: ericvs2014* *LinkedIn: Eric Van Steenbergen <http://es.linkedin.com/pub/eric-van-steenbergen/b/8a4/51b>*
_______________________________________________ Pulp-list mailing list Pulp-list@redhat.com https://listman.redhat.com/mailman/listinfo/pulp-list