Hi Samiron, The original stuck task 49b83f70-e6d6-4cdb-9c5a-93c20c31d697 was assigned to reserved_resource_worker-1@mysrv so it makes sense that when you restarted reserved_resource_worker-1 any tasks assigned to it are automatically canceled. Workers clear out assigned-but-unfinished tasks as canceled on startup. Assigned but unfinished tasks can be either waiting or running.
Perhaps you are affected by issue #1801 [0]? Can you verify that you are running the newest python-kombu suggested here[1]? Can you show your version of python-kombu: `rpm -qa | grep kombu`? Also can you show the task details (with -vv) of your second. If you are affected by #1801 it should not have been assigned to any worker. [0]: https://pulp.plan.io/issues/1801 [1]: https://www.redhat.com/archives/pulp-list/2016-April/msg00020.html -Brian On 05/04/2016 12:07 AM, Mallick, Samiron wrote: > Hey Brian, thanks for the reply. > > *From the below output I could see "resource_worker-1" is responsible > for this task and I have 4 worker displayed on the server.____* > > __ __ > > # pulp-admin tasks list____ > > +----------------------------------------------------------------------+____ > > Tasks____ > > +----------------------------------------------------------------------+____ > > __ __ > > Operations: sync____ > > Resources: rhel-6-server-supplementary-rpms (repository)____ > > State: Waiting____ > > Start Time: Unstarted____ > > Finish Time: Incomplete____ > > Task Id: 49b83f70-e6d6-4cdb-9c5a-93c20c31d697____ > > __ __ > > __ __ > > # pulp-admin -vv tasks details --task-id > 49b83f70-e6d6-4cdb-9c5a-93c20c31d697____ > > +----------------------------------------------------------------------+____ > > Task Details____ > > +----------------------------------------------------------------------+____ > > __ __ > > 2016-05-04 04:55:33,231 - DEBUG - sending GET request to > /pulp/api/v2/tasks/49b83f70-e6d6-4cdb-9c5a-93c20c31d697/____ > > 2016-05-04 04:55:33,362 - INFO - GET request to > /pulp/api/v2/tasks/49b83f70-e6d6-4cdb-9c5a-93c20c31d697/ with parameters > None____ > > 2016-05-04 04:55:33,362 - INFO - Response status : 200____ > > __ __ > > 2016-05-04 04:55:33,363 - INFO - Response body :____ > > {____ > > "exception": null,____ > > "task_type": "pulp.server.managers.repo.sync.sync",____ > > "_href": "/pulp/api/v2/tasks/49b83f70-e6d6-4cdb-9c5a-93c20c31d697/",____ > > "task_id": "49b83f70-e6d6-4cdb-9c5a-93c20c31d697",____ > > "tags": [____ > > "pulp:repository:rhel-6-server-supplementary-rpms",____ > > "pulp:action:sync"____ > > ],____ > > "finish_time": null,____ > > "_ns": "task_status",____ > > "start_time": null,____ > > "traceback": null,____ > > "spawned_tasks": [],____ > > "progress_report": {},____ > > "queue": "[email protected]",____ > > "state": "waiting",____ > > "worker_name": "reserved_resource_worker-1@mysrv",____ > > "result": null,____ > > "error": null,____ > > "_id": {____ > > "$oid": "572964399b70a2ea1d2694aa"____ > > },____ > > "id": "572964399b70a2ea1d2694aa"____ > > }____ > > __ __ > > Operations: sync____ > > Resources: rhel-6-server-supplementary-rpms (repository)____ > > State: Waiting____ > > Start Time: Unstarted____ > > Finish Time: Incomplete____ > > Result: Incomplete____ > > Task Id: 49b83f70-e6d6-4cdb-9c5a-93c20c31d697____ > > Progress Report:____ > > __ __ > > __ __ > > # pulp-admin status____ > > +----------------------------------------------------------------------+____ > > Status of the server____ > > +----------------------------------------------------------------------+____ > > __ __ > > Api Version: 2____ > > Database Connection:____ > > Connected: True____ > > Known Workers:____ > > _id: scheduler@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:53:34Z____ > > _id: reserved_resource_worker-3@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:00Z____ > > _id: reserved_resource_worker-2@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:00Z____ > > _id: resource_manager@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:00Z____ > > _id: reserved_resource_worker-1@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:01Z____ > > _id: reserved_resource_worker-0@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:03Z____ > > Messaging Connection:____ > > Connected: True____ > > Versions:____ > > Platform Version: 2.8.2____ > > __ __ > > # ps -awfux | grep celery____ > > root 4637 0.0 0.0 112644 960 pts/0 S+ 04:56 > 0:00 \_ grep --color=auto celery____ > > apache 1592 0.0 1.4 667716 56368 ? Ssl May03 0:26 > /usr/bin/python /usr/bin/celery worker -A pulp.server.async.app -n > resource_manager@%h -Q resource_manager -c 1 --events --umask 18 > --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30____ > > apache 2921 0.0 1.4 667664 54296 ? Sl May03 0:13 \_ > /usr/bin/python /usr/bin/celery worker -A pulp.server.async.app -n > resource_manager@%h -Q resource_manager -c 1 --events --umask 18 > --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30____ > > apache 1616 0.0 1.4 667996 56400 ? Ssl May03 0:27 > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-0@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-0.pid > --heartbeat-interval=30____ > > apache 2919 0.0 1.4 741536 54564 ? Sl May03 0:11 \_ > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-0@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-0.pid > --heartbeat-interval=30____ > > apache 1626 0.0 1.5 668560 59524 ? Ssl May03 0:29 > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-1@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-1.pid > --heartbeat-interval=30____ > > apache 4561 0.0 1.4 668560 56260 ? S 04:47 0:00 \_ > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-1@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-1.pid > --heartbeat-interval=30____ > > apache 1631 0.0 1.5 667748 58508 ? Ssl May03 0:27 > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-2@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-2.pid > --heartbeat-interval=30____ > > apache 2922 4.2 8.0 1042956 311476 ? Sl May03 48:25 \_ > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-2@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-2.pid > --heartbeat-interval=30____ > > apache 1637 0.0 1.4 667744 56368 ? Ssl May03 0:27 > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-3@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-3.pid > --heartbeat-interval=30____ > > apache 2920 0.0 1.4 815420 54760 ? Sl May03 0:13 \_ > /usr/bin/python /usr/bin/celery worker -n reserved_resource_worker-3@%h > -A pulp.server.async.app -c 1 --events --umask 18 > --pidfile=/var/run/pulp/reserved_resource_worker-3.pid > --heartbeat-interval=30____ > > apache 4620 6.5 0.8 663652 31432 ? Ssl 04:56 0:00 > /usr/bin/python /usr/bin/celery beat > --app=pulp.server.async.celery_instance.celery > --scheduler=pulp.server.async.scheduler.Scheduler____ > > __ __ > > __ __ > > *As I saw errors in output of pulp_worker-1, I restarted each worker > individually, and all error seems gone.____* > > __ __ > > # systemctl status pulp_workers.service____ > > ● pulp_workers.service - Pulp Celery Workers____ > > Loaded: loaded (/usr/lib/systemd/system/pulp_workers.service; > enabled; vendor preset: disabled)____ > > Active: active (exited) since Wed 2016-05-04 05:36:38 CEST; 3s ago____ > > Process: 5717 ExecStop=/usr/bin/python -m > pulp.server.async.manage_workers stop (code=exited, status=0/SUCCESS)____ > > Process: 5731 ExecStart=/usr/bin/python -m > pulp.server.async.manage_workers start (code=exited, status=0/SUCCESS)____ > > Main PID: 5731 (code=exited, status=0/SUCCESS)____ > > __ __ > > May 04 05:36:38 mysrv systemd[1]: Starting Pulp Celery Workers...____ > > May 04 05:36:38 mysrv systemd[1]: Started Pulp Celery Workers.____ > > __ __ > > # systemctl status pulp_worker-0____ > > ? pulp_worker-0.service - Pulp Worker #0____ > > Loaded: loaded (/run/systemd/system/pulp_worker-0.service; static; > vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:10:44 CEST; 1min 26s > ago____ > > Main PID: 4753 (celery)____ > > CGroup: /system.slice/pulp_worker-0.service____ > > +-4753 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-0@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4766 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-0@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:10:46 mysrv celery[4753]: - ** ---------- .> transport: > qpid://mysrv:5672// <http://srpadinf0244.mgt.insim.biz:5672//>____ > > May 04 05:10:46 mysrv celery[4753]: - ** ---------- .> results: > disabled____ > > May 04 05:10:46 mysrv celery[4753]: - *** --- * --- .> concurrency: 1 > (prefork)____ > > May 04 05:10:46 mysrv celery[4753]: -- ******* ----____ > > May 04 05:10:46 mysrv celery[4753]: --- ***** ----- [queues]____ > > May 04 05:10:46 mysrv celery[4753]: -------------- .> celery > exchange=celery(direct) key=celery____ > > May 04 05:10:46 mysrv celery[4753]: .> > [email protected] exchange=C.dq(direct) key=rese...s > <http://sim.biz/>rv____ > > May 04 05:10:46 mysrv pulp[4753]: kombu.transport.qpid:INFO: Connected > to qpid with SASL mechanism ANONYMOUS____ > > May 04 05:10:46 mysrv pulp[4753]: celery.worker.consumer:INFO: Connected > to qpid://mysrv:5672// <http://srpadinf0244.mgt.insim.biz:5672//>____ > > May 04 05:10:46 mysrv pulp[4753]: kombu.transport.qpid:INFO: Connected > to qpid with SASL mechanism ANONYMOUS____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > __ __ > > # systemctl status pulp_worker-1____ > > ? pulp_worker-1.service - Pulp Worker #1____ > > Loaded: loaded (/run/systemd/system/pulp_worker-1.service; static; > vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:08:16 CEST; 3min 57s > ago____ > > Main PID: 4718 (celery)____ > > CGroup: /system.slice/pulp_worker-1.service____ > > +-4718 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-1@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4733 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-1@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...3cc3c36]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...ce7430b]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.job:INFO: Task > pulp.server.controllers.repository.download_deferred[aad88f32-...9s: > None____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.db.reaper.reap_expired_documents[02...8322faa]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...ddadf87]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...d0cf8c6]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...72edf98]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...1e9e4bc]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.download_def...30f8627]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: Received > task: pulp.server.controllers.repository.queue_downlo...fd23e13]____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > __ __ > > # systemctl status pulp_worker-2____ > > ? pulp_worker-2.service - Pulp Worker #2____ > > Loaded: loaded (/run/systemd/system/pulp_worker-2.service; static; > vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:11:06 CEST; 1min 10s > ago____ > > Main PID: 4776 (celery)____ > > CGroup: /system.slice/pulp_worker-2.service____ > > +-4776 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-2@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4789 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-2@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:11:07 mysrv celery[4776]: - ** ---------- .> transport: > qpid://mysrv:5672// <http://srpadinf0244.mgt.insim.biz:5672//>____ > > May 04 05:11:07 mysrv celery[4776]: - ** ---------- .> results: > disabled____ > > May 04 05:11:07 mysrv celery[4776]: - *** --- * --- .> concurrency: 1 > (prefork)____ > > May 04 05:11:07 mysrv celery[4776]: -- ******* ----____ > > May 04 05:11:07 mysrv celery[4776]: --- ***** ----- [queues]____ > > May 04 05:11:07 mysrv celery[4776]: -------------- .> celery > exchange=celery(direct) key=celery____ > > May 04 05:11:07 mysrv celery[4776]: .> > [email protected] > exchange=C.dq(direct) key=rese...s <http://sim.biz/>rv____ > > May 04 05:11:07 mysrv pulp[4776]: kombu.transport.qpid:INFO: Connected > to qpid with SASL mechanism ANONYMOUS____ > > May 04 05:11:07 mysrv pulp[4776]: celery.worker.consumer:INFO: Connected > to qpid://mysrv:5672// <http://srpadinf0244.mgt.insim.biz:5672//>____ > > May 04 05:11:07 mysrv pulp[4776]: kombu.transport.qpid:INFO: Connected > to qpid with SASL mechanism ANONYMOUS____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > __ __ > > # systemctl status pulp_worker-3____ > > ? pulp_worker-3.service - Pulp Worker #3____ > > Loaded: loaded (/run/systemd/system/pulp_worker-3.service; static; > vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:11:21 CEST; 59s ago____ > > Main PID: 4798 (celery)____ > > CGroup: /system.slice/pulp_worker-3.service____ > > +-4798 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-3@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4811 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-3@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:11:22 mysrv celery[4798]: - ** ---------- .> transport: > qpid://mysrv:5672// <http://srpadinf0244.mgt.insim.biz:5672//>____ > > May 04 05:11:22 mysrv celery[4798]: - ** ---------- .> results: > disabled____ > > May 04 05:11:22 mysrv celery[4798]: - *** --- * --- .> concurrency: 1 > (prefork)____ > > May 04 05:11:22 mysrv celery[4798]: -- ******* ----____ > > May 04 05:11:22 mysrv celery[4798]: --- ***** ----- [queues]____ > > May 04 05:11:22 mysrv celery[4798]: -------------- .> celery > exchange=celery(direct) key=celery____ > > May 04 05:11:22 mysrv celery[4798]: .> > [email protected] > exchange=C.dq(direct) key=rese...s <http://sim.biz/>rv____ > > May 04 05:11:22 mysrv pulp[4798]: kombu.transport.qpid:INFO: Connected > to qpid with SASL mechanism ANONYMOUS____ > > May 04 05:11:22 mysrv pulp[4798]: celery.worker.consumer:INFO: Connected > to qpid://mysrv:5672// <http://srpadinf0244.mgt.insim.biz:5672//>____ > > May 04 05:11:22 mysrv pulp[4798]: kombu.transport.qpid:INFO: Connected > to qpid with SASL mechanism ANONYMOUS____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > *Now I have no tasks queued at all. I ran the repo sync again. and again > it went to waiting.....____* > > __ __ > > # pulp-admin tasks list____ > > +----------------------------------------------------------------------+____ > > Tasks____ > > +----------------------------------------------------------------------+____ > > __ __ > > No tasks found____ > > __ __ > > # pulp-admin rpm repo sync run > --repo-id=rhel-6-server-supplementary-rpms____ > > +----------------------------------------------------------------------+____ > > Synchronizing Repository [rhel-6-server-supplementary-rpms]____ > > +----------------------------------------------------------------------+____ > > __ __ > > This command may be exited via ctrl+c without affecting the request.____ > > __ __ > > __ __ > > [/]____ > > Waiting to begin... > > > > *//* > > > On Wed, May 4, 2016 at 1:52 AM, Brian Bouterse <[email protected] > <mailto:[email protected]>> wrote: > > Kodiak is right that the second task stuck at "Waiting to Begin" is > likely waiting behind another operation on that same repo. Canceling the > one prior will likely allow the later one to start. > > How many workers are running and how many do you expect? You can see > what Pulp thinks with: `pulp-admin status` > > You can compare that to your pulp processes on all of your Pulp servers > with `sudo ps -awfux | grep celery`. > > Also you can look at the task details with -vv to see the worker the > halted task is assigned to. Something like `pulp-admin -vv tasks details > --task-id 03842c9d-e053-4a6f-a4c4-2d7302be9c8c.` > > Unfortunately you'll have to see the worker in the raw response with -vv > because of [0]. > > [0]: https://pulp.plan.io/issues/1832 > > -Brian > > > On 05/03/2016 11:53 AM, Kodiak Firesmith wrote: > > I believe you may need to cancel the pending repo sync task before you > > can delete the repo. Maybe try: > > pulp-admin tasks cancel --task-id=2d776d63-fd8a-4e0a-8f32-d2276c85187c > > pulp-admin tasks cancel --task-id=03842c9d-e053-4a6f-a4c4-2d7302be9c8c > > > > Then: > > pulp-admin rpm repo delete --repo-id=rhel-6-server-supplementary-rpms > > > > > > On Tue, May 3, 2016 at 11:47 AM, Mallick, Samiron > > <[email protected] <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>> wrote: > > > > Could anyone please tell me what went wrong with the > repository. One > > of my EL7 server registered and was able to fetch contents > from CDN. > > Recently I found one of the repo stuck after downloading RPMs. > It’s > > never ending. I rebooted my server, cancelled tasks, deleted the > > repo and recreated, but no luck. No if I run sync, it directly > going > > to waiting stage. Earlier I observed it was starting the task but > > the start time was same as I ran the job first time. Even now I am > > not able to delete the repo as well as it is showing “Waiting to > > begin”. I am running Pulp v2.8. Any idea would be greatly > appreciated. > > > > > > > > # rpm -qa pulp-server > > > > pulp-server-2.8.2-1.el7.noarch > > > > > > > > # pulp-admin rpm repo sync run > > --repo-id=rhel-6-server-supplementary-rpms > > > > > +----------------------------------------------------------------------+ > > > > Synchronizing Repository [rhel-6-server-supplementary-rpms] > > > > > +----------------------------------------------------------------------+ > > > > > > > > This command may be exited via ctrl+c without affecting the > request. > > > > > > > > > > > > Downloading metadata... > > > > [\] > > > > ... completed > > > > > > > > Downloading repository content... > > > > [-] > > > > [==================================================] 100% > > > > RPMs: 0/0 items > > > > Delta RPMs: 0/0 items > > > > > > > > ... completed > > > > > > > > Downloading distribution files... > > > > [==================================================] 100% > > > > Distributions: 0/0 items > > > > ... completed > > > > > > > > Importing errata... > > > > [/] > > > > ... completed > > > > > > > > Importing package groups/categories... > > > > [-] > > > > ... completed > > > > > > > > Cleaning duplicate packages... > > > > [|] > > > > ... completed > > > > > > > > *** AND STUCK HERE *** > > > > > > > > # pulp-admin tasks list > > > > > +----------------------------------------------------------------------+ > > > > Tasks > > > > > +----------------------------------------------------------------------+ > > > > > > > > Operations: sync > > > > Resources: rhel-6-server-supplementary-rpms (repository) > > > > State: Running > > > > Start Time: 2016-05-03T07:06:36Z > > > > Finish Time: Incomplete > > > > Task Id: 2d776d63-fd8a-4e0a-8f32-d2276c85187c > > > > > > > > Operations: publish > > > > Resources: rhel-6-server-supplementary-rpms (repository) > > > > State: Waiting > > > > Start Time: Unstarted > > > > Finish Time: Incomplete > > > > Task Id: 03842c9d-e053-4a6f-a4c4-2d7302be9c8c > > > > > > > > # date > > > > Tue May 3 09:22:30 CEST 2016 > > > > # pulp-admin rpm repo sync schedules list > > --repo-id=rhel-6-server-supplementary-rpms > > > > > +----------------------------------------------------------------------+ > > > > Schedules > > > > > +----------------------------------------------------------------------+ > > > > > > > > There are no schedules defined for this operation. > > > > > > > > # pulp-admin rpm repo delete > --repo-id=rhel-6-server-supplementary-rpms > > > > This command may be exited via ctrl+c without affecting the > request. > > > > > > > > > > > > [-] > > > > Running... > > > > [-] > > > > Waiting to begin... > > > > > > > > *** AND STUCK HERE *** > > > > > > _______________________________________________ > > Pulp-list mailing list > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > > https://www.redhat.com/mailman/listinfo/pulp-list > > > > > > > > > > _______________________________________________ > > Pulp-list mailing list > > [email protected] <mailto:[email protected]> > > https://www.redhat.com/mailman/listinfo/pulp-list > > > > _______________________________________________ > Pulp-list mailing list > [email protected] <mailto:[email protected]> > https://www.redhat.com/mailman/listinfo/pulp-list > > _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
