On 08/01/2016 09:39 AM, Rick Langston wrote:
> The backend service all say ok but when a run a katelli-service status I
> can see that celery.bet fails to status. if I restart the server it and
> immediately check the status it says running but checking the status again
> shows it timed out.
>
Is this the sequence of events?
* service pulp_celerybeat start (outputs success)
* service pulp_celerybeat status (outputs success)
* wait some number of seconds
* service pulp_celerybeat status (outputs error)
>
> No memory errros noted
>
>
> [root@dscaprv01 tmp]# systemctl status pulp_celerybeat.service
> ● pulp_celerybeat.service - Pulp's Celerybeat
> Loaded: loaded (/usr/lib/systemd/system/pulp_celerybeat.service;
> enabled; vendor preset: disabled)
> Active: inactive (dead) since Mon 2016-08-01 08:34:31 CDT; 18s ago
> Process: 5887 ExecStart=/usr/bin/celery beat
> --app=pulp.server.async.celery_instance.celery
> --scheduler=pulp.server.async.scheduler.Scheduler (code=exited,
> status=0/SUCCESS)
> Main PID: 5887 (code=exited, status=0/SUCCESS)
>
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net pulp[5887]: celery.beat:CRITICAL:
> (5887-79264) raise Timeout("Connection attach timed out")
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net pulp[5887]: celery.beat:CRITICAL:
> (5887-79264) Timeout: Connection attach timed out
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: celery beat v3.1.11
> (Cipater) is starting.
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: __ - ... __
> - _
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: Configuration ->
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . broker ->
> qpid://dscaprv01.corp.acxiom.net:5671//
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . loader ->
> celery.loaders.app.AppLoader
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . scheduler ->
> pulp.server.async.scheduler.Scheduler
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . logfile ->
> [stderr]@%INFO
> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . maxinterval ->
> now (0s)
>
>
> On Monday, August 1, 2016 at 8:22:46 AM UTC-5, Chris Duryee wrote:
>>
>>
>>
>> On 08/01/2016 08:50 AM, Rick Langston wrote:
>>> I do see this issue in messages but not sure if its related
>>>
>>> Aug 1 07:25:16 dscaprv01 pulp: celery.beat:CRITICAL: (28691-76416) beat
>>> raised exception <class 'qpid.messaging.exceptions.Timeout'>:
>>> Timeout('Connection attach timed out',)
>>> Aug 1 07:25:16 dscaprv01 pulp: celery.beat:CRITICAL: (28691-76416)
>>> Traceback (most recent call last):
>>> Aug 1 07:25:16 dscaprv01 pulp: celery.beat:CRITICAL: (28691-76416)
>> File
>>> "/usr/lib/python2.7/site-packages/celery/apps/beat.py", line 112, in
>>> start_scheduler
>>> Aug 1 07:25:16 dscaprv01 pulp: celery.beat:CRITICAL: (28691-76416)
>>> beat.start()
>>>
>>
>>
>> That is the likely culprit:)
>>
>> Next time your task hangs, check in the "/about" page on your Katello
>> instance and ensure everything under "Backend System Status" says "OK"
>> with no further message.
>>
>> If there are pulp errors, a possible quick fix is to ensure qpidd is
>> still running, then restart pulp_workers, pulp_celerybeat and
>> pulp_resource_manager. I suspect your task will get picked up after that.
>>
>> Also, please check dmesg for out-of-memory errors. There are some other
>> possible things we can check, but I would be curious first about the
>> backend system status output.
>>
>>>
>>> On Monday, August 1, 2016 at 7:12:26 AM UTC-5, Chris Duryee wrote:
>>>>
>>>>
>>>>
>>>> On 08/01/2016 07:54 AM, Rick Langston wrote:
>>>>> Hello
>>>>>
>>>>> I seem to be having some odd behavior with this version. With a fresh
>>>>> install on centos 7 I have setup a product which completes normally
>> but
>>>>> when I discover a repo and save them i get these meta data task that
>>>> seem
>>>>> to just wait forever. Any ideas what can be the culprit
>>>>>
>>>>
>>>> Are there any related errors in /var/log/messages?
>>>>
>>>>>
>>>>> Action:
>>>>>
>>>>> Actions::Pulp::Repository::DistributorPublish
>>>>>
>>>>> State: waiting for Pulp to start the task
>>>>> Input:
>>>>>
>>>>> {"pulp_id"=>"test-centos-6_updates_x86_64",
>>>>> "distributor_type_id"=>"yum_distributor",
>>>>> "source_pulp_id"=>nil,
>>>>> "dependency"=>nil,
>>>>> "remote_user"=>"admin",
>>>>> "remote_cp_user"=>"admin",
>>>>> "locale"=>"en"}
>>>>>
>>>>> Output:
>>>>>
>>>>> {"pulp_tasks"=>
>>>>> [{"exception"=>nil,
>>>>> "task_type"=>"pulp.server.managers.repo.publish.publish",
>>>>>
>> "_href"=>"/pulp/api/v2/tasks/a40815d5-9ba4-463a-8216-338cdcc4b1cc/",
>>>>> "task_id"=>"a40815d5-9ba4-463a-8216-338cdcc4b1cc",
>>>>> "tags"=>
>>>>> ["pulp:repository:test-centos-6_updates_x86_64",
>>>> "pulp:action:publish"],
>>>>> "finish_time"=>nil,
>>>>> "_ns"=>"task_status",
>>>>> "start_time"=>nil,
>>>>> "traceback"=>nil,
>>>>> "spawned_tasks"=>[],
>>>>> "progress_report"=>{},
>>>>> "queue"=>"None.dq",
>>>>> "state"=>"waiting",
>>>>> "worker_name"=>nil,
>>>>> "result"=>nil,
>>>>> "error"=>nil,
>>>>> "_id"=>{"$oid"=>"579f32aa95c48c6a54257674"},
>>>>> "id"=>"579f32aa95c48c6a54257674"}],
>>>>> "poll_attempts"=>{"total"=>100, "failed"=>0}}
>>>>>
>>>>
>>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
