On 08/09/2016 07:44 AM, Rick Langston wrote: > I have been looking at a katello bug 13799 which may be related to a pulp > 1363 that may cause this issues as it relates to qpid > > Thoughts?
They do look related, but I'm not really an expert in this area so it's hard to say. The place with the most experts on celery and pulp is [email protected]. Someone on there may have ideas as to why the issue happens on physical hardware but not a VM. It sounds like it's easy to reproduce the issue on your setup which is usually the hard part :) > > On Monday, August 8, 2016 at 7:12:55 AM UTC-5, Rick Langston wrote: >> >> Also note as the service tries to start at approx 10s it fails to connect >> but with enough tries it will connect and stay connected until reboot or >> bounce of service >> >> >> Aug 08 07:04:33 servername.corp.company.net systemd[1]: Started Pulp's >> Celerybeat. >> Aug 08 07:04:33 servername.corp.company.net systemd[1]: Starting Pulp's >> Celerybeat... >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: celery.beat:INFO: >> beat: Starting... >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: >> pulp.server.db.connection:INFO: Attempting to connect to localhost:27017 >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: >> pulp.server.async.scheduler:INFO: Worker Timeout Monitor Started >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: >> pulp.server.db.connection:INFO: Attempting to connect to localhost:27017 >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: >> kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: >> pulp.server.async.scheduler:INFO: Event Monitor Starting >> Aug 08 07:04:39 servername.corp.company.net pulp[6381]: >> pulp.server.db.connection:INFO: Write concern for Mongo connection: {} >> [root@servername ~]# /bin/systemctl status pulp_celerybeat.service >> ● pulp_celerybeat.service - Pulp's Celerybeat >> Loaded: loaded (/usr/lib/systemd/system/pulp_celerybeat.service; >> enabled; vendor preset: disabled) >> Active: active (running) since Mon 2016-08-08 07:04:33 CDT; 10s ago >> Main PID: 6381 (celery) >> CGroup: /system.slice/pulp_celerybeat.service >> └─6381 /usr/bin/python /usr/bin/celery beat >> --app=pulp.server.async.celery_instance.celery >> --scheduler=pulp.server.async.scheduler.Scheduler >> >> Aug 08 07:04:44 servername.corp.company.net pulp[6381]: >> celery.beat:CRITICAL: >> (6381-73184) raise Timeout("Connection attach timed out") >> Aug 08 07:04:44 servername.corp.company.net pulp[6381]: >> celery.beat:CRITICAL: (6381-73184) Timeout: Connection attach timed out >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: celery beat >> v3.1.11 (Cipater) is starting. >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: __ - ... >> __ - _ >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: Configuration -> >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: . broker -> >> qpid://servername.corp.company.net:5671// >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: . loader -> >> celery.loaders.app.AppLoader >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: . scheduler -> >> pulp.server.async.scheduler.Scheduler >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: . logfile -> >> [stderr]@%INFO >> Aug 08 07:04:44 servername.corp.company.net celery[6381]: . maxinterval >> -> now (0s) >> >> >> >> >> >> >> >> >> >> On Friday, August 5, 2016 at 1:41:51 PM UTC-5, Rick Langston wrote: >>> >>> Yes the VMs are running 1CPU 1 CORE with 4G memory and they work. the >>> server in question is running 2 CPUs with 8 cores each with 32G of ram >>> >>> On Friday, August 5, 2016 at 10:09:52 AM UTC-5, Chris Duryee wrote: >>>> >>>> >>>> >>>> On 08/05/2016 06:48 AM, Rick Langston wrote: >>>>> Well the rebuild came back with the same issue. This issue seems to >>>> be >>>>> tied to this hardware somehow as I can build using the same process in >>>> a VM >>>>> and everything is as it should be. Diags have been ran on the >>>> hardware >>>>> with no issues found. we also removed the external storage and mounted >>>>> everything local with no change. >>>>> >>>>> Are there any tunes that we need to be aware of in regards to >>>>> CPU,disk,memory or open files sockets etc? >>>> >>>> Does the hardware have the same number of CPUs and same amount of memory >>>> as the VM? Pulp spins up workers based on the number of available CPUs, >>>> so there may be some difference there. >>>> >>>>> >>>>> Thank you >>>>> >>>>> >>>>> On Thursday, August 4, 2016 at 12:03:13 PM UTC-5, Chris Duryee wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 08/04/2016 07:38 AM, Rick Langston wrote: >>>>>>> Well this is strange. it looks as if the process has stabilized. I >>>>>> didn't >>>>>>> see anything in the capture that jumped out at me but will try and >>>>>>> normalize the build today and see if it continues to work. >>>>>>> >>>>>> >>>>>> Well, I am glad it works now at least:) If it becomes unstable again >>>> let >>>>>> us know and we can look more. >>>>>> >>>>>>> Thank you for your help with this >>>>>> >>>>>> No problem, thanks for debugging with us. >>>>>> >>>>>>> >>>>>>> On Wednesday, August 3, 2016 at 10:41:49 AM UTC-5, Chris Duryee >>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 08/03/2016 07:41 AM, Rick Langston wrote: >>>>>>>>> Things seem to stem from pulp and a connection >>>>>>>> >>>>>>>> cool, this narrows it down some:) >>>>>>>> >>>>>>>> Let's try this to see if it captures the issue: >>>>>>>> >>>>>>>> # stop everything >>>>>>>> katello-service stop >>>>>>>> >>>>>>>> # start mongodb >>>>>>>> systemctl start mongod >>>>>>>> >>>>>>>> # become qpidd user >>>>>>>> sudo su - qpidd -s /bin/bash (to become qpid user) >>>>>>>> >>>>>>>> #start qpid with trace output >>>>>>>> /usr/sbin/qpidd --config /etc/qpid/qpidd.conf -t 2>&1 | tee >>>>>> /tmp/qpid.out >>>>>>>> >>>>>>>> in another window: >>>>>>>> >>>>>>>> systemctl start pulp_celerybeat >>>>>>>> systemctl start pulp_resource_manager >>>>>>>> systemctl start pulp_workers >>>>>>>> >>>>>>>> <wait a minute or so> >>>>>>>> >>>>>>>> then stop the three pulp services, ^C to exit qpidd, and do a >>>>>>>> "katello-service start" to spin everything up again. >>>>>>>> >>>>>>>> Hopefully this will reproduce the issue, and we'll have the full >>>> qpid >>>>>>>> log in /tmp/qpid.out to see what happened. The log is at trace >>>> setting >>>>>>>> and will be extremely verbose. >>>>>>>> >>>>>>>>> >>>>>>>>> [root@servername ~]# /bin/systemctl status >>>> pulp_celerybeat.service >>>>>>>>> ● pulp_celerybeat.service - Pulp's Celerybeat >>>>>>>>> Loaded: loaded >>>> (/usr/lib/systemd/system/pulp_celerybeat.service; >>>>>>>>> enabled; vendor preset: disabled) >>>>>>>>> Active: inactive (dead) since Wed 2016-08-03 06:32:14 CDT; 1s >>>> ago >>>>>>>>> Process: 5388 ExecStart=/usr/bin/celery beat >>>>>>>>> --app=pulp.server.async.celery_instance.celery >>>>>>>>> --scheduler=pulp.server.async.scheduler.Scheduler (code=exited, >>>>>>>>> status=0/SUCCESS) >>>>>>>>> Main PID: 5388 (code=exited, status=0/SUCCESS) >>>>>>>>> >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net pulp[5388]: >>>>>>>>> celery.beat:CRITICAL: (5388-64256) Timeout: Connection attach >>>> timed >>>>>> out >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: celery >>>> beat >>>>>>>>> v3.1.11 (Cipater) is starting. >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: __ - >>>> >>>>>>>> ... __ >>>>>>>>> - _ >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: >>>>>> Configuration >>>>>>>> -> >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: . >>>> broker -> >>>>>>>>> qpid://servername.corp.company.net:5671// >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: . >>>> loader -> >>>>>>>>> celery.loaders.app.AppLoader >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: . >>>> scheduler >>>>>>>> -> >>>>>>>>> pulp.server.async.scheduler.Scheduler >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: . >>>> logfile >>>>>> -> >>>>>>>>> [stderr]@%INFO >>>>>>>>> Aug 03 06:32:13 servername.corp.company.net celery[5388]: . >>>>>> maxinterval >>>>>>>> -> >>>>>>>>> now (0s) >>>>>>>>> Aug 03 06:32:14 servername.corp.company.net pulp[5388]: >>>>>>>>> qpid.messaging:DEBUG: poll(1 fds, timeout=4000.0) >>>>>>>>> >>>>>>>>> On Tuesday, August 2, 2016 at 10:12:41 AM UTC-5, Chris Duryee >>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 08/02/2016 10:23 AM, Rick Langston wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tuesday, August 2, 2016 at 7:30:07 AM UTC-5, Chris Duryee >>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> and it will eventually die as it did last night. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Where would I send the debug file? >>>>>>>>>> >>>>>>>>>> I guess the easiest way is to just email me direct with the >>>>>> attachment, >>>>>>>>>> but we can keep the discussion on-list. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 08/01/2016 04:14 PM, Rick Langston wrote: >>>>>>>>>>>>> Strange, If I restart the service "/bin/systemctl status >>>>>>>>>>>>> pulp_celerybeat.service" it will eventually start. Any ideas? >>>>>>>>>>>> >>>>>>>>>>>> I'm glad it eventually restarts:) that is strange though that >>>> fails >>>>>>>> at >>>>>>>>>>>> first but eventually works. >>>>>>>>>>>> >>>>>>>>>>>> Can you send the output of foreman-debug? This will create an >>>> an >>>>>>>>>> archive >>>>>>>>>>>> with system information and log files that we can peruse to >>>> debug >>>>>>>>>> further. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Monday, August 1, 2016 at 12:19:27 PM UTC-5, Chris Duryee >>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 08/01/2016 10:52 AM, Chris Duryee wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 08/01/2016 09:49 AM, Rick Langston wrote: >>>>>>>>>>>>>>>> Thank you for help on this >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> yes here is the actual capture >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for the capture of the logs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What is the output of systemctl status qpidd.service? I am >>>>>> curious >>>>>>>>>> if >>>>>>>>>>>>>>> only celerybeat is unable to connect, or if there is some >>>> other >>>>>>>>>> issue. >>>>>>>>>>>>>> >>>>>>>>>>>>>> one more thing to check, as root please run: >>>>>>>>>>>>>> >>>>>>>>>>>>>> qpid-stat --ssl-certificate >>>>>> /etc/pki/katello/certs/java-client.crt >>>>>>>>>>>>>> --ssl-key /etc/pki/katello/private/java-client.key -b >>>>>>>>>>>>>> "amqps://$(hostname -f):5671" -q >>>>>>>>>>>>>> >>>>>>>>>>>>>> If qpid is accepting connections, it should output some stuff >>>>>> about >>>>>>>>>>>>>> queues. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you want to restart qpidd, the best way is to just do a >>>>>>>>>>>>>>> katello-service stop. A number of other services depend on >>>> qpid, >>>>>>>> so >>>>>>>>>>>> it's >>>>>>>>>>>>>>> easiest to just down everything, ensure qpidd is stopped, >>>> then >>>>>>>> bring >>>>>>>>>>>> it >>>>>>>>>>>>>>> all back up via the script. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> command line output >>>>>>>>>>>>>>>> [root@dscaprv01 tmp]# systemctl restart >>>> pulp_celerybeat.service >>>>>>>>>>>>>>>> [root@dscaprv01 tmp]# systemctl status >>>> pulp_celerybeat.service >>>>>>>>>>>>>>>> ● pulp_celerybeat.service - Pulp's Celerybeat >>>>>>>>>>>>>>>> Loaded: loaded >>>>>>>> (/usr/lib/systemd/system/pulp_celerybeat.service; >>>>>>>>>>>>>>>> enabled; vendor preset: disabled) >>>>>>>>>>>>>>>> Active: active (running) since Mon 2016-08-01 08:34:20 >>>> CDT; >>>>>> 7s >>>>>>>>>> ago >>>>>>>>>>>>>>>> Main PID: 5887 (celery) >>>>>>>>>>>>>>>> CGroup: /system.slice/pulp_celerybeat.service >>>>>>>>>>>>>>>> └─5887 /usr/bin/python /usr/bin/celery beat >>>>>>>>>>>>>>>> --app=pulp.server.async.celery_instance.celery >>>>>>>>>>>>>>>> --scheduler=pulp.server.async.scheduler.Scheduler >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Aug 01 08:34:20 dscaprv01.corp.acxiom.net systemd[1]: >>>> Started >>>>>>>>>> Pulp's >>>>>>>>>>>>>>>> Celerybeat. >>>>>>>>>>>>>>>> Aug 01 08:34:20 dscaprv01.corp.acxiom.net systemd[1]: >>>> Starting >>>>>>>>>>>> Pulp's >>>>>>>>>>>>>>>> Celerybeat... >>>>>>>>>>>>>>>> Aug 01 08:34:25 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>> celery.beat:INFO: >>>>>>>>>>>>>>>> beat: Starting... >>>>>>>>>>>>>>>> Aug 01 08:34:25 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>>>> pulp.server.db.connection:INFO: Attempting to connect to >>>>>>>>>>>>>> localhost:27017 >>>>>>>>>>>>>>>> Aug 01 08:34:25 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>>>> pulp.server.async.scheduler:INFO: Worker Timeout Monitor >>>>>> Started >>>>>>>>>>>>>>>> Aug 01 08:34:25 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>>>> pulp.server.db.connection:INFO: Attempting to connect to >>>>>>>>>>>>>> localhost:27017 >>>>>>>>>>>>>>>> Aug 01 08:34:26 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>>>> pulp.server.db.connection:INFO: Write concern for Mongo >>>>>>>> connection: >>>>>>>>>>>> {} >>>>>>>>>>>>>>>> [root@dscaprv01 tmp]# systemctl status >>>> pulp_celerybeat.service >>>>>>>>>>>>>>>> ● pulp_celerybeat.service - Pulp's Celerybeat >>>>>>>>>>>>>>>> Loaded: loaded >>>>>>>> (/usr/lib/systemd/system/pulp_celerybeat.service; >>>>>>>>>>>>>>>> enabled; vendor preset: disabled) >>>>>>>>>>>>>>>> Active: inactive (dead) since Mon 2016-08-01 08:34:31 >>>> CDT; >>>>>> 18s >>>>>>>>>> ago >>>>>>>>>>>>>>>> Process: 5887 ExecStart=/usr/bin/celery beat >>>>>>>>>>>>>>>> --app=pulp.server.async.celery_instance.celery >>>>>>>>>>>>>>>> --scheduler=pulp.server.async.scheduler.Scheduler >>>> (code=exited, >>>>>>>>>>>>>>>> status=0/SUCCESS) >>>>>>>>>>>>>>>> Main PID: 5887 (code=exited, status=0/SUCCESS) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>> celery.beat:CRITICAL: >>>>>>>>>>>>>>>> (5887-79264) raise Timeout("Connection attach timed >>>> out") >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net pulp[5887]: >>>>>>>>>>>>>> celery.beat:CRITICAL: >>>>>>>>>>>>>>>> (5887-79264) Timeout: Connection attach timed out >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: >>>> celery >>>>>>>>>> beat >>>>>>>>>>>>>> v3.1.11 >>>>>>>>>>>>>>>> (Cipater) is starting. >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: __ >>>> >>>>>> - >>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>>>> __ >>>>>>>>>>>>>>>> - _ >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: >>>>>>>>>>>> Configuration >>>>>>>>>>>>>> -> >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . >>>>>> broker >>>>>>>>>> -> >>>>>>>>>>>>>>>> qpid://dscaprv01.corp.acxiom.net:5671// >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . >>>>>> loader >>>>>>>>>> -> >>>>>>>>>>>>>>>> celery.loaders.app.AppLoader >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . >>>>>>>>>> scheduler >>>>>>>>>>>> -> >>>>>>>>>>>>>>>> pulp.server.async.scheduler.Scheduler >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . >>>>>>>> logfile >>>>>>>>>> -> >>>>>>>>>>>>>>>> [stderr]@%INFO >>>>>>>>>>>>>>>> Aug 01 08:34:30 dscaprv01.corp.acxiom.net celery[5887]: . >>>>>>>>>>>> maxinterval >>>>>>>>>>>>>> -> >>>>>>>>>>>>>>>> now (0s) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> /var/log/messages output >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ==> /var/log/messages <== >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>>>> beat >>>>>>>>>>>>>>>> raised exception <class >>>> 'qpid.messaging.exceptions.Timeout'>: >>>>>>>>>>>>>>>> Timeout('Connection attach timed out',) >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/usr/lib/python2.7/site-packages/celery/apps/beat.py", >>>> line >>>>>> 112, >>>>>>>>>> in >>>>>>>>>>>>>>>> start_scheduler >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>> beat.start() >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/usr/lib/python2.7/site-packages/celery/beat.py", line >>>> 462, in >>>>>>>>>> start >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>> interval = self.scheduler.tick() >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> >>>>>>>> "/usr/lib/python2.7/site-packages/pulp/server/async/scheduler.py", >>>>>>>>>>>> line >>>>>>>>>>>>>>>> 265, in tick >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> ret >>>>>>>>>>>>>>>> = self.call_tick(self, celerybeat_name) >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> >>>>>>>> "/usr/lib/python2.7/site-packages/pulp/server/async/scheduler.py", >>>>>>>>>>>> line >>>>>>>>>>>>>>>> 230, in call_tick >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> ret >>>>>>>>>>>>>>>> = super(Scheduler, self).tick() >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/usr/lib/python2.7/site-packages/celery/beat.py", line >>>> 220, in >>>>>>>>>> tick >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITICAL: >>>>>>>> (7070-59168) >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>>>> next_time_to_run = self.maybe_due(entry, self.publisher) >>>>>>>>>>>>>>>> Aug 1 08:45:03 dscaprv01 pulp: celery.beat:CRITIC >>> >>> > -- You received this message because you are subscribed to the Google Groups "Foreman users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/foreman-users. For more options, visit https://groups.google.com/d/optout.
