cluther wrote:
> Try doing the previous debugging steps, but specify the --cycle
> command line option to zenperfsnmp or whichever daemon you want to
> test with. I'd really like to see what is happening in zenhub when
> this iteration errors occurs.
With --cycle option for zencommand, here's the zenhub output:
Code:
DEBUG:zen.Events:EventClassInst=Start
DEBUG:zen.DbConnectionPool:Retrieved a connection; Pool size: 0
DEBUG:zen.Events:update status set clearid = '7f0000013633fcd1fffecac' where
device='localhost.localdomain' and component='zencommand' and eventKey='' and
(eventClass='/App/Stop' or eventClass='/App/Start'): --> 1
DEBUG:zen.Events:insert into log (evid, userName, text) select evid, "admin",
"auto cleared" from status where clearid = "7f0000013633fcd1fffecac": --> 1
DEBUG:zen.Events:DELETE FROM status WHERE clearid IS NOT NULL: --> 1
DEBUG:zen.Events:insert into history set
firstTime=1198864720.998,severity=0,component='zencommand',agent='zencommand',summary='started',dedupid='localhost.localdomain|zencommand|/App/Start||0|started',manager='localhost',eventKey='',device='localhost.localdomain',eventClass='/App/Start',lastTime=1198864720.998,message='started',deletedTime=null,evid='7f0000013633fcd1fffecac':
--> 1
DEBUG:zen.DbConnectionPool:Returned a connection; Pool size: 1
DEBUG:zen.DbConnectionPool:Retrieved a connection; Pool size: 0
DEBUG:zen.Events:insert into heartbeat set
device='localhost.localdomain',component='zencommand',timeout=180 on duplicate
key update lastTime=Null, timeout=180: --> 2
DEBUG:zen.DbConnectionPool:Returned a connection; Pool size: 1
DEBUG:zen.DbConnectionPool:Retrieved a connection; Pool size: 0
DEBUG:zen.Events:insert into heartbeat set
device='localhost.localdomain',component='zencommand',timeout=180 on duplicate
key update lastTime=Null, timeout=180: --> 2
DEBUG:zen.DbConnectionPool:Returned a connection; Pool size: 1
Here's the output from zencommand during this time:
Code:
DEBUG:zen.zencommand:run
DEBUG:zen.zencommand:Connecting to localhost
DEBUG:zen.zencommand:Logging in as admin
WARNING:zen.zencommand:Reconnected to ZenHub
DEBUG:zen.zencommand:setting up services EventService, CommandConfig
DEBUG:zen.zencommand:chaining getInitialServices with d2
DEBUG:zen.zencommand:callback after getting service EventService
DEBUG:zen.zencommand:callback after getting service CommandConfig
DEBUG:zen.zencommand:Calling connected.
DEBUG:zen.zencommand:connected
DEBUG:zen.zencommand:Sending event {'severity': 0, 'component': 'zencommand',
'agent': 'zencommand', 'summary': 'started', 'manager': 'localhost', 'device':
'localhost.localdomain', 'eventClass': '/App/Start'}
DEBUG:zen.zencommand:Fetching config
DEBUG:zen.zencommand:Updated configCycleInterval config to 360
DEBUG:zen.zencommand:Loading classes ['Products.ZenModel.MinMaxThreshold']
INFO:zen.zencommand:---------- - schedule has 0 commands
DEBUG:zen.zencommand:Finished config fetch
INFO:zen.zencommand:---------- - schedule has 0 commands
DEBUG:zen.zencommand:Sending event {'manager': 'localhost', 'timeout': 180,
'device': 'localhost.localdomain', 'eventClass': '/Heartbeat', 'component':
'zencommand', 'agent': 'zencommand'}
DEBUG:zen.zencommand:Sending event {'manager': 'localhost', 'timeout': 180,
'device': 'localhost.localdomain', 'eventClass': '/Heartbeat', 'component':
'zencommand', 'agent': 'zencommand'}
It just keeps cycling just fine and never errors out. So I did this for an
experiment.
1. Run zenoss start script as root to fire up the system as usual using the
official /etc/init.d/zenoss script.
This gets zenhub and all other daemons but the three in question running just
fine as normal.
2. Run the zencommand run -v 10 on command line to see what happens with
verbose output. Here's the log from running zencommand against zenhub started
with official start scripts:
Code:
DEBUG:zen.zencommand:run
DEBUG:zen.zencommand:Connecting to localhost
DEBUG:zen.zencommand:Logging in as admin
WARNING:zen.zencommand:Reconnected to ZenHub
DEBUG:zen.zencommand:setting up services EventService, CommandConfig
DEBUG:zen.zencommand:chaining getInitialServices with d2
DEBUG:zen.zencommand:callback after getting service EventService
DEBUG:zen.zencommand:callback after getting service CommandConfig
DEBUG:zen.zencommand:Calling connected.
DEBUG:zen.zencommand:connected
DEBUG:zen.zencommand:Sending event {'severity': 0, 'component': 'zencommand',
'agent': 'zencommand', 'summary': 'started', 'manager': 'localhost', 'device':
'localhost.localdomain', 'eventClass': '/App/Start'}
DEBUG:zen.zencommand:Fetching config
DEBUG:zen.zencommand:Updated configCycleInterval config to 360
DEBUG:zen.zencommand:Loading classes [Failure instance: Traceback from remote
host -- Traceback unavailable
]
ERROR:zen.zencommand:iteration over non-sequence
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenRRD/zencommand.py", line 580, in doFetchConfig
self.remote_updateThresholdClasses(driver.next())
File "/opt/zenoss/Products/ZenRRD/RRDDaemon.py", line 76, in
remote_updateThresholdClasses
for c in classes:
TypeError: iteration over non-sequence
ERROR:zen.zencommand:iteration over non-sequence
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenRRD/zencommand.py", line 607, in start
driver.next()
File "/opt/zenoss/Products/ZenUtils/Driver.py", line 58, in next
raise ex
TypeError: iteration over non-sequence
ERROR:zen.zencommand:iteration over non-sequence
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenUtils/Driver.py", line 46, in _next
self.iter.next().addBoth(self._finish)
File "/opt/zenoss/Products/ZenRRD/zencommand.py", line 613, in start
raise ex
TypeError: iteration over non-sequence
DEBUG:zen.zencommand:Sending event {'severity': 3, 'component': 'zencommand',
'agent': 'zencommand', 'summary': 'stopped', 'manager': 'localhost', 'device':
'localhost.localdomain', 'eventClass': '/App/Stop'}
DEBUG:zen.zencommand:Sending event {'manager': 'localhost', 'timeout': 180,
'device': 'localhost.localdomain', 'eventClass': '/Heartbeat', 'component':
'zencommand', 'agent': 'zencommand'}
DEBUG:zen.zencommand:removing service EventService
DEBUG:zen.zencommand:removing service CommandConfig
INFO:zen.zencommand:zencommand shutting down
So it fails when loading classes. When running zenhub in foreground the
zencommand can load classes and prints out this debug line:
Code:
DEBUG:zen.zencommand:Loading classes ['Products.ZenModel.MinMaxThreshold']
When running zenhub in background as a daemon with usual start scripts the
zencommand can no longer load classes and prints out this debug line:
Code:
DEBUG:zen.zencommand:Loading classes [Failure instance: Traceback from remote
host -- Traceback unavailable
]
Followed by the stack trace we've been discussing in this thread.
So my question is, where are these classes (Products.ZenModel.MinMaxThreshold)
coming from? The Zope database or some code on disk?
Do we have some sort of permissions or path issue where running the hub in
daemon mode is not configured correctly, but running in foreground works okay
for whatever reason?
-------------------- m2f --------------------
Read this topic online here:
http://community.zenoss.com/forums/viewtopic.php?p=14759#14759
-------------------- m2f --------------------
_______________________________________________
zenoss-users mailing list
[email protected]
http://lists.zenoss.org/mailman/listinfo/zenoss-users
