[ovirt-users] Re: engine mails about FSM states
14.12.2018 15:52, Martin Sivak пишет:
Hi,
> Host id is not set
This is an internal error report that should normally not happen. It
means the ovirt-ha-agent asked for a storage operation before it
registered itself with the broker. If this happens seldom then it looks
like a race condition.
I'm seeing this events also on my setup randomly and seldom.
versions:
ovirt-release42-4.2.7.1-1.el7.noarch
ovirt-hosted-engine-ha-2.2.18-1.el7.noarch
ovirt-host-4.2.3-1.el7.x86_64
...
I have emails from 15 nov, 21 nov, 01 dec, 05 dec...
Last event at 09 dec 16:35:
-
MainThread::INFO::2018-12-09
16:34:05,223::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop)
Current state EngineUp (score: 3400)
MainThread::INFO::2018-12-09
16:34:15,360::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop)
Current state EngineUp (score: 3400)
... ALL OK HERE UNTIL..
MainThread::ERROR::2018-12-09
16:34:25,373::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Unhandled monitoring loop exception
Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 428, in start_monitoring
self._monitoring_loop()
.
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 135, in get_stats_from_storage
result = self._proxy.get_stats()
.
error: [Errno 2] No such file or directory
.
-
MainThread::ERROR::2018-12-09
16:34:25,380::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
Trying to restart agent
.
MainThread::INFO::2018-12-09
16:34:46,353::brokerlink::77::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
Starting monitor storage-domain, options {'sd_uuid':
'0571ac7b-a28e-4e20-9cd8-4803e40ec602'}
MainThread::INFO::2018-12-09
16:34:46,354::brokerlink::85::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
Success, id storage-domain
.
MainThread::INFO::2018-12-09
16:35:25,062::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify)
Success, was notification of state_transition (EngineStarting-EngineUp)
sent? sent
.
MainThread::INFO::2018-12-09
16:35:31,209::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop)
Current state EngineUp (score: 3400)
-
I would recommend opening a bug report with all the logs we talked about
and all the RPM versions (ovirt-hosted-engine-ha and
ovirt-hosted-engine-setup packages). Use this link to go directly to the
right component:
https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-hosted-engine-ha
Are anybody open this bug report ? I do not have account in RH bugzilla..
(Accidentally this (9 dec) log file is rotated while i wrote this email...)
On Fri, Dec 14, 2018 at 12:35 PM fsoyer > wrote:
In borker.log I found this, just before 05:59am:
Thread-3::INFO::2018-12-13
05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree:
82101
Thread-1::INFO::2018-12-13
05:58:46,322::ping::60::ping.Ping::(action) Successfully pinged
10.0.1.254
Thread-5::INFO::2018-12-13
05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-2::INFO::2018-12-13
05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action)
Found bridge ovirtmgmt with ports
StatusStorageThread::ERROR::2018-12-13
05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run)
Failed to update state.
Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 82, in run
if (self._status_broker._inquire_whiteboard_lock() or
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 190, in _inquire_whiteboard_lock
self.host_id, self._lease_file)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 128, in host_id
raise ex.HostIdNotLockedError("Host id is not set")
HostIdNotLockedError: Host id is not set
StatusStorageThread::ERROR::2018-12-13
05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart)
Trying to restart the broker
"Host is not set" ???
--
Regards,
*Frank*
Le Vendredi, Décembre 14, 2018 12:27 CET, Martin Sivak
mailto:msi...@redhat.com>> a écrit:
Hi,
check the broker.log as well. The connect
[ovirt-users] Re: engine mails about FSM states
Hi,
> Host id is not set
This is an internal error report that should normally not happen. It means
the ovirt-ha-agent asked for a storage operation before it registered
itself with the broker. If this happens seldom then it looks like a race
condition.
I would recommend opening a bug report with all the logs we talked about
and all the RPM versions (ovirt-hosted-engine-ha and
ovirt-hosted-engine-setup packages). Use this link to go directly to the
right component:
https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-hosted-engine-ha
Since all is well except the emails I recommend filtering out the emails as
a workaround before this can be fully investigated and fixed.
Simone, Denis: I can't do more here, looks like a race in agent - broker
initialization and host id management.
Best regards
Martin Sivak
On Fri, Dec 14, 2018 at 12:35 PM fsoyer wrote:
> In borker.log I found this, just before 05:59am:
>
> Thread-3::INFO::2018-12-13
> 05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree: 82101
> Thread-1::INFO::2018-12-13 05:58:46,322::ping::60::ping.Ping::(action)
> Successfully pinged 10.0.1.254
> Thread-5::INFO::2018-12-13
> 05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats)
> VM is up on this host with healthy engine
> Thread-2::INFO::2018-12-13
> 05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found
> bridge ovirtmgmt with ports
> StatusStorageThread::ERROR::2018-12-13
> 05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run)
> Failed to update state.
> Traceback (most recent call last):
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
> line 82, in run
> if (self._status_broker._inquire_whiteboard_lock() or
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
> line 190, in _inquire_whiteboard_lock
> self.host_id, self._lease_file)
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
> line 128, in host_id
> raise ex.HostIdNotLockedError("Host id is not set")
> HostIdNotLockedError: Host id is not set
> StatusStorageThread::ERROR::2018-12-13
> 05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart)
> Trying to restart the broker
>
> "Host is not set" ???
> --
>
> Regards,
>
> *Frank*
>
>
> Le Vendredi, Décembre 14, 2018 12:27 CET, Martin Sivak
> a écrit:
>
>
> Hi,
>
> check the broker.log as well. The connect is used to talk to
> ovirt-ha-broker service socket.
>
> Best regards
>
> Martin Sivak
>
>
>
> On Fri, Dec 14, 2018 at 12:20 PM fsoyer wrote:
>
>> I think I have it in agent.log. What can be this "file not found" ?
>>
>> MainThread::ERROR::2018-12-13
>> 05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
>> Unhandled monitoring loop exception
>> Traceback (most recent call last):
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
>> line 428, in start_monitoring
>> self._monitoring_loop()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
>> line 447, in _monitoring_loop
>> for old_state, state, delay in self.fsm:
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py",
>> line 127, in next
>> new_data = self.refresh(self._state.data)
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py",
>> line 81, in refresh
>> stats.update(self.hosted_engine.collect_stats())
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
>> line 736, in collect_stats
>> all_stats = self._broker.get_stats_from_storage()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 135, in get_stats_from_storage
>> result = self._proxy.get_stats()
>> File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
>> return self.__send(self.__name, args)
>> File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
>> verbose=self.__verbose
>> File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
>> return self.single_request(host, handler, request_body, verbose)
>> File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
>> self.send_content(h, request_body)
>> File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
>> connection.endheaders(request_body)
>> File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
>> self._send_output(message_body)
>> File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
>> self.send(msg)
>> File "/usr/lib64/python2.7/httplib.py", line 843, in send
>> self.connect()
>> File
>>
[ovirt-users] Re: engine mails about FSM states
In borker.log I found this, just before 05:59am:Thread-3::INFO::2018-12-13
05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree: 82101
Thread-1::INFO::2018-12-13 05:58:46,322::ping::60::ping.Ping::(action)
Successfully pinged 10.0.1.254
Thread-5::INFO::2018-12-13
05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats)
VM is up on this host with healthy engine
Thread-2::INFO::2018-12-13
05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action) Found bridge
ovirtmgmt with ports
StatusStorageThread::ERROR::2018-12-13
05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run)
Failed to update state.
Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 82, in run
if (self._status_broker._inquire_whiteboard_lock() or
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 190, in _inquire_whiteboard_lock
self.host_id, self._lease_file)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
line 128, in host_id
raise ex.HostIdNotLockedError("Host id is not set")
HostIdNotLockedError: Host id is not set
StatusStorageThread::ERROR::2018-12-13
05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart)
Trying to restart the broker"Host is not set" ???
--
Regards,
Frank
Le Vendredi, Décembre 14, 2018 12:27 CET, Martin Sivak a
écrit:
Hi, check the broker.log as well. The connect is used to talk to
ovirt-ha-broker service socket. Best regards Martin Sivak On Fri, Dec 14,
2018 at 12:20 PM fsoyer wrote:I think I have it in
agent.log. What can be this "file not found" ?
MainThread::ERROR::2018-12-13
05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Unhandled monitoring loop exception
Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 428, in start_monitoring
self._monitoring_loop()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 447, in _monitoring_loop
for old_state, state, delay in self.fsm:
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py",
line 127, in next
new_data = self.refresh(self._state.data)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py",
line 81, in refresh
stats.update(self.hosted_engine.collect_stats())
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 736, in collect_stats
all_stats = self._broker.get_stats_from_storage()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
line 135, in get_stats_from_storage
result = self._proxy.get_stats()
File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
verbose=self.__verbose
File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
self.send_content(h, request_body)
File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
connection.endheaders(request_body)
File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output
self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 843, in send
self.connect()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line
52, in connect
self.sock.connect(base64.b16decode(self.host))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory
MainThread::ERROR::2018-12-13
05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
Traceback (most recent call last):
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line
131, in _run_agent
return action(he)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line
55, in action_proper
return he.start_monitoring()
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 435, in start_monitoring
self.publish(stopped)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 337, in publish
self._push_to_storage(blocks)
File
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
line 707, in _push_to_storage
[ovirt-users] Re: engine mails about FSM states
Hi, check the broker.log as well. The connect is used to talk to ovirt-ha-broker service socket. Best regards Martin Sivak On Fri, Dec 14, 2018 at 12:20 PM fsoyer wrote: > I think I have it in agent.log. What can be this "file not found" ? > > MainThread::ERROR::2018-12-13 > 05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) > Unhandled monitoring loop exception > Traceback (most recent call last): > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", > line 428, in start_monitoring > self._monitoring_loop() > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", > line 447, in _monitoring_loop > for old_state, state, delay in self.fsm: > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", > line 127, in next > new_data = self.refresh(self._state.data) > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", > line 81, in refresh > stats.update(self.hosted_engine.collect_stats()) > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", > line 736, in collect_stats > all_stats = self._broker.get_stats_from_storage() > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", > line 135, in get_stats_from_storage > result = self._proxy.get_stats() > File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ > return self.__send(self.__name, args) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request > verbose=self.__verbose > File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request > return self.single_request(host, handler, request_body, verbose) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request > self.send_content(h, request_body) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content > connection.endheaders(request_body) > File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders > self._send_output(message_body) > File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output > self.send(msg) > File "/usr/lib64/python2.7/httplib.py", line 843, in send > self.connect() > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", > line 52, in connect > self.sock.connect(base64.b16decode(self.host)) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) > error: [Errno 2] No such file or directory > MainThread::ERROR::2018-12-13 > 05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) > Traceback (most recent call last): > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", > line 131, in _run_agent > return action(he) > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", > line 55, in action_proper > return he.start_monitoring() > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", > line 435, in start_monitoring > self.publish(stopped) > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", > line 337, in publish > self._push_to_storage(blocks) > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", > line 707, in _push_to_storage > self._broker.put_stats_on_storage(self.host_id, blocks) > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", > line 105, in put_stats_on_storage > self._proxy.put_stats(host_id, xmlrpclib.Binary(data)) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ > return self.__send(self.__name, args) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request > verbose=self.__verbose > File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request > return self.single_request(host, handler, request_body, verbose) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request > self.send_content(h, request_body) > File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content > connection.endheaders(request_body) > File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders > self._send_output(message_body) > File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output > self.send(msg) > File "/usr/lib64/python2.7/httplib.py", line 843, in send > self.connect() > File > "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", > line 52, in connect > self.sock.connect(base64.b16decode(self.host)) > File "/usr/lib64/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) > error: [Errno 2] No such file or directory > > MainThread::ERROR::2018-12-13 >
[ovirt-users] Re: engine mails about FSM states
I think I have it in agent.log. What can be this "file not found" ? MainThread::ERROR::2018-12-13 05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Unhandled monitoring loop exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 428, in start_monitoring self._monitoring_loop() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 447, in _monitoring_loop for old_state, state, delay in self.fsm: File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", line 127, in next new_data = self.refresh(self._state.data) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", line 81, in refresh stats.update(self.hosted_engine.collect_stats()) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 736, in collect_stats all_stats = self._broker.get_stats_from_storage() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 135, in get_stats_from_storage result = self._proxy.get_stats() File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory MainThread::ERROR::2018-12-13 05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent return action(he) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper return he.start_monitoring() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 435, in start_monitoring self.publish(stopped) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 337, in publish self._push_to_storage(blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 707, in _push_to_storage self._broker.put_stats_on_storage(self.host_id, blocks) File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 105, in put_stats_on_storage self._proxy.put_stats(host_id, xmlrpclib.Binary(data)) File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request self.send_content(h, request_body) File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content connection.endheaders(request_body) File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 881, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 843, in send self.connect() File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 52, in connect self.sock.connect(base64.b16decode(self.host)) File "/usr/lib64/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory MainThread::ERROR::2018-12-13 05:59:04,043::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent MainThread::INFO::2018-12-13 05:59:04,044::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down MainThread::INFO::2018-12-13 05:59:14,923::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run) ovirt-hosted-engine-ha agent 2.2.16 started --
[ovirt-users] Re: engine mails about FSM states
Hi, no StartState is not common, it is only ever entered when the agent boots up. So something restarted or killed the agent process. Check the agent log in /var/log/ovirt-hosted-engine-ha for errors. Best regards Martin Sivak On Fri, Dec 14, 2018 at 12:05 PM fsoyer wrote: > > Hi Martin, > my problem is that nobody restarted the agent. Do you mean that this is not a > normal behavior ? Is it possible that it restarts itself ? > > Thanks > -- > > Regards, > > Frank > > > > Le Jeudi, Décembre 13, 2018 15:25 CET, Martin Sivak a > écrit: > > > Hi, > > those are state change notifications from the hosted engine agent. It > basically means somebody restarted the ha-agent process and it found > out the VM is still running fine and returned to the proper state. > > Configuring it is possible using the broker.conf file in > /etc/ovirt-hosted-engine-ha (look for the notification section) or the > hosted-engine tool (search --help for set config) depending on the > version of hosted engine you are using. > > Best regards > > -- > Martin Sivak > > > On Thu, Dec 13, 2018 at 3:10 PM fsoyer wrote: > > > > Hi, > > I don't find revelant answer about this. Sorry il this was already asked. > > I receive randomly (one or two tims a week, differents hours) 3 mails with > > this subjects : > > first : ovirt-hosted-engine state transition StartState-ReinitializeFSM > > second : ovirt-hosted-engine state transition ReinitializeFSM-EngineStarting > > third : ovirt-hosted-engine state transition EngineStarting-EngineUp > > all at exactly the same time. The "events" in GUI doesn't indicate anything > > about this. No impact on engine or VMs. > > So I wonder what this messages means ? And, if case of just "info" > > messages, is there a way to disable them ? > > > > Thanks. > > -- > > > > Reagrds, > > > > Frank > > > > ___ > > Users mailing list -- users@ovirt.org > > To unsubscribe send an email to users-le...@ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDEHASTCQHFHX62U4K4ZCOSK/ > > > > ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6B7QUFZXKMNMRTK4KKDEVZUEAP2FROII/
[ovirt-users] Re: engine mails about FSM states
Hi Martin, my problem is that nobody restarted the agent. Do you mean that this is not a normal behavior ? Is it possible that it restarts itself ? Thanks -- Regards, Frank Le Jeudi, Décembre 13, 2018 15:25 CET, Martin Sivak a écrit: Hi, those are state change notifications from the hosted engine agent. It basically means somebody restarted the ha-agent process and it found out the VM is still running fine and returned to the proper state. Configuring it is possible using the broker.conf file in /etc/ovirt-hosted-engine-ha (look for the notification section) or the hosted-engine tool (search --help for set config) depending on the version of hosted engine you are using. Best regards -- Martin Sivak On Thu, Dec 13, 2018 at 3:10 PM fsoyer wrote: > > Hi, > I don't find revelant answer about this. Sorry il this was already asked. > I receive randomly (one or two tims a week, differents hours) 3 mails with > this subjects : > first : ovirt-hosted-engine state transition StartState-ReinitializeFSM > second : ovirt-hosted-engine state transition ReinitializeFSM-EngineStarting > third : ovirt-hosted-engine state transition EngineStarting-EngineUp > all at exactly the same time. The "events" in GUI doesn't indicate anything > about this. No impact on engine or VMs. > So I wonder what this messages means ? And, if case of just "info" messages, > is there a way to disable them ? > > Thanks. > -- > > Reagrds, > > Frank > > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDEHASTCQHFHX62U4K4ZCOSK/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/JKVPQ2ZTQHH2U4C6JJN6ZMBYHBGK2P5E/
[ovirt-users] Re: engine mails about FSM states
Hi, those are state change notifications from the hosted engine agent. It basically means somebody restarted the ha-agent process and it found out the VM is still running fine and returned to the proper state. Configuring it is possible using the broker.conf file in /etc/ovirt-hosted-engine-ha (look for the notification section) or the hosted-engine tool (search --help for set config) depending on the version of hosted engine you are using. Best regards -- Martin Sivak On Thu, Dec 13, 2018 at 3:10 PM fsoyer wrote: > > Hi, > I don't find revelant answer about this. Sorry il this was already asked. > I receive randomly (one or two tims a week, differents hours) 3 mails with > this subjects : > first : ovirt-hosted-engine state transition StartState-ReinitializeFSM > second : ovirt-hosted-engine state transition ReinitializeFSM-EngineStarting > third : ovirt-hosted-engine state transition EngineStarting-EngineUp > all at exactly the same time. The "events" in GUI doesn't indicate anything > about this. No impact on engine or VMs. > So I wonder what this messages means ? And, if case of just "info" messages, > is there a way to disable them ? > > Thanks. > -- > > Reagrds, > > Frank > > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDEHASTCQHFHX62U4K4ZCOSK/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/67JW6HHNSDAOR2SIUOG4HJFWO55N3MLG/
