Project:
http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-master/
Build:
http://jenkins.ovirt.org/job/ovirt-system-tests_he-node-ng-suite-master/222/
Build Number: 222
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #222
[Milan Zamazal] Wait for snapshot events correctly in
preview_snapshot_with_memory
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 008_restart_he_vm.restart_he_vm
Error Message:
could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data": true,
"extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2518
(Wed Jul 25 04:20:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2519
(Wed Jul 25 04:20:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2512 (Wed Jul
25 04:19:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2512 (Wed Jul 25
04:19:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2, "eng
-------------------- >> begin captured logging << --------------------
lago.ssh: DEBUG: start task:932dbfdb-8f78-42c6-9b1e-6aadee5d8091:Get ssh client
for lago-he-node-ng-suite-master-host-1:
lago.ssh: DEBUG: end task:932dbfdb-8f78-42c6-9b1e-6aadee5d8091:Get ssh client
for lago-he-node-ng-suite-master-host-1:
lago.ssh: DEBUG: Running 7763a99a on lago-he-node-ng-suite-master-host-1:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 7763a99a on lago-he-node-ng-suite-master-host-1
returned with 0
lago.ssh: DEBUG: Command 7763a99a on lago-he-node-ng-suite-master-host-1 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2275 (Wed Jul
25 04:16:03 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2275 (Wed Jul 25
04:16:03
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32": "634493fe",
"local_conf_timestamp": 2275, "host-ts": 2275}, "2": {"conf_on_shared_storage":
true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2281 (Wed Jul
25 04:16:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2281 (Wed Jul 25
04:16:08
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "69ca08d9", "local_conf_timestamp": 2281,
"host-ts": 2281}, "global_maintenance": true}
root: INFO: * Shutting down HE VM on host:
lago-he-node-ng-suite-master-host-0
lago.ssh: DEBUG: start task:2bd52306-1957-43f5-a4cd-a16a4223bbd2:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:2bd52306-1957-43f5-a4cd-a16a4223bbd2:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 78525040 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-shutdown
lago.ssh: DEBUG: Command 78525040 on lago-he-node-ng-suite-master-host-0
returned with 0
root: INFO: * Command succeeded
root: INFO: * Waiting for VM to be down...
lago.ssh: DEBUG: start task:75e52da9-c741-4089-8bda-708623dbb4cc:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:75e52da9-c741-4089-8bda-708623dbb4cc:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 797baba6 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 797baba6 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command 797baba6 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2285 (Wed Jul
25 04:16:13 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2286 (Wed Jul 25
04:16:13
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineUp\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32": "bc8dc98f",
"local_conf_timestamp": 2286, "host-ts": 2285}, "2": {"conf_on_shared_storage":
true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2281 (Wed Jul
25 04:16:08 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2281 (Wed Jul 25
04:16:08
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=EngineDown\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "69ca08d9", "local_conf_timestamp": 2281,
"host-ts": 2281}, "global_maintenance": true}
lago.ssh: DEBUG: start task:ac00a93a-e808-4f72-bb6f-f29c7bc259a6:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:ac00a93a-e808-4f72-bb6f-f29c7bc259a6:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 812bb044 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 812bb044 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command 812bb044 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2295 (Wed Jul
25 04:16:22 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2295 (Wed Jul 25
04:16:23
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Powering down"}, "score": 3000, "stopped": false,
"maintenance": false, "crc32": "bf71ee59", "local_conf_timestamp": 2295,
"host-ts": 2295}, "2": {"conf_on_shared_storage": true, "live-data": true,
"extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2291
(Wed Jul 25 04:16:18 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2291
(Wed Jul 25 04:16:19
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "47e1cb10", "local_conf_timestamp": 2291,
"host-ts": 2291}, "global_maintenance": true}
lago.ssh: DEBUG: start task:bb5c3ebf-f315-4147-9664-83fbdfb0b5e8:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:bb5c3ebf-f315-4147-9664-83fbdfb0b5e8:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 83e14d9e on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 83e14d9e on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command 83e14d9e on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2305 (Wed Jul
25 04:16:33 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2305 (Wed Jul 25
04:16:33
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm":
"down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "16c42d10", "local_conf_timestamp": 2305,
"host-ts": 2305}, "2": {"conf_on_shared_storage": true, "live-data": true,
"extra": "metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2301
(Wed Jul 25 04:16:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2301
(Wed Jul 25 04:16:28
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "21734107", "local_conf_timestamp": 2301,
"host-ts": 2301}, "global_maintenance": true}
root: INFO: * VM is down.
root: INFO: * Stopping services...
lago.ssh: DEBUG: start task:a91ce735-904b-4589-8d49-374d4d4e5233:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:a91ce735-904b-4589-8d49-374d4d4e5233:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 84cdbff8 on lago-he-node-ng-suite-master-host-0:
systemctl stop vdsmd ovirt-ha-broker ovirt-ha-agent
lago.ssh: DEBUG: Command 84cdbff8 on lago-he-node-ng-suite-master-host-0
returned with 0
root: INFO: * Starting services...
lago.ssh: DEBUG: start task:69e2b63b-31fc-464e-b7ff-c499a95be86e:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:69e2b63b-31fc-464e-b7ff-c499a95be86e:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 8dac9a18 on lago-he-node-ng-suite-master-host-0:
systemctl start vdsmd ovirt-ha-broker ovirt-ha-agent
lago.ssh: DEBUG: Command 8dac9a18 on lago-he-node-ng-suite-master-host-0
returned with 0
root: INFO: * Waiting for agent to be ready...
lago.ssh: DEBUG: start task:5c41febb-2a25-4889-a7c1-5c9da7a33f96:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:5c41febb-2a25-4889-a7c1-5c9da7a33f96:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 909bffb6 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command 909bffb6 on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command 909bffb6 on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:56d0e0c8-3a30-4b6e-acf5-ba1e387ceb7c:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:56d0e0c8-3a30-4b6e-acf5-ba1e387ceb7c:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 977e0a9a on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command 977e0a9a on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command 977e0a9a on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:4b36c31c-c6b7-4bbb-b038-b67e2a26e788:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:4b36c31c-c6b7-4bbb-b038-b67e2a26e788:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 9a1513fc on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command 9a1513fc on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command 9a1513fc on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:e3e68036-c86b-4a3e-ad6d-788272025c42:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:e3e68036-c86b-4a3e-ad6d-788272025c42:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 9cbc9a94 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command 9cbc9a94 on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command 9cbc9a94 on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:7b9f8a19-b7dc-4d00-9202-e07d88888dfe:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:7b9f8a19-b7dc-4d00-9202-e07d88888dfe:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 9f7865c4 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command 9f7865c4 on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command 9f7865c4 on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:1944f900-1947-46ca-b701-3f71d3025a23:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:1944f900-1947-46ca-b701-3f71d3025a23:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running a212e07a on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command a212e07a on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command a212e07a on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:289afe1e-8a48-4e1f-b585-c7afe283cc2a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:289afe1e-8a48-4e1f-b585-c7afe283cc2a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running a4b92d66 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command a4b92d66 on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command a4b92d66 on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:65b385d4-2abe-4c8a-9f9f-37904c3320b5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:65b385d4-2abe-4c8a-9f9f-37904c3320b5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running a755a3f6 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command a755a3f6 on lago-he-node-ng-suite-master-host-0
returned with 1
lago.ssh: DEBUG: Command a755a3f6 on lago-he-node-ng-suite-master-host-0 output:
The hosted engine configuration has not been retrieved from shared storage.
Please ensure that ovirt-ha-agent is running and the storage server is
reachable.
lago.ssh: DEBUG: start task:179389d5-7a94-4742-9328-73c41ed07ed0:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:179389d5-7a94-4742-9328-73c41ed07ed0:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running a9fe6f48 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status
lago.ssh: DEBUG: Command a9fe6f48 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command a9fe6f48 on lago-he-node-ng-suite-master-host-0 output:
!! Cluster is in GLOBAL MAINTENANCE mode !!
--== Host lago-he-node-ng-suite-master-host-0 (id: 1) status ==--
conf_on_shared_storage : True
Status up-to-date : True
Hostname : lago-he-node-ng-suite-master-host-0
Host ID : 1
Engine status : {"reason": "bad vm status", "health":
"bad", "vm": "down", "detail": "Down"}
Score : 0
stopped : False
Local maintenance : False
crc32 : 744e472a
local_conf_timestamp : 2368
Host timestamp : 2367
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=2367 (Wed Jul 25 04:17:35 2018)
host-id=1
score=0
vm_conf_refresh_time=2368 (Wed Jul 25 04:17:35 2018)
conf_on_shared_storage=True
maintenance=False
state=ReinitializeFSM
stopped=False
--== Host lago-he-node-ng-suite-master-host-1 (id: 2) status ==--
conf_on_shared_storage : True
Status up-to-date : False
Hostname : lago-he-node-ng-suite-master-host-1
Host ID : 2
Engine status : unknown stale-data
Score : 3400
stopped : False
Local maintenance : False
crc32 : c58bc9b1
local_conf_timestamp : 2361
Host timestamp : 2361
Extra metadata (valid at timestamp):
metadata_parse_version=1
metadata_feature_version=1
timestamp=2361 (Wed Jul 25 04:17:28 2018)
host-id=2
score=3400
vm_conf_refresh_time=2361 (Wed Jul 25 04:17:29 2018)
conf_on_shared_storage=True
maintenance=False
state=GlobalMaintenance
stopped=False
!! Cluster is in GLOBAL MAINTENANCE mode !!
root: INFO: * Agent is ready.
root: INFO: * Starting VM...
lago.ssh: DEBUG: start task:771d13e3-aa83-4b0c-ad2f-73254bee44b5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:771d13e3-aa83-4b0c-ad2f-73254bee44b5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running ab02495a on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-start
lago.ssh: DEBUG: Command ab02495a on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command ab02495a on lago-he-node-ng-suite-master-host-0 output:
VM exists and is down, cleaning up and restarting
root: INFO: * Command succeeded
root: INFO: * Waiting for VM to be UP...
lago.ssh: DEBUG: start task:9b23a87d-00d7-4b1b-9df3-d4f2a93a347a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:9b23a87d-00d7-4b1b-9df3-d4f2a93a347a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running ad1c960a on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command ad1c960a on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command ad1c960a on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2367 (Wed Jul
25 04:17:35 2018)\nhost-id=1\nscore=0\nvm_conf_refresh_time=2368 (Wed Jul 25
04:17:35
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=ReinitializeFSM\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down",
"detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32":
"744e472a", "local_conf_timestamp": 2368, "host-ts": 2367}, "2":
{"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2361 (Wed Jul
25 04:17:28 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2361 (Wed Jul 25
04:17:29
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c58bc9b1", "local_conf_timestamp": 2361,
"host-ts": 2361}, "global_maintenance": true}
lago.ssh: DEBUG: start task:38723e85-fc65-4a4c-903e-49754f691574:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:38723e85-fc65-4a4c-903e-49754f691574:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running b40283bc on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command b40283bc on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command b40283bc on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2377 (Wed Jul
25 04:17:44 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2377 (Wed Jul 25
04:17:45
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down",
"detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "40391805", "local_conf_timestamp": 2377, "host-ts": 2377}, "2":
{"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2381 (Wed Jul
25 04:17:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2381 (Wed Jul 25
04:17:49
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "8b5c590e", "local_conf_timestamp": 2381,
"host-ts": 2381}, "global_maintenance": true}
lago.ssh: DEBUG: start task:cd1092f6-e8c6-40ee-99d2-6a6edd7be1b5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:cd1092f6-e8c6-40ee-99d2-6a6edd7be1b5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running b6c56286 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command b6c56286 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command b6c56286 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2387 (Wed Jul
25 04:17:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2388 (Wed Jul 25
04:17:55
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d2ce9013", "local_conf_timestamp": 2388, "host-ts": 2387},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2381 (Wed Jul
25 04:17:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2381 (Wed Jul 25
04:17:49
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "8b5c590e", "local_conf_timestamp": 2381,
"host-ts": 2381}, "global_maintenance": true}
root: INFO: * VM is UP.
root: INFO: * Waiting for engine to start...
lago.ssh: DEBUG: start task:567f9428-43b1-44f5-ad18-0f4a01c91b13:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:567f9428-43b1-44f5-ad18-0f4a01c91b13:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running b7c81b4c on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command b7c81b4c on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command b7c81b4c on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2387 (Wed Jul
25 04:17:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2388 (Wed Jul 25
04:17:55
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d2ce9013", "local_conf_timestamp": 2388, "host-ts": 2387},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2381 (Wed Jul
25 04:17:48 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2381 (Wed Jul 25
04:17:49
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "8b5c590e", "local_conf_timestamp": 2381,
"host-ts": 2381}, "global_maintenance": true}
lago.ssh: DEBUG: start task:029af7ab-0480-465a-ad7d-9d3a37b5b355:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:029af7ab-0480-465a-ad7d-9d3a37b5b355:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running bec6f814 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command bec6f814 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command bec6f814 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2398 (Wed Jul
25 04:18:05 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2398 (Wed Jul 25
04:18:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "412e4ba9", "local_conf_timestamp": 2398, "host-ts": 2398},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2391 (Wed Jul
25 04:17:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2392 (Wed Jul 25
04:17:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "3c7ecb7d", "local_conf_timestamp": 2392,
"host-ts": 2391}, "global_maintenance": true}
lago.ssh: DEBUG: start task:13363bf6-bedc-4e09-814b-bf03d9839892:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:13363bf6-bedc-4e09-814b-bf03d9839892:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running c1a71ce4 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command c1a71ce4 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command c1a71ce4 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2408 (Wed Jul
25 04:18:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2408 (Wed Jul 25
04:18:16
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d9432151", "local_conf_timestamp": 2408, "host-ts": 2408},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2402 (Wed Jul
25 04:18:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2402 (Wed Jul 25
04:18:09
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "9a202027", "local_conf_timestamp": 2402,
"host-ts": 2402}, "global_maintenance": true}
lago.ssh: DEBUG: start task:8e86ea8a-ed97-49c2-aa31-0d07662c480f:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:8e86ea8a-ed97-49c2-aa31-0d07662c480f:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running c4dffe62 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command c4dffe62 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command c4dffe62 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2408 (Wed Jul
25 04:18:16 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2408 (Wed Jul 25
04:18:16
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d9432151", "local_conf_timestamp": 2408, "host-ts": 2408},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2402 (Wed Jul
25 04:18:09 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2402 (Wed Jul 25
04:18:09
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "9a202027", "local_conf_timestamp": 2402,
"host-ts": 2402}, "global_maintenance": true}
lago.ssh: DEBUG: start task:e801a931-3f5c-463c-b00e-07905169e39b:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:e801a931-3f5c-463c-b00e-07905169e39b:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running c86e9048 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command c86e9048 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command c86e9048 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2418 (Wed Jul
25 04:18:26 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2419 (Wed Jul 25
04:18:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "625c3f40", "local_conf_timestamp": 2419, "host-ts": 2418},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2412 (Wed Jul
25 04:18:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2413 (Wed Jul 25
04:18:20
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c30cd378", "local_conf_timestamp": 2413,
"host-ts": 2412}, "global_maintenance": true}
lago.ssh: DEBUG: start task:fc9d9c05-cc70-49f4-9195-5801981130da:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:fc9d9c05-cc70-49f4-9195-5801981130da:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running cb60cc6c on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command cb60cc6c on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command cb60cc6c on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2418 (Wed Jul
25 04:18:26 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2419 (Wed Jul 25
04:18:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "625c3f40", "local_conf_timestamp": 2419, "host-ts": 2418},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2412 (Wed Jul
25 04:18:20 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2413 (Wed Jul 25
04:18:20
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c30cd378", "local_conf_timestamp": 2413,
"host-ts": 2412}, "global_maintenance": true}
lago.ssh: DEBUG: start task:a08eb28b-2662-4ab3-a80b-77ef2581a073:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:a08eb28b-2662-4ab3-a80b-77ef2581a073:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running cee93edc on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command cee93edc on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command cee93edc on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2428 (Wed Jul
25 04:18:35 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2428 (Wed Jul 25
04:18:36
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "ace23939", "local_conf_timestamp": 2428, "host-ts": 2428}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2422 (Wed Jul
25 04:18:29 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2422 (Wed Jul 25
04:18:29
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c677a7b5", "local_conf_timestamp": 2422,
"host-ts": 2422}, "global_maintenance": true}
lago.ssh: DEBUG: start task:5475c36c-7873-4bb4-b0a4-89b7a176ed10:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:5475c36c-7873-4bb4-b0a4-89b7a176ed10:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running d2517044 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command d2517044 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command d2517044 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2438 (Wed Jul
25 04:18:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2439 (Wed Jul 25
04:18:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "b02c5c74", "local_conf_timestamp": 2439, "host-ts": 2438}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2432 (Wed Jul
25 04:18:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2432 (Wed Jul 25
04:18:40
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "e85c647c", "local_conf_timestamp": 2432,
"host-ts": 2432}, "global_maintenance": true}
lago.ssh: DEBUG: start task:378598d4-4368-44ff-b912-b640c5a049b0:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:378598d4-4368-44ff-b912-b640c5a049b0:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running d56c33a4 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command d56c33a4 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command d56c33a4 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2438 (Wed Jul
25 04:18:46 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2439 (Wed Jul 25
04:18:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "b02c5c74", "local_conf_timestamp": 2439, "host-ts": 2438}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2432 (Wed Jul
25 04:18:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2432 (Wed Jul 25
04:18:40
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "e85c647c", "local_conf_timestamp": 2432,
"host-ts": 2432}, "global_maintenance": true}
lago.ssh: DEBUG: start task:7f5db09d-6b15-4582-8818-3c99d0291ce5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:7f5db09d-6b15-4582-8818-3c99d0291ce5:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running d863856c on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command d863856c on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command d863856c on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2448 (Wed Jul
25 04:18:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2448 (Wed Jul 25
04:18:56
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "0f24c3a2", "local_conf_timestamp": 2448, "host-ts": 2448}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2442 (Wed Jul
25 04:18:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2442 (Wed Jul 25
04:18:49
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "228f2f03", "local_conf_timestamp": 2442,
"host-ts": 2442}, "global_maintenance": true}
lago.ssh: DEBUG: start task:639271a3-0803-4020-966c-200e8adb5117:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:639271a3-0803-4020-966c-200e8adb5117:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running db6b7cf6 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command db6b7cf6 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command db6b7cf6 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2448 (Wed Jul
25 04:18:55 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2448 (Wed Jul 25
04:18:56
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "0f24c3a2", "local_conf_timestamp": 2448, "host-ts": 2448}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2442 (Wed Jul
25 04:18:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2442 (Wed Jul 25
04:18:49
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "228f2f03", "local_conf_timestamp": 2442,
"host-ts": 2442}, "global_maintenance": true}
lago.ssh: DEBUG: start task:d1c10391-bcf5-4501-ae6f-47a95d795283:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:d1c10391-bcf5-4501-ae6f-47a95d795283:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running de65a36e on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command de65a36e on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command de65a36e on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2458 (Wed Jul
25 04:19:06 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2458 (Wed Jul 25
04:19:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "64ed9679", "local_conf_timestamp": 2458, "host-ts": 2458}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2452 (Wed Jul
25 04:18:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2452 (Wed Jul 25
04:18:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "0ca4ecca", "local_conf_timestamp": 2452,
"host-ts": 2452}, "global_maintenance": true}
lago.ssh: DEBUG: start task:4e03db00-d1d4-43b0-bcfd-a21d3c32b7a7:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:4e03db00-d1d4-43b0-bcfd-a21d3c32b7a7:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running e173ed40 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command e173ed40 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command e173ed40 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2458 (Wed Jul
25 04:19:06 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2458 (Wed Jul 25
04:19:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "64ed9679", "local_conf_timestamp": 2458, "host-ts": 2458}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2452 (Wed Jul
25 04:18:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2452 (Wed Jul 25
04:18:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "0ca4ecca", "local_conf_timestamp": 2452,
"host-ts": 2452}, "global_maintenance": true}
lago.ssh: DEBUG: start task:d8173e8d-c1e7-49d4-8863-b1007ff2704f:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:d8173e8d-c1e7-49d4-8863-b1007ff2704f:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running e462c878 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command e462c878 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command e462c878 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2468 (Wed Jul
25 04:19:15 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2468 (Wed Jul 25
04:19:15
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "d8b66814", "local_conf_timestamp": 2468, "host-ts": 2468}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2462 (Wed Jul
25 04:19:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2463 (Wed Jul 25
04:19:10
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "09df9807", "local_conf_timestamp": 2463,
"host-ts": 2462}, "global_maintenance": true}
lago.ssh: DEBUG: start task:27e25ae4-3414-4640-8869-9cffad8f6814:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:27e25ae4-3414-4640-8869-9cffad8f6814:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running e794bfec on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command e794bfec on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command e794bfec on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2468 (Wed Jul
25 04:19:15 2018)\nhost-id=1\nscore=3400\nvm_conf_refresh_time=2468 (Wed Jul 25
04:19:15
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "d8b66814", "local_conf_timestamp": 2468, "host-ts": 2468}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2462 (Wed Jul
25 04:19:10 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2463 (Wed Jul 25
04:19:10
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "09df9807", "local_conf_timestamp": 2463,
"host-ts": 2462}, "global_maintenance": true}
lago.ssh: DEBUG: start task:d23aa18c-4462-4239-8db5-c29feecac71e:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:d23aa18c-4462-4239-8db5-c29feecac71e:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running eb998bd6 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command eb998bd6 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command eb998bd6 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2478 (Wed Jul
25 04:19:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2479 (Wed Jul 25
04:19:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "41565544", "local_conf_timestamp": 2479, "host-ts": 2478}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2472 (Wed Jul
25 04:19:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2472 (Wed Jul 25
04:19:20
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "50f36b58", "local_conf_timestamp": 2472,
"host-ts": 2472}, "global_maintenance": true}
lago.ssh: DEBUG: start task:37e54f8a-9fe3-47cd-8896-bb560cc5e4d1:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:37e54f8a-9fe3-47cd-8896-bb560cc5e4d1:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running eed21502 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command eed21502 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command eed21502 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2478 (Wed Jul
25 04:19:26 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2479 (Wed Jul 25
04:19:26
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "41565544", "local_conf_timestamp": 2479, "host-ts": 2478}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2472 (Wed Jul
25 04:19:19 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2472 (Wed Jul 25
04:19:20
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "50f36b58", "local_conf_timestamp": 2472,
"host-ts": 2472}, "global_maintenance": true}
lago.ssh: DEBUG: start task:b6793d51-1124-40aa-9f42-a2107d7649cc:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:b6793d51-1124-40aa-9f42-a2107d7649cc:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running f25fec4e on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command f25fec4e on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command f25fec4e on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2488 (Wed Jul
25 04:19:35 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2488 (Wed Jul 25
04:19:35
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "16f668c8", "local_conf_timestamp": 2488, "host-ts": 2488}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2482 (Wed Jul
25 04:19:30 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2483 (Wed Jul 25
04:19:30
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "470808b8", "local_conf_timestamp": 2483,
"host-ts": 2482}, "global_maintenance": true}
lago.ssh: DEBUG: start task:6fb40f9a-5636-47d5-a5e7-cf1835987996:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:6fb40f9a-5636-47d5-a5e7-cf1835987996:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running f5dc5560 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command f5dc5560 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command f5dc5560 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2498 (Wed Jul
25 04:19:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2499 (Wed Jul 25
04:19:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "0a380d85", "local_conf_timestamp": 2499, "host-ts": 2498}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2492 (Wed Jul
25 04:19:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2492 (Wed Jul 25
04:19:39
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "1e24fbe7", "local_conf_timestamp": 2492,
"host-ts": 2492}, "global_maintenance": true}
lago.ssh: DEBUG: start task:54953ae8-9f55-4cb1-bb4c-7ec68dc9879f:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:54953ae8-9f55-4cb1-bb4c-7ec68dc9879f:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running f99d7d32 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command f99d7d32 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command f99d7d32 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2498 (Wed Jul
25 04:19:46 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2499 (Wed Jul 25
04:19:46
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "0a380d85", "local_conf_timestamp": 2499, "host-ts": 2498}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2492 (Wed Jul
25 04:19:39 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2492 (Wed Jul 25
04:19:39
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "1e24fbe7", "local_conf_timestamp": 2492,
"host-ts": 2492}, "global_maintenance": true}
lago.ssh: DEBUG: start task:2d207ff4-7778-45e7-abe3-56af39bec1c8:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:2d207ff4-7778-45e7-abe3-56af39bec1c8:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running fd26f32a on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command fd26f32a on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command fd26f32a on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2508 (Wed Jul
25 04:19:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2508 (Wed Jul 25
04:19:56
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "ef34089b", "local_conf_timestamp": 2508, "host-ts": 2508}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2502 (Wed Jul
25 04:19:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2502 (Wed Jul 25
04:19:50
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78b671f0", "local_conf_timestamp": 2502,
"host-ts": 2502}, "global_maintenance": true}
lago.ssh: DEBUG: start task:4819a00d-a93c-4b9d-9439-77204d723a0a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:4819a00d-a93c-4b9d-9439-77204d723a0a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 00ce0cca on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 00ce0cca on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command 00ce0cca on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2508 (Wed Jul
25 04:19:55 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2508 (Wed Jul 25
04:19:56
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "ef34089b", "local_conf_timestamp": 2508, "host-ts": 2508}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2502 (Wed Jul
25 04:19:49 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2502 (Wed Jul 25
04:19:50
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78b671f0", "local_conf_timestamp": 2502,
"host-ts": 2502}, "global_maintenance": true}
lago.ssh: DEBUG: start task:2eadab93-166a-4267-8a57-0d457b253235:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:2eadab93-166a-4267-8a57-0d457b253235:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 04474b64 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 04474b64 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command 04474b64 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2518 (Wed Jul
25 04:20:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2519 (Wed Jul 25
04:20:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2512 (Wed Jul
25 04:19:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2512 (Wed Jul 25
04:19:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "569db239", "local_conf_timestamp": 2512,
"host-ts": 2512}, "global_maintenance": true}
lago.ssh: DEBUG: start task:697a24f7-bb86-4e63-abc3-0ee588786d0a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: end task:697a24f7-bb86-4e63-abc3-0ee588786d0a:Get ssh client
for lago-he-node-ng-suite-master-host-0:
lago.ssh: DEBUG: Running 07152f64 on lago-he-node-ng-suite-master-host-0:
hosted-engine --vm-status --json
lago.ssh: DEBUG: Command 07152f64 on lago-he-node-ng-suite-master-host-0
returned with 0
lago.ssh: DEBUG: Command 07152f64 on lago-he-node-ng-suite-master-host-0 output:
{"1": {"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2518 (Wed Jul
25 04:20:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2519 (Wed Jul 25
04:20:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2512 (Wed Jul
25 04:19:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2512 (Wed Jul 25
04:19:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2, "eng
ovirtlago.testlib: ERROR: * Unhandled exception in <function <lambda> at
0x7f2f900dd140>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in
assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 165, in <lambda>
for k, v in _get_he_status(host).items()
File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 106, in _get_he_status
raise RuntimeError('could not parse JSON: %s' % ret.out)
RuntimeError: could not parse JSON: {"1": {"conf_on_shared_storage": true,
"live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2518 (Wed Jul
25 04:20:06 2018)\nhost-id=1\nscore=3000\nvm_conf_refresh_time=2519 (Wed Jul 25
04:20:06
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\nmetadata_feature_version=1\ntimestamp=2512 (Wed Jul
25 04:19:59 2018)\nhost-id=2\nscore=3400\nvm_conf_refresh_time=2512 (Wed Jul 25
04:19:59
2018)\nconf_on_shared_storage=True\nmaintenance=False\nstate=GlobalMaintenance\nstopped=False\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2, "eng
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in
wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 49, in restart_he_vm
_wait_for_engine_health(host)
File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 163, in _wait_for_engine_health
testlib.assert_true_within_long(lambda: any(
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 271, in
assert_true_within_long
assert_equals_within_long(func, True, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 258, in
assert_equals_within_long
func, value, LONG_TIMEOUT, allowed_exceptions=allowed_exceptions
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in
assert_equals_within
res = func()
File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 165, in <lambda>
for k, v in _get_he_status(host).items()
File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 106, in _get_he_status
raise RuntimeError('could not parse JSON: %s' % ret.out)
'could not parse JSON: {"1": {"conf_on_shared_storage": true, "live-data":
true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2518 (Wed
Jul 25 04:20:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2519 (Wed
Jul 25 04:20:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2512 (Wed
Jul 25 04:19:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2512 (Wed
Jul 25 04:19:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"eng\n-------------------- >> begin captured logging <<
--------------------\nlago.ssh: DEBUG: start
task:932dbfdb-8f78-42c6-9b1e-6aadee5d8091:Get ssh client for
lago-he-node-ng-suite-master-host-1:\nlago.ssh: DEBUG: end
task:932dbfdb-8f78-42c6-9b1e-6aadee5d8091:Get ssh client for
lago-he-node-ng-suite-master-host-1:\nlago.ssh: DEBUG: Running 7763a99a on
lago-he-node-ng-suite-master-host-1: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 7763a99a on
lago-he-node-ng-suite-master-host-1 returned with 0\nlago.ssh: DEBUG: Command
7763a99a on lago-he-node-ng-suite-master-host-1 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2275 (Wed
Jul 25 04:16:03 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2275 (Wed
Jul 25 04:16:03
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32": "634493fe",
"local_conf_timestamp": 2275, "host-ts": 2275}, "2": {"conf_on_shared_storage":
true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2281 (Wed
Jul 25 04:16:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2281 (Wed
Jul 25 04:16:08
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "69ca08d9", "local_conf_timestamp": 2281,
"host-ts": 2281}, "global_maintenance": true}\n\nroot: INFO: * Shutting
down HE VM on host: lago-he-node-ng-suite-master-host-0\nlago.ssh: DEBUG: start
task:2bd52306-1957-43f5-a4cd-a16a4223bbd2:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:2bd52306-1957-43f5-a4cd-a16a4223bbd2:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 78525040 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-shutdown\nlago.ssh:
DEBUG: Command 78525040 on lago-he-node-ng-suite-master-host-0 returned with
0\nroot: INFO: * Command succeeded\nroot: INFO: * Waiting for VM to be
down...\nlago.ssh: DEBUG: start task:75e52da9-c741-4089-8bda-708623dbb4cc:Get
ssh client for lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:75e52da9-c741-4089-8bda-708623dbb4cc:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 797baba6 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 797baba6 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
797baba6 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2285 (Wed
Jul 25 04:16:13 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2286 (Wed
Jul 25 04:16:13
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUp\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"health": "good", "vm": "up", "detail": "Up"}, "score": 3400,
"stopped": false, "maintenance": false, "crc32": "bc8dc98f",
"local_conf_timestamp": 2286, "host-ts": 2285}, "2": {"conf_on_shared_storage":
true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2281 (Wed
Jul 25 04:16:08 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2281 (Wed
Jul 25 04:16:08
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineDown\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "69ca08d9", "local_conf_timestamp": 2281,
"host-ts": 2281}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:ac00a93a-e808-4f72-bb6f-f29c7bc259a6:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:ac00a93a-e808-4f72-bb6f-f29c7bc259a6:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 812bb044 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 812bb044 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
812bb044 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2295 (Wed
Jul 25 04:16:22 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2295 (Wed
Jul 25 04:16:23
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Powering down"}, "score": 3000, "stopped": false,
"maintenance": false, "crc32": "bf71ee59", "local_conf_timestamp": 2295,
"host-ts": 2295}, "2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2291 (Wed
Jul 25 04:16:18 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2291 (Wed
Jul 25 04:16:19
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "47e1cb10", "local_conf_timestamp": 2291,
"host-ts": 2291}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:bb5c3ebf-f315-4147-9664-83fbdfb0b5e8:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:bb5c3ebf-f315-4147-9664-83fbdfb0b5e8:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 83e14d9e on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 83e14d9e on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
83e14d9e on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2305 (Wed
Jul 25 04:16:33 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2305 (Wed
Jul 25 04:16:33
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm":
"down_unexpected", "detail": "Down"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "16c42d10", "local_conf_timestamp": 2305,
"host-ts": 2305}, "2": {"conf_on_shared_storage": true, "live-data": true,
"extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2301 (Wed
Jul 25 04:16:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2301 (Wed
Jul 25 04:16:28
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "21734107", "local_conf_timestamp": 2301,
"host-ts": 2301}, "global_maintenance": true}\n\nroot: INFO: * VM is
down.\nroot: INFO: * Stopping services...\nlago.ssh: DEBUG: start
task:a91ce735-904b-4589-8d49-374d4d4e5233:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:a91ce735-904b-4589-8d49-374d4d4e5233:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 84cdbff8 on
lago-he-node-ng-suite-master-host-0: systemctl stop vdsmd ovirt-ha-broker
ovirt-ha-agent\nlago.ssh: DEBUG: Command 84cdbff8 on
lago-he-node-ng-suite-master-host-0 returned with 0\nroot: INFO: * Starting
services...\nlago.ssh: DEBUG: start
task:69e2b63b-31fc-464e-b7ff-c499a95be86e:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:69e2b63b-31fc-464e-b7ff-c499a95be86e:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 8dac9a18 on
lago-he-node-ng-suite-master-host-0: systemctl start vdsmd ovirt-ha-broker
ovirt-ha-agent\nlago.ssh: DEBUG: Command 8dac9a18 on
lago-he-node-ng-suite-master-host-0 returned with 0\nroot: INFO: * Waiting
for agent to be ready...\nlago.ssh: DEBUG: start
task:5c41febb-2a25-4889-a7c1-5c9da7a33f96:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:5c41febb-2a25-4889-a7c1-5c9da7a33f96:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 909bffb6 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command 909bffb6 on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command 909bffb6 on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:56d0e0c8-3a30-4b6e-acf5-ba1e387ceb7c:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:56d0e0c8-3a30-4b6e-acf5-ba1e387ceb7c:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 977e0a9a on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command 977e0a9a on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command 977e0a9a on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:4b36c31c-c6b7-4bbb-b038-b67e2a26e788:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:4b36c31c-c6b7-4bbb-b038-b67e2a26e788:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 9a1513fc on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command 9a1513fc on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command 9a1513fc on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:e3e68036-c86b-4a3e-ad6d-788272025c42:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:e3e68036-c86b-4a3e-ad6d-788272025c42:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 9cbc9a94 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command 9cbc9a94 on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command 9cbc9a94 on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:7b9f8a19-b7dc-4d00-9202-e07d88888dfe:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:7b9f8a19-b7dc-4d00-9202-e07d88888dfe:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 9f7865c4 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command 9f7865c4 on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command 9f7865c4 on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:1944f900-1947-46ca-b701-3f71d3025a23:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:1944f900-1947-46ca-b701-3f71d3025a23:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running a212e07a on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command a212e07a on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command a212e07a on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:289afe1e-8a48-4e1f-b585-c7afe283cc2a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:289afe1e-8a48-4e1f-b585-c7afe283cc2a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running a4b92d66 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command a4b92d66 on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command a4b92d66 on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:65b385d4-2abe-4c8a-9f9f-37904c3320b5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:65b385d4-2abe-4c8a-9f9f-37904c3320b5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running a755a3f6 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command a755a3f6 on lago-he-node-ng-suite-master-host-0 returned with
1\nlago.ssh: DEBUG: Command a755a3f6 on lago-he-node-ng-suite-master-host-0
output:\n The hosted engine configuration has not been retrieved from shared
storage. Please ensure that ovirt-ha-agent is running and the storage server is
reachable.\n\nlago.ssh: DEBUG: start
task:179389d5-7a94-4742-9328-73c41ed07ed0:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:179389d5-7a94-4742-9328-73c41ed07ed0:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running a9fe6f48 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status\nlago.ssh:
DEBUG: Command a9fe6f48 on lago-he-node-ng-suite-master-host-0 returned with
0\nlago.ssh: DEBUG: Command a9fe6f48 on lago-he-node-ng-suite-master-host-0
output:\n \n\n!! Cluster is in GLOBAL MAINTENANCE mode !!\n\n\n\n--== Host
lago-he-node-ng-suite-master-host-0 (id: 1) status
==--\n\nconf_on_shared_storage : True\nStatus up-to-date
: True\nHostname :
lago-he-node-ng-suite-master-host-0\nHost ID :
1\nEngine status : {"reason": "bad vm status", "health":
"bad", "vm": "down", "detail": "Down"}\nScore :
0\nstopped : False\nLocal maintenance
: False\ncrc32 : 744e472a\nlocal_conf_timestamp
: 2368\nHost timestamp : 2367\nExtra metadata
(valid at
timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=2367
(Wed Jul 25 04:17:35
2018)\n\thost-id=1\n\tscore=0\n\tvm_conf_refresh_time=2368 (Wed Jul 25 04:17:35
2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=ReinitializeFSM\n\tstopped=False\n\n\n--==
Host lago-he-node-ng-suite-master-host-1 (id: 2) status
==--\n\nconf_on_shared_storage : True\nStatus up-to-date
: False\nHostname :
lago-he-node-ng-suite-master-host-1\nHost ID :
2\nEngine status : unknown stale-data\nScore
: 3400\nstopped : False\nLocal
maintenance : False\ncrc32 :
c58bc9b1\nlocal_conf_timestamp : 2361\nHost timestamp
: 2361\nExtra metadata (valid at
timestamp):\n\tmetadata_parse_version=1\n\tmetadata_feature_version=1\n\ttimestamp=2361
(Wed Jul 25 04:17:28
2018)\n\thost-id=2\n\tscore=3400\n\tvm_conf_refresh_time=2361 (Wed Jul 25
04:17:29
2018)\n\tconf_on_shared_storage=True\n\tmaintenance=False\n\tstate=GlobalMaintenance\n\tstopped=False\n\n\n!!
Cluster is in GLOBAL MAINTENANCE mode !!\n\n\nroot: INFO: * Agent is
ready.\nroot: INFO: * Starting VM...\nlago.ssh: DEBUG: start
task:771d13e3-aa83-4b0c-ad2f-73254bee44b5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:771d13e3-aa83-4b0c-ad2f-73254bee44b5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running ab02495a on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-start\nlago.ssh: DEBUG:
Command ab02495a on lago-he-node-ng-suite-master-host-0 returned with
0\nlago.ssh: DEBUG: Command ab02495a on lago-he-node-ng-suite-master-host-0
output:\n VM exists and is down, cleaning up and restarting\n\nroot: INFO:
* Command succeeded\nroot: INFO: * Waiting for VM to be UP...\nlago.ssh:
DEBUG: start task:9b23a87d-00d7-4b1b-9df3-d4f2a93a347a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:9b23a87d-00d7-4b1b-9df3-d4f2a93a347a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running ad1c960a on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command ad1c960a on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
ad1c960a on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2367 (Wed
Jul 25 04:17:35 2018)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=2368 (Wed
Jul 25 04:17:35
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=ReinitializeFSM\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down",
"detail": "Down"}, "score": 0, "stopped": false, "maintenance": false, "crc32":
"744e472a", "local_conf_timestamp": 2368, "host-ts": 2367}, "2":
{"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2361 (Wed
Jul 25 04:17:28 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2361 (Wed
Jul 25 04:17:29
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c58bc9b1", "local_conf_timestamp": 2361,
"host-ts": 2361}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:38723e85-fc65-4a4c-903e-49754f691574:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:38723e85-fc65-4a4c-903e-49754f691574:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running b40283bc on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command b40283bc on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
b40283bc on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2377 (Wed
Jul 25 04:17:44 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2377 (Wed
Jul 25 04:17:45
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "down",
"detail": "Down"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "40391805", "local_conf_timestamp": 2377, "host-ts": 2377}, "2":
{"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2381 (Wed
Jul 25 04:17:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2381 (Wed
Jul 25 04:17:49
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "8b5c590e", "local_conf_timestamp": 2381,
"host-ts": 2381}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:cd1092f6-e8c6-40ee-99d2-6a6edd7be1b5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:cd1092f6-e8c6-40ee-99d2-6a6edd7be1b5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running b6c56286 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command b6c56286 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
b6c56286 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2387 (Wed
Jul 25 04:17:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2388 (Wed
Jul 25 04:17:55
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d2ce9013", "local_conf_timestamp": 2388, "host-ts": 2387},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2381 (Wed
Jul 25 04:17:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2381 (Wed
Jul 25 04:17:49
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "8b5c590e", "local_conf_timestamp": 2381,
"host-ts": 2381}, "global_maintenance": true}\n\nroot: INFO: * VM is
UP.\nroot: INFO: * Waiting for engine to start...\nlago.ssh: DEBUG: start
task:567f9428-43b1-44f5-ad18-0f4a01c91b13:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:567f9428-43b1-44f5-ad18-0f4a01c91b13:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running b7c81b4c on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command b7c81b4c on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
b7c81b4c on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2387 (Wed
Jul 25 04:17:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2388 (Wed
Jul 25 04:17:55
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d2ce9013", "local_conf_timestamp": 2388, "host-ts": 2387},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2381 (Wed
Jul 25 04:17:48 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2381 (Wed
Jul 25 04:17:49
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "8b5c590e", "local_conf_timestamp": 2381,
"host-ts": 2381}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:029af7ab-0480-465a-ad7d-9d3a37b5b355:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:029af7ab-0480-465a-ad7d-9d3a37b5b355:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running bec6f814 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command bec6f814 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
bec6f814 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2398 (Wed
Jul 25 04:18:05 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2398 (Wed
Jul 25 04:18:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "412e4ba9", "local_conf_timestamp": 2398, "host-ts": 2398},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2391 (Wed
Jul 25 04:17:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2392 (Wed
Jul 25 04:17:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "3c7ecb7d", "local_conf_timestamp": 2392,
"host-ts": 2391}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:13363bf6-bedc-4e09-814b-bf03d9839892:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:13363bf6-bedc-4e09-814b-bf03d9839892:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running c1a71ce4 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command c1a71ce4 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
c1a71ce4 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2408 (Wed
Jul 25 04:18:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2408 (Wed
Jul 25 04:18:16
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d9432151", "local_conf_timestamp": 2408, "host-ts": 2408},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2402 (Wed
Jul 25 04:18:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2402 (Wed
Jul 25 04:18:09
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "9a202027", "local_conf_timestamp": 2402,
"host-ts": 2402}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:8e86ea8a-ed97-49c2-aa31-0d07662c480f:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:8e86ea8a-ed97-49c2-aa31-0d07662c480f:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running c4dffe62 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command c4dffe62 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
c4dffe62 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2408 (Wed
Jul 25 04:18:16 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2408 (Wed
Jul 25 04:18:16
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "d9432151", "local_conf_timestamp": 2408, "host-ts": 2408},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2402 (Wed
Jul 25 04:18:09 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2402 (Wed
Jul 25 04:18:09
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "9a202027", "local_conf_timestamp": 2402,
"host-ts": 2402}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:e801a931-3f5c-463c-b00e-07905169e39b:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:e801a931-3f5c-463c-b00e-07905169e39b:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running c86e9048 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command c86e9048 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
c86e9048 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2418 (Wed
Jul 25 04:18:26 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2419 (Wed
Jul 25 04:18:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "625c3f40", "local_conf_timestamp": 2419, "host-ts": 2418},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2412 (Wed
Jul 25 04:18:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2413 (Wed
Jul 25 04:18:20
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c30cd378", "local_conf_timestamp": 2413,
"host-ts": 2412}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:fc9d9c05-cc70-49f4-9195-5801981130da:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:fc9d9c05-cc70-49f4-9195-5801981130da:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running cb60cc6c on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command cb60cc6c on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
cb60cc6c on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2418 (Wed
Jul 25 04:18:26 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2419 (Wed
Jul 25 04:18:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "bad vm status", "health": "bad", "vm": "up",
"detail": "Powering up"}, "score": 3400, "stopped": false, "maintenance":
false, "crc32": "625c3f40", "local_conf_timestamp": 2419, "host-ts": 2418},
"2": {"conf_on_shared_storage": true, "live-data": false, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2412 (Wed
Jul 25 04:18:20 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2413 (Wed
Jul 25 04:18:20
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c30cd378", "local_conf_timestamp": 2413,
"host-ts": 2412}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:a08eb28b-2662-4ab3-a80b-77ef2581a073:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:a08eb28b-2662-4ab3-a80b-77ef2581a073:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running cee93edc on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command cee93edc on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
cee93edc on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2428 (Wed
Jul 25 04:18:35 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2428 (Wed
Jul 25 04:18:36
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "ace23939", "local_conf_timestamp": 2428, "host-ts": 2428}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2422 (Wed
Jul 25 04:18:29 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2422 (Wed
Jul 25 04:18:29
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "c677a7b5", "local_conf_timestamp": 2422,
"host-ts": 2422}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:5475c36c-7873-4bb4-b0a4-89b7a176ed10:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:5475c36c-7873-4bb4-b0a4-89b7a176ed10:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running d2517044 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command d2517044 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
d2517044 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2438 (Wed
Jul 25 04:18:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2439 (Wed
Jul 25 04:18:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "b02c5c74", "local_conf_timestamp": 2439, "host-ts": 2438}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2432 (Wed
Jul 25 04:18:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2432 (Wed
Jul 25 04:18:40
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "e85c647c", "local_conf_timestamp": 2432,
"host-ts": 2432}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:378598d4-4368-44ff-b912-b640c5a049b0:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:378598d4-4368-44ff-b912-b640c5a049b0:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running d56c33a4 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command d56c33a4 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
d56c33a4 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2438 (Wed
Jul 25 04:18:46 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2439 (Wed
Jul 25 04:18:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "b02c5c74", "local_conf_timestamp": 2439, "host-ts": 2438}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2432 (Wed
Jul 25 04:18:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2432 (Wed
Jul 25 04:18:40
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "e85c647c", "local_conf_timestamp": 2432,
"host-ts": 2432}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:7f5db09d-6b15-4582-8818-3c99d0291ce5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:7f5db09d-6b15-4582-8818-3c99d0291ce5:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running d863856c on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command d863856c on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
d863856c on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2448 (Wed
Jul 25 04:18:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2448 (Wed
Jul 25 04:18:56
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "0f24c3a2", "local_conf_timestamp": 2448, "host-ts": 2448}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2442 (Wed
Jul 25 04:18:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2442 (Wed
Jul 25 04:18:49
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "228f2f03", "local_conf_timestamp": 2442,
"host-ts": 2442}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:639271a3-0803-4020-966c-200e8adb5117:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:639271a3-0803-4020-966c-200e8adb5117:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running db6b7cf6 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command db6b7cf6 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
db6b7cf6 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2448 (Wed
Jul 25 04:18:55 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2448 (Wed
Jul 25 04:18:56
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "0f24c3a2", "local_conf_timestamp": 2448, "host-ts": 2448}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2442 (Wed
Jul 25 04:18:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2442 (Wed
Jul 25 04:18:49
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "228f2f03", "local_conf_timestamp": 2442,
"host-ts": 2442}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:d1c10391-bcf5-4501-ae6f-47a95d795283:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:d1c10391-bcf5-4501-ae6f-47a95d795283:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running de65a36e on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command de65a36e on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
de65a36e on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2458 (Wed
Jul 25 04:19:06 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2458 (Wed
Jul 25 04:19:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "64ed9679", "local_conf_timestamp": 2458, "host-ts": 2458}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2452 (Wed
Jul 25 04:18:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2452 (Wed
Jul 25 04:18:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "0ca4ecca", "local_conf_timestamp": 2452,
"host-ts": 2452}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:4e03db00-d1d4-43b0-bcfd-a21d3c32b7a7:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:4e03db00-d1d4-43b0-bcfd-a21d3c32b7a7:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running e173ed40 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command e173ed40 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
e173ed40 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2458 (Wed
Jul 25 04:19:06 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2458 (Wed
Jul 25 04:19:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "64ed9679", "local_conf_timestamp": 2458, "host-ts": 2458}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2452 (Wed
Jul 25 04:18:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2452 (Wed
Jul 25 04:18:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "0ca4ecca", "local_conf_timestamp": 2452,
"host-ts": 2452}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:d8173e8d-c1e7-49d4-8863-b1007ff2704f:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:d8173e8d-c1e7-49d4-8863-b1007ff2704f:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running e462c878 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command e462c878 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
e462c878 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2468 (Wed
Jul 25 04:19:15 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2468 (Wed
Jul 25 04:19:15
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "d8b66814", "local_conf_timestamp": 2468, "host-ts": 2468}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2462 (Wed
Jul 25 04:19:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2463 (Wed
Jul 25 04:19:10
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "09df9807", "local_conf_timestamp": 2463,
"host-ts": 2462}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:27e25ae4-3414-4640-8869-9cffad8f6814:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:27e25ae4-3414-4640-8869-9cffad8f6814:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running e794bfec on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command e794bfec on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
e794bfec on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2468 (Wed
Jul 25 04:19:15 2018)\\nhost-id=1\\nscore=3400\\nvm_conf_refresh_time=2468 (Wed
Jul 25 04:19:15
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3400, "stopped": false, "maintenance": false,
"crc32": "d8b66814", "local_conf_timestamp": 2468, "host-ts": 2468}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2462 (Wed
Jul 25 04:19:10 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2463 (Wed
Jul 25 04:19:10
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "09df9807", "local_conf_timestamp": 2463,
"host-ts": 2462}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:d23aa18c-4462-4239-8db5-c29feecac71e:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:d23aa18c-4462-4239-8db5-c29feecac71e:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running eb998bd6 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command eb998bd6 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
eb998bd6 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2478 (Wed
Jul 25 04:19:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2479 (Wed
Jul 25 04:19:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "41565544", "local_conf_timestamp": 2479, "host-ts": 2478}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2472 (Wed
Jul 25 04:19:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2472 (Wed
Jul 25 04:19:20
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "50f36b58", "local_conf_timestamp": 2472,
"host-ts": 2472}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:37e54f8a-9fe3-47cd-8896-bb560cc5e4d1:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:37e54f8a-9fe3-47cd-8896-bb560cc5e4d1:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running eed21502 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command eed21502 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
eed21502 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2478 (Wed
Jul 25 04:19:26 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2479 (Wed
Jul 25 04:19:26
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "41565544", "local_conf_timestamp": 2479, "host-ts": 2478}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2472 (Wed
Jul 25 04:19:19 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2472 (Wed
Jul 25 04:19:20
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "50f36b58", "local_conf_timestamp": 2472,
"host-ts": 2472}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:b6793d51-1124-40aa-9f42-a2107d7649cc:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:b6793d51-1124-40aa-9f42-a2107d7649cc:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running f25fec4e on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command f25fec4e on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
f25fec4e on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2488 (Wed
Jul 25 04:19:35 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2488 (Wed
Jul 25 04:19:35
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "16f668c8", "local_conf_timestamp": 2488, "host-ts": 2488}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2482 (Wed
Jul 25 04:19:30 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2483 (Wed
Jul 25 04:19:30
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "470808b8", "local_conf_timestamp": 2483,
"host-ts": 2482}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:6fb40f9a-5636-47d5-a5e7-cf1835987996:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:6fb40f9a-5636-47d5-a5e7-cf1835987996:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running f5dc5560 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command f5dc5560 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
f5dc5560 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2498 (Wed
Jul 25 04:19:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2499 (Wed
Jul 25 04:19:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "0a380d85", "local_conf_timestamp": 2499, "host-ts": 2498}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2492 (Wed
Jul 25 04:19:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2492 (Wed
Jul 25 04:19:39
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "1e24fbe7", "local_conf_timestamp": 2492,
"host-ts": 2492}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:54953ae8-9f55-4cb1-bb4c-7ec68dc9879f:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:54953ae8-9f55-4cb1-bb4c-7ec68dc9879f:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running f99d7d32 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command f99d7d32 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
f99d7d32 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2498 (Wed
Jul 25 04:19:46 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2499 (Wed
Jul 25 04:19:46
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "0a380d85", "local_conf_timestamp": 2499, "host-ts": 2498}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2492 (Wed
Jul 25 04:19:39 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2492 (Wed
Jul 25 04:19:39
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "1e24fbe7", "local_conf_timestamp": 2492,
"host-ts": 2492}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:2d207ff4-7778-45e7-abe3-56af39bec1c8:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:2d207ff4-7778-45e7-abe3-56af39bec1c8:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running fd26f32a on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command fd26f32a on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
fd26f32a on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2508 (Wed
Jul 25 04:19:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2508 (Wed
Jul 25 04:19:56
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "ef34089b", "local_conf_timestamp": 2508, "host-ts": 2508}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2502 (Wed
Jul 25 04:19:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2502 (Wed
Jul 25 04:19:50
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78b671f0", "local_conf_timestamp": 2502,
"host-ts": 2502}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:4819a00d-a93c-4b9d-9439-77204d723a0a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:4819a00d-a93c-4b9d-9439-77204d723a0a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 00ce0cca on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 00ce0cca on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
00ce0cca on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2508 (Wed
Jul 25 04:19:55 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2508 (Wed
Jul 25 04:19:56
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "ef34089b", "local_conf_timestamp": 2508, "host-ts": 2508}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2502 (Wed
Jul 25 04:19:49 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2502 (Wed
Jul 25 04:19:50
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "78b671f0", "local_conf_timestamp": 2502,
"host-ts": 2502}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:2eadab93-166a-4267-8a57-0d457b253235:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:2eadab93-166a-4267-8a57-0d457b253235:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 04474b64 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 04474b64 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
04474b64 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2518 (Wed
Jul 25 04:20:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2519 (Wed
Jul 25 04:20:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2512 (Wed
Jul 25 04:19:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2512 (Wed
Jul 25 04:19:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"engine-status": {"reason": "vm not running on this host", "health": "bad",
"vm": "down", "detail": "unknown"}, "score": 3400, "stopped": false,
"maintenance": false, "crc32": "569db239", "local_conf_timestamp": 2512,
"host-ts": 2512}, "global_maintenance": true}\n\nlago.ssh: DEBUG: start
task:697a24f7-bb86-4e63-abc3-0ee588786d0a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: end
task:697a24f7-bb86-4e63-abc3-0ee588786d0a:Get ssh client for
lago-he-node-ng-suite-master-host-0:\nlago.ssh: DEBUG: Running 07152f64 on
lago-he-node-ng-suite-master-host-0: hosted-engine --vm-status
--json\nlago.ssh: DEBUG: Command 07152f64 on
lago-he-node-ng-suite-master-host-0 returned with 0\nlago.ssh: DEBUG: Command
07152f64 on lago-he-node-ng-suite-master-host-0 output:\n {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2518 (Wed
Jul 25 04:20:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2519 (Wed
Jul 25 04:20:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2512 (Wed
Jul 25 04:19:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2512 (Wed
Jul 25 04:19:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"eng\novirtlago.testlib: ERROR: * Unhandled exception in <function <lambda>
at 0x7f2f900dd140>\nTraceback (most recent call last):\n File
"/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, in
assert_equals_within\n res = func()\n File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 165, in <lambda>\n for k, v in _get_he_status(host).items()\n File
"/home/jenkins/workspace/ovirt-system-tests_he-node-ng-suite-master/ovirt-system-tests/he-node-ng-suite-master/test-scenarios/008_restart_he_vm.py",
line 106, in _get_he_status\n raise RuntimeError(\'could not parse JSON:
%s\' % ret.out)\nRuntimeError: could not parse JSON: {"1":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2518 (Wed
Jul 25 04:20:06 2018)\\nhost-id=1\\nscore=3000\\nvm_conf_refresh_time=2519 (Wed
Jul 25 04:20:06
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-0", "host-id": 1,
"engine-status": {"reason": "failed liveliness check", "health": "bad", "vm":
"up", "detail": "Up"}, "score": 3000, "stopped": false, "maintenance": false,
"crc32": "f3fa6dd6", "local_conf_timestamp": 2519, "host-ts": 2518}, "2":
{"conf_on_shared_storage": true, "live-data": true, "extra":
"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=2512 (Wed
Jul 25 04:19:59 2018)\\nhost-id=2\\nscore=3400\\nvm_conf_refresh_time=2512 (Wed
Jul 25 04:19:59
2018)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=GlobalMaintenance\\nstopped=False\\n",
"hostname": "lago-he-node-ng-suite-master-host-1", "host-id": 2,
"eng\n--------------------- >> end captured logging << ---------------------'_______________________________________________
Infra mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/PMOIE7F46GYSTM5SN5HHXHGURD64AMCQ/
