>From provided log snippets I see that engine was unable to connect to both of your hosts. Is the network OK?
In vdsm log it seems that there is an issue with you storage domain. I am not able to say anything about the issue you are facing without having more complete logs. On Thu, May 11, 2017 at 6:28 PM, Oved Ourfali <[email protected]> wrote: > What 4.0 version do you use? > Can you attach complete engine and vdsm logs? > CC-ing Piotr as well. > > > On May 11, 2017 6:44 PM, "Alexis HAUSER" <[email protected]> > wrote: > >> After rebooting the manager VM, hosts are connecting/non responsive and >> data domains inactive. Here are the engine and vdsmd logs. Any ideas ? >> >> >> >> >> Engine logs : >> >> >> >> 2017-05-11 17:28:09,302 WARN [org.ovirt.engine.core.dal.dbb >> roker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5) >> [55f1aab5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, >> Message: Failed to verify Power Management configuration for Host >> rhvserv-05. >> 2017-05-11 17:28:09,346 INFO >> [org.ovirt.engine.core.bll.HandleVdsVersionCommand] >> (DefaultQuartzScheduler5) [48bc69cd] Running command: >> HandleVdsVersionCommand internal: true. Entities affected : ID: >> 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS >> 2017-05-11 17:28:09,349 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] >> (org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not >> responding. >> 2017-05-11 17:28:09,364 WARN [org.ovirt.engine.core.dal.dbb >> roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) >> [639977e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, >> Message: Host rhvserv-05 is not responding. Host cannot be fenced >> automatically because power management for the host is disabled. >> 2017-05-11 17:28:11,299 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler3) >> [c0e6a2e] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:11,299 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:11,327 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:12,484 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3) >> [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName = rhvserv-05, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='04565f10-9abf-4709-9445-9dc6ed97e136', >> vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id: >> f807ece >> 2017-05-11 17:28:12,487 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3) >> [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log id: f807ece >> 2017-05-11 17:28:12,532 INFO [org.ovirt.engine.core.bll.Han >> dleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler3) >> [4e882ea0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand >> internal: true. Entities affected : ID: 04565f10-9abf-4709-9445-9dc6ed97e136 >> Type: VDS >> 2017-05-11 17:28:12,539 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] >> (DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand >> internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-000000000023 >> Type: StoragePool >> 2017-05-11 17:28:12,545 INFO [org.ovirt.engine.core.bll.sto >> rage.pool.ConnectHostToStoragePoolServersCommand] >> (DefaultQuartzScheduler3) [46cc3f58] Running command: >> ConnectHostToStoragePoolServersCommand internal: true. Entities affected >> : ID: 58f8df36-019f-02bc-00e7-000000000023 Type: StoragePool >> 2017-05-11 17:28:12,556 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) >> [46cc3f58] START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, >> StorageServerConnectionManagementVDSParameters:{runAsync='true', >> hostId='04565f10-9abf-4709-9445-9dc6ed97e136', >> storagePoolId='58f8df36-019f-02bc-00e7-000000000023', >> storageType='ISCSI', connectionList='[StorageServer >> Connections:{id='10c0528b-f08d-4d1d-8c63-8a05fd9d58b9', >> connection='10.35.21.1', iqn='iqn.1984-05.com.dell:powe >> rvault.md3200i.6782bcb00073e332000000004edde164', vfsType='null', >> mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', >> iface='null', netIfaceName='null'}]'}), log id: 1beb27b6 >> 2017-05-11 17:28:13,031 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) >> [46cc3f58] FINISH, ConnectStorageServerVDSCommand, return: >> {10c0528b-f08d-4d1d-8c63-8a05fd9d58b9=0}, log id: 1beb27b6 >> 2017-05-11 17:28:13,032 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) >> [46cc3f58] START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, >> StorageServerConnectionManagementVDSParameters:{runAsync='true', >> hostId='04565f10-9abf-4709-9445-9dc6ed97e136', >> storagePoolId='58f8df36-019f-02bc-00e7-000000000023', storageType='NFS', >> connectionList='[StorageServerConnections:{id='e604d0d2-0810-4c25-b9ed-610f9923cb1a', >> connection='nfsserv-01:/nfs/export', iqn='null', vfsType='null', >> mountOptions='null', nfsVersion='V3', nfsRetrans='null', nfsTimeo='null', >> iface='null', netIfaceName='null'}, StorageServerConnections:{id=' >> 2c1aba4b-786b-4c76-a02f-d9e09e8afff4', connection='nfsserv-01:/nfs/iso', >> iqn='null', vfsType='null', mountOptions='null', nfsVersion='V3', >> nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]'}), >> log id: 15c54cd7 >> 2017-05-11 17:28:14,301 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:14,304 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler1) >> [62006eaa] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:14,304 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler1) [62006eaa] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:14,334 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler7) >> [1efe65a7] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:14,334 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler7) [1efe65a7] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:17,308 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:17,336 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:17,340 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:17,340 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-04' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:20,315 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler1) >> [62006eaa] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:20,315 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler1) [62006eaa] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:20,343 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:23,316 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:23,325 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:23,325 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:23,348 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler7) >> [1efe65a7] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:23,348 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler7) [1efe65a7] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:26,329 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:26,350 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:26,354 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler7) >> [1efe65a7] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:26,354 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler7) [1efe65a7] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:29,335 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:29,335 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:29,356 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:32,337 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:32,340 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler10) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:32,340 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler10) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:32,362 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler9) >> [4cf3eed0] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:32,362 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler9) [4cf3eed0] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:35,342 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:35,370 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:38,349 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler7) >> [1efe65a7] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:38,349 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler7) [1efe65a7] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:38,374 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:38,375 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-04' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:41,352 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:41,354 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler9) >> [4cf3eed0] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:41,354 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler9) [4cf3eed0] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:41,377 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:41,380 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler7) >> [1efe65a7] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:41,380 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler7) [1efe65a7] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:44,358 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:44,382 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:47,362 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler1) >> [62006eaa] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:47,363 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler1) [62006eaa] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:47,388 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler7) >> [1efe65a7] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:47,388 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler7) [1efe65a7] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:50,365 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:50,368 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler5) >> [48bc69cd] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:50,368 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler5) [48bc69cd] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:50,390 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:50,394 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler2) >> [590af365] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:50,394 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler2) [590af365] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:53,371 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:53,396 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:56,379 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler10) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:56,379 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler10) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:56,402 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler5) >> [48bc69cd] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:56,403 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler5) [48bc69cd] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:56,488 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetAllVmStatsVDSCommand] (DefaultQuartzScheduler6) [] >> Command 'GetAllVmStatsVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: VDSGenericException: VDSNetworkException: Vds timeout occured >> 2017-05-11 17:28:56,488 INFO [org.ovirt.engine.core.vdsbrok >> er.monitoring.PollVmStatsRefresher] (DefaultQuartzScheduler6) [] Failed >> to fetch vms info for host 'rhvserv-04' - skipping VMs monitoring. >> 2017-05-11 17:28:59,382 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:28:59,384 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler10) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:59,384 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler10) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:59,404 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:59,408 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler6) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:59,408 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler6) [] Failure to refresh host 'rhvserv-04' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:02,386 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:29:02,409 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:29:02,490 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetAllVmStatsVDSCommand] (DefaultQuartzScheduler8) >> [56fb4c82] Command 'GetAllVmStatsVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: VDSGenericException: VDSNetworkException: Vds timeout occured >> 2017-05-11 17:29:02,490 INFO [org.ovirt.engine.core.vdsbrok >> er.monitoring.PollVmStatsRefresher] (DefaultQuartzScheduler8) [56fb4c82] >> Failed to fetch vms info for host 'rhvserv-03' - skipping VMs monitoring. >> 2017-05-11 17:29:05,393 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:05,393 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:05,415 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler10) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:05,415 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler10) [] Failure to refresh host 'rhvserv-04' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:08,395 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:29:08,398 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler8) >> [56fb4c82] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:08,398 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler8) [56fb4c82] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:08,417 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:29:08,420 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler2) >> [590af365] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:08,420 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler2) [590af365] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:11,402 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:29:11,423 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:29:14,409 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:14,409 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:14,430 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler5) >> [48bc69cd] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:14,430 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler5) [48bc69cd] Failure to refresh host 'rhvserv-04' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:17,411 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> 2017-05-11 17:29:17,414 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler6) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:17,414 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler6) [] Failure to refresh host 'rhvserv-03' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:17,432 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:29:17,436 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler4) [] >> Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-04, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='0d0cd690-b64a-42bb-a167-43fcedd634e4', >> vds='Host[rhvserv-04,0d0cd690-b64a-42bb-a167-43fcedd634e4]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:17,436 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler4) [] Failure to refresh host 'rhvserv-04' runtime >> info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:29:17,491 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-03.mydomain.com/192.16 >> 8.93.213 >> >> >> vdsmd logs >> >> >> Thread-113::ERROR::2017-05-11 17:41:33,734::sdc::146::Storag >> e.StorageDomainCache::(_findDomain) domain >> 5b978dda-d1ef-46fe-9996-20aee42cf303 >> not found >> Traceback (most recent call last): >> File "/usr/share/vdsm/storage/sdc.py", line 144, in _findDomain >> dom = findMethod(sdUUID) >> File "/usr/share/vdsm/storage/sdc.py", line 174, in >> _findUnfetchedDomain >> raise se.StorageDomainDoesNotExist(sdUUID) >> StorageDomainDoesNotExist: Storage domain does not exist: >> (u'5b978dda-d1ef-46fe-9996-20aee42cf303',) >> Thread-113::ERROR::2017-05-11 >> 17:41:33,735::monitor::328::Storage.Monitor::(_setupLoop) >> Setting up monitor for 5b978dda-d1ef-46fe-9996-20aee42cf303 failed >> Traceback (most recent call last): >> File "/usr/share/vdsm/storage/monitor.py", line 325, in _setupLoop >> self._setupMonitor() >> File "/usr/share/vdsm/storage/monitor.py", line 348, in _setupMonitor >> self._produceDomain() >> File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 405, in >> wrapper >> value = meth(self, *a, **kw) >> File "/usr/share/vdsm/storage/monitor.py", line 366, in _produceDomain >> self.domain = sdCache.produce(self.sdUUID) >> File "/usr/share/vdsm/storage/sdc.py", line 101, in produce >> domain.getRealDomain() >> File "/usr/share/vdsm/storage/sdc.py", line 53, in getRealDomain >> return self._cache._realProduce(self._sdUUID) >> File "/usr/share/vdsm/storage/sdc.py", line 125, in _realProduce >> domain = self._findDomain(sdUUID) >> File "/usr/share/vdsm/storage/sdc.py", line 144, in _findDomain >> dom = findMethod(sdUUID) >> File "/usr/share/vdsm/storage/sdc.py", line 174, in >> _findUnfetchedDomain >> raise se.StorageDomainDoesNotExist(sdUUID) >> StorageDomainDoesNotExist: Storage domain does not exist: >> (u'5b978dda-d1ef-46fe-9996-20aee42cf303',) >> Thread-112::DEBUG::2017-05-11 >> 17:41:33,783::lvm::288::Storage.Misc.excCmd::(cmd) >> SUCCESS: <err> = ''; <rc> = 0 >> Thread-112::INFO::2017-05-11 17:41:33,783::sd::604::Storage >> .StorageDomain::(_registerResourceNamespaces) Resource namespace >> edd229cb-b72f-4988-8c10-d83c84ef4a8a_imageNS already registered >> Thread-112::INFO::2017-05-11 17:41:33,783::sd::612::Storage >> .StorageDomain::(_registerResourceNamespaces) Resource namespace >> edd229cb-b72f-4988-8c10-d83c84ef4a8a_volumeNS already registered >> Thread-112::INFO::2017-05-11 17:41:33,783::blockSD::846::St >> orage.StorageDomain::(_registerResourceNamespaces) Resource namespace >> edd229cb-b72f-4988-8c10-d83c84ef4a8a_lvmActivationNS already registered >> Thread-112::DEBUG::2017-05-11 >> 17:41:33,784::lvm::288::Storage.Misc.excCmd::(cmd) >> /usr/bin/taskset --cpu-list 0-15 /usr/bin/sudo -n /usr/sbin/lvm vgck >> --config ' devices { preferred_names = ["^/dev/mapper/"] >> ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 >> filter = [ '\''a|/dev/mapper/36782bcb00073e33200002d3858fa1a81|'\'', >> '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 >> wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = >> 0 } ' edd229cb-b72f-4988-8c10-d83c84ef4a8a (cwd None) >> jsonrpc.Executor/6::DEBUG::2017-05-11 >> 17:41:33,792::lvm::288::Storage.Misc.excCmd::(cmd) >> SUCCESS: <err> = ''; <rc> = 0 >> jsonrpc.Executor/6::INFO::2017-05-11 >> 17:41:33,792::logUtils::52::dispatcher::(wrapper) >> Run and protect: getStorageDomainStats, Return response: {'stats': >> {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': True, >> 'diskfree': '498484641792', 'disktotal': '557943095296', 'mdafree': >> '67102208'}} >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::task::1193::Stor >> age.TaskManager.Task::(prepare) >> Task=`234cbec4-a422-47c1-a50a-243526aeddc7`::finished: >> {'stats': {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': True, >> 'diskfree': '498484641792', 'disktotal': '557943095296', 'mdafree': >> '67102208'}} >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::task::597::Stora >> ge.TaskManager.Task::(_updateState) >> Task=`234cbec4-a422-47c1-a50a-243526aeddc7`::moving >> from state preparing -> state finished >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::resourceManager: >> :952::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll >> requests {} resources {u'Storage.595de2cf-89ba-407b-aae5-d0a7a0656ba1': >> < ResourceRef 'Storage.595de2cf-89ba-407b-aae5-d0a7a0656ba1', isValid: >> 'True' obj: 'None'>} >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::resourceManager: >> :989::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll >> requests {} >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::resourceManager: >> :628::Storage.ResourceManager::(releaseResource) Trying to release >> resource 'Storage.595de2cf-89ba-407b-aae5-d0a7a0656ba1' >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::resourceManager: >> :647::Storage.ResourceManager::(releaseResource) Released resource >> 'Storage.595de2cf-89ba-407b-aae5-d0a7a0656ba1' (0 active users) >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::resourceManager: >> :653::Storage.ResourceManager::(releaseResource) Resource >> 'Storage.595de2cf-89ba-407b-aae5-d0a7a0656ba1' is free, finding out if >> anyone is waiting for it. >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,793::resourceManager: >> :661::Storage.ResourceManager::(releaseResource) No one is waiting for >> resource 'Storage.595de2cf-89ba-407b-aae5-d0a7a0656ba1', Clearing >> records. >> jsonrpc.Executor/6::DEBUG::2017-05-11 >> 17:41:33,793::task::995::Storage.TaskManager.Task::(_decref) >> Task=`234cbec4-a422-47c1-a50a-243526aeddc7`::ref 0 aborting False >> jsonrpc.Executor/6::DEBUG::2017-05-11 17:41:33,794::__init__::555::j >> sonrpc.JsonRpcServer::(_handle_request) Return 'StorageDomain.getStats' >> in bridge with {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': >> True, 'diskfree': '498484641792', 'disktotal': '557943095296', 'mdafree': >> '67102208'} >> jsonrpc.Executor/6::INFO::2017-05-11 17:41:33,794::__init__::513::j >> sonrpc.JsonRpcServer::(_serveRequest) RPC call StorageDomain.getStats >> succeeded in 1.05 seconds >> JsonRpc (StompReactor)::ERROR::2017-05-11 >> 17:41:33,816::betterAsyncore::113::vds.dispatcher::(recv) >> SSL error during reading data: unexpected eof >> Thread-112::DEBUG::2017-05-11 >> 17:41:33,846::lvm::288::Storage.Misc.excCmd::(cmd) >> SUCCESS: <err> = ''; <rc> = 0 >> Thread-12::DEBUG::2017-05-11 >> 17:41:34,913::check::296::storage.check::(_start_process) >> START check '/dev/595de2cf-89ba-407b-aae5-d0a7a0656ba1/metadata' >> cmd=['/usr/bin/taskset', '--cpu-list', '0-15', '/usr/bin/dd', >> 'if=/dev/595de2cf-89ba-407b-aae5-d0a7a0656ba1/metadata', 'of=/dev/null', >> 'bs=4096', 'count=1', 'iflag=direct'] delay=0.00 >> Thread-12::DEBUG::2017-05-11 >> 17:41:34,954::asyncevent::564::storage.asyncevent::(reap) >> Process <cpopen.CPopen object at 0x1b15b90> terminated (count=1) >> Thread-12::DEBUG::2017-05-11 >> 17:41:34,954::check::327::storage.check::(_check_completed) >> FINISH check '/dev/595de2cf-89ba-407b-aae5-d0a7a0656ba1/metadata' rc=0 >> err=bytearray(b'1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) >> copied, 0.000537649 s, 7.6 MB/s\n') elapsed=0.05 >> Reactor thread::INFO::2017-05-11 17:41:35,823::protocoldetector >> ::76::ProtocolDetector.AcceptorImpl::(handle_accept) Accepted connection >> from ::1:49348 >> Reactor thread::DEBUG::2017-05-11 17:41:35,827::protocoldetector >> ::92::ProtocolDetector.Detector::(__init__) Using required_size=11 >> Reactor thread::INFO::2017-05-11 17:41:35,828::protocoldetector >> ::128::ProtocolDetector.Detector::(handle_read) Detected protocol stomp >> from ::1:49348 >> Reactor thread::INFO::2017-05-11 17:41:35,828::stompreactor::10 >> 1::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >> Reactor thread::DEBUG::2017-05-11 17:41:35,829::stompreactor::49 >> 2::protocoldetector.StompDetector::(handle_socket) Stomp detected from >> ('::1', 49348) >> JsonRpc (StompReactor)::INFO::2017-05-11 17:41:35,829::stompreactor::12 >> 8::Broker.StompAdapter::(_cmd_subscribe) Subscribe command received >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.ovirt.org/mailman/listinfo/users >> >>
_______________________________________________ Users mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/users
