Re: [ovirt-users] 4.0 : hosts connecting/non responsive and data domains inactive
>From provided log snippets I see that engine was unable to connect to both of your hosts. Is the network OK? In vdsm log it seems that there is an issue with you storage domain. I am not able to say anything about the issue you are facing without having more complete logs. On Thu, May 11, 2017 at 6:28 PM, Oved Ourfali wrote: > What 4.0 version do you use? > Can you attach complete engine and vdsm logs? > CC-ing Piotr as well. > > > On May 11, 2017 6:44 PM, "Alexis HAUSER" > wrote: > >> After rebooting the manager VM, hosts are connecting/non responsive and >> data domains inactive. Here are the engine and vdsmd logs. Any ideas ? >> >> >> >> >> Engine logs : >> >> >> >> 2017-05-11 17:28:09,302 WARN [org.ovirt.engine.core.dal.dbb >> roker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5) >> [55f1aab5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, >> Message: Failed to verify Power Management configuration for Host >> rhvserv-05. >> 2017-05-11 17:28:09,346 INFO >> [org.ovirt.engine.core.bll.HandleVdsVersionCommand] >> (DefaultQuartzScheduler5) [48bc69cd] Running command: >> HandleVdsVersionCommand internal: true. Entities affected : ID: >> 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS >> 2017-05-11 17:28:09,349 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] >> (org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not >> responding. >> 2017-05-11 17:28:09,364 WARN [org.ovirt.engine.core.dal.dbb >> roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) >> [639977e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, >> Message: Host rhvserv-05 is not responding. Host cannot be fenced >> automatically because power management for the host is disabled. >> 2017-05-11 17:28:11,299 ERROR [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler3) >> [c0e6a2e] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', >> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution >> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:11,299 ERROR >> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] >> (DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03' >> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: >> Connection failed >> 2017-05-11 17:28:11,327 INFO >> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] >> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16 >> 8.93.214 >> 2017-05-11 17:28:12,484 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3) >> [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName = rhvserv-05, >> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', >> hostId='04565f10-9abf-4709-9445-9dc6ed97e136', >> vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id: >> f807ece >> 2017-05-11 17:28:12,487 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3) >> [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log id: f807ece >> 2017-05-11 17:28:12,532 INFO [org.ovirt.engine.core.bll.Han >> dleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler3) >> [4e882ea0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand >> internal: true. Entities affected : ID: 04565f10-9abf-4709-9445-9dc6ed97e136 >> Type: VDS >> 2017-05-11 17:28:12,539 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] >> (DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand >> internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-0023 >> Type: StoragePool >> 2017-05-11 17:28:12,545 INFO [org.ovirt.engine.core.bll.sto >> rage.pool.ConnectHostToStoragePoolServersCommand] >> (DefaultQuartzScheduler3) [46cc3f58] Running command: >> ConnectHostToStoragePoolServersCommand internal: true. Entities affected >> : ID: 58f8df36-019f-02bc-00e7-0023 Type: StoragePool >> 2017-05-11 17:28:12,556 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) >> [46cc3f58] START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, >> StorageServerConnectionManagementVDSParameters:{runAsync='true', >> hostId='04565f10-9abf-4709-9445-9dc6ed97e136', >> storagePoolId='58f8df36-019f-02bc-00e7-0023', >> storageType='ISCSI', connectionList='[StorageServer >> Connections:{id='10c0528b-f08d-4d1d-8c63-8a05fd9d58b9', >> connection='10.35.21.1', iqn='iqn.1984-05.com.dell:powe >> rvault.md3200i.6782bcb00073e3324edde164', vfsType='null', >> mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', >> iface='null', netIfaceName='null'}]'}), log id: 1beb27b6 >> 2017-05-11 17:28:13,031 INFO [org.ovirt.engine.core.vdsbrok >> er.vdsbroker.Conne
Re: [ovirt-users] 4.0 : hosts connecting/non responsive and data domains inactive
What 4.0 version do you use? Can you attach complete engine and vdsm logs? CC-ing Piotr as well. On May 11, 2017 6:44 PM, "Alexis HAUSER" wrote: > After rebooting the manager VM, hosts are connecting/non responsive and > data domains inactive. Here are the engine and vdsmd logs. Any ideas ? > > > > > Engine logs : > > > > 2017-05-11 17:28:09,302 WARN [org.ovirt.engine.core.dal. > dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5) > [55f1aab5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, > Message: Failed to verify Power Management configuration for Host > rhvserv-05. > 2017-05-11 17:28:09,346 INFO > [org.ovirt.engine.core.bll.HandleVdsVersionCommand] > (DefaultQuartzScheduler5) [48bc69cd] Running command: > HandleVdsVersionCommand internal: true. Entities affected : ID: > 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS > 2017-05-11 17:28:09,349 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] > (org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not > responding. > 2017-05-11 17:28:09,364 WARN [org.ovirt.engine.core.dal. > dbbroker.auditloghandling.AuditLogDirector] > (org.ovirt.thread.pool-6-thread-27) > [639977e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, > Message: Host rhvserv-05 is not responding. Host cannot be fenced > automatically because power management for the host is disabled. > 2017-05-11 17:28:11,299 ERROR > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] > (DefaultQuartzScheduler3) [c0e6a2e] Command > 'GetCapabilitiesVDSCommand(HostName > = rhvserv-03, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', > hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', > vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution > failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: > Connection failed > 2017-05-11 17:28:11,299 ERROR > [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] > (DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03' > runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: > Connection failed > 2017-05-11 17:28:11,327 INFO > [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] > (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192. > 168.93.214 > 2017-05-11 17:28:12,484 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] > (DefaultQuartzScheduler3) [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName > = rhvserv-05, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', > hostId='04565f10-9abf-4709-9445-9dc6ed97e136', > vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id: > f807ece > 2017-05-11 17:28:12,487 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] > (DefaultQuartzScheduler3) [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log > id: f807ece > 2017-05-11 17:28:12,532 INFO [org.ovirt.engine.core.bll. > HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler3) > [4e882ea0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand > internal: true. Entities affected : ID: 04565f10-9abf-4709-9445-9dc6ed97e136 > Type: VDS > 2017-05-11 17:28:12,539 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] > (DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand > internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-0023 > Type: StoragePool > 2017-05-11 17:28:12,545 INFO [org.ovirt.engine.core.bll.storage.pool. > ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler3) > [46cc3f58] Running command: ConnectHostToStoragePoolServersCommand > internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-0023 > Type: StoragePool > 2017-05-11 17:28:12,556 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker. > ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58] > START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, > StorageServerConnectionManagementVDSParameters:{runAsync='true', > hostId='04565f10-9abf-4709-9445-9dc6ed97e136', > storagePoolId='58f8df36-019f-02bc-00e7-0023', > storageType='ISCSI', connectionList='[StorageServerConnections:{id=' > 10c0528b-f08d-4d1d-8c63-8a05fd9d58b9', connection='10.35.21.1', > iqn='iqn.1984-05.com.dell:powervault.md3200i. > 6782bcb00073e3324edde164', vfsType='null', mountOptions='null', > nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', > netIfaceName='null'}]'}), log id: 1beb27b6 > 2017-05-11 17:28:13,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker. > ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58] > FINISH, ConnectStorageServerVDSCommand, return: > {10c0528b-f08d-4d1d-8c63-8a05fd9d58b9=0}, > log id: 1beb27b6 > 2017-05-11 17:28:13,032 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker. > ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58] > START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, > StorageServerConnectionManageme
[ovirt-users] 4.0 : hosts connecting/non responsive and data domains inactive
After rebooting the manager VM, hosts are connecting/non responsive and data domains inactive. Here are the engine and vdsmd logs. Any ideas ? Engine logs : 2017-05-11 17:28:09,302 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5) [55f1aab5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rhvserv-05. 2017-05-11 17:28:09,346 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler5) [48bc69cd] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS 2017-05-11 17:28:09,349 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not responding. 2017-05-11 17:28:09,364 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) [639977e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rhvserv-05 is not responding. Host cannot be fenced automatically because power management for the host is disabled. 2017-05-11 17:28:11,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler3) [c0e6a2e] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed 2017-05-11 17:28:11,299 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03' runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed 2017-05-11 17:28:11,327 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.168.93.214 2017-05-11 17:28:12,484 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3) [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName = rhvserv-05, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='04565f10-9abf-4709-9445-9dc6ed97e136', vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id: f807ece 2017-05-11 17:28:12,487 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3) [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log id: f807ece 2017-05-11 17:28:12,532 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler3) [4e882ea0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS 2017-05-11 17:28:12,539 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-0023 Type: StoragePool 2017-05-11 17:28:12,545 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler3) [46cc3f58] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-0023 Type: StoragePool 2017-05-11 17:28:12,556 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58] START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, StorageServerConnectionManagementVDSParameters:{runAsync='true', hostId='04565f10-9abf-4709-9445-9dc6ed97e136', storagePoolId='58f8df36-019f-02bc-00e7-0023', storageType='ISCSI', connectionList='[StorageServerConnections:{id='10c0528b-f08d-4d1d-8c63-8a05fd9d58b9', connection='10.35.21.1', iqn='iqn.1984-05.com.dell:powervault.md3200i.6782bcb00073e3324edde164', vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]'}), log id: 1beb27b6 2017-05-11 17:28:13,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58] FINISH, ConnectStorageServerVDSCommand, return: {10c0528b-f08d-4d1d-8c63-8a05fd9d58b9=0}, log id: 1beb27b6 2017-05-11 17:28:13,032 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58] START, ConnectStorageServerVDSCommand(HostName = rhvserv-05, StorageServerConnectionManagementVDSParameters:{runAsync='true', hostId='04565f10-9abf-4709-9445-9dc6ed97e136', storagePoolId='58f8df36-019f-02bc-00e7-0023', storageType='NFS', connectionList='[StorageServerConnections:{id='e604d0d2-0810-4c25-b9ed-610f9923cb1a', connection='nfsserv-01:/nfs/export', iqn