OK after a host reboot I was able to get the Engine VM up again and into the
Web interface. Although continually when I try a second host to the datacenter
within oVirt I run into the error:
"Host mennode2 cannot access the Storage Domain(s) hosted_storage attached to
the Data Center Train1. Setting Host state to Non Operational."
Note: I can successfully read (and mount) the NFS exports oVirt is complaining
about:
[root@mennode2 ~]# showmount -e menmaster.traindemo.local
Export list for menmaster.traindemo.local:
/shares *
/shares/exports *
/shares/data *
/shares/isos *
[root@mennode2 ~]#
[root@mennode2 tmp]# mount -t nfs menmaster.traindemo.local:/shares test
[root@mennode2 tmp]# cd test
[root@mennode2 test]# ls
7d18ff24-57a3-4b4a-9934-0263191fe2e4 data __DIRECT_IO_TEST__ exports isos
[root@mennode2 test]#
One thing I DO notice is the path of my exports is the difference in the path
for the hosted_storage domain. I wonder if the second host would have issues
resolving this?
Data. ==> menmaster.traindemo.local:/shares/data
Export ==> menmaster.traindemo.local:/shares/exports
Hosted_storage. ==> localhost:/shares
ISO. ==> menmaster.traindemo.local:/shares/exports
Below I have copied exports from the VDSM and OVIRT-ENGINE log and have copied
the output of the hosted-engine.conf file. Any help in pinpointing the source
of the problem is greatly appreciated!!
VSDM logs:
2017-09-27 23:10:01,700-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call
Host.getStats succeeded in 0.06 seconds (__init__:539)
2017-09-27 23:10:03,104-0400 INFO (jsonrpc/5) [vdsm.api] START
getSpmStatus(spUUID=u'59c7f8f3-0063-00a8-02c7-0000000000f3', options=None)
from=::ffff:10.0.0.227,39748, flow_id=19e2dbb3,
task_id=7a05a5bd-9b15-43be-890d-c6f5d7650e5c (api:46)
2017-09-27 23:10:03,109-0400 INFO (jsonrpc/5) [vdsm.api] FINISH getSpmStatus
return={'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 8L}}
from=::ffff:10.0.0.227,39748, flow_id=19e2dbb3,
task_id=7a05a5bd-9b15-43be-890d-c6f5d7650e5c (api:52)
2017-09-27 23:10:03,110-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call
StoragePool.getSpmStatus succeeded in 0.01 seconds (__init__:539)
2017-09-27 23:10:03,189-0400 INFO (jsonrpc/7) [vdsm.api] START
getStoragePoolInfo(spUUID=u'59c7f8f3-0063-00a8-02c7-0000000000f3',
options=None) from=::ffff:10.0.0.227,39866, flow_id=19e2dbb3,
task_id=0f8b49e9-9d82-457e-a2a2-39dc7ed9f022 (api:46)
2017-09-27 23:10:03,196-0400 INFO (jsonrpc/7) [vdsm.api] FINISH
getStoragePoolInfo return={'info': {'name': 'No Description', 'isoprefix':
u'/rhev/data-center/mnt/menmaster.traindemo.local:_shares_isos/da001a29-eca5-44d6-a097-129dd9be623f/images/11111111-1111-1111-1111-111111111111',
'pool_status': 'connected', 'lver': 8L, 'domains':
u'da001a29-eca5-44d6-a097-129dd9be623f:Active,f36157cc-b25a-400a-ab0f-a071e8a8eea7:Active,7d18ff24-57a3-4b4a-9934-0263191fe2e4:Active,795d4a1d-3ceb-4773-99de-8e7cf05112f3:Active',
'master_uuid': u'f36157cc-b25a-400a-ab0f-a071e8a8eea7', 'version': '4',
'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo':
{u'da001a29-eca5-44d6-a097-129dd9be623f': {'status': u'Active', 'diskfree':
'1044166737920', 'isoprefix':
u'/rhev/data-center/mnt/menmaster.traindemo.local:_shares_isos/da001a29-eca5-44d6-a097-129dd9be623f/images/11111111-1111-1111-1111-111111111111',
'alerts': [], 'disktotal': '1049702170624', 'version': 0},
u'f36157cc-b25a-400a-ab0f-a071e8a8eea7': {'status': u'Active', 'diskfree':
'1044166737920', 'isoprefix': '', 'alerts': [], 'disktotal': '1049702170624',
'version': 4}, u'7d18ff24-57a3-4b4a-9934-0263191fe2e4': {'status': u'Active',
'diskfree': '1044166737920', 'isoprefix': '', 'alerts': [], 'disktotal':
'1049702170624', 'version': 4}, u'795d4a1d-3ceb-4773-99de-8e7cf05112f3':
{'status': u'Active', 'diskfree': '1044166737920', 'isoprefix': '', 'alerts':
[], 'disktotal': '1049702170624', 'version': 0}}} from=::ffff:10.0.0.227,39866,
flow_id=19e2dbb3, task_id=0f8b49e9-9d82-457e-a2a2-39dc7ed9f022 (api:52)
2017-09-27 23:10:03,198-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call
StoragePool.getInfo succeeded in 0.01 seconds (__init__:539)
2017-09-27 23:10:04,212-0400 INFO (Reactor thread)
[ProtocolDetector.AcceptorImpl] Accepted connection from ::1:35980
(protocoldetector:72)
2017-09-27 23:10:04,223-0400 INFO (Reactor thread) [ProtocolDetector.Detector]
Detected protocol stomp from ::1:35980 (protocoldetector:127)
2017-09-27 23:10:04,224-0400 INFO (Reactor thread) [Broker.StompAdapter]
Processing CONNECT request (stompreactor:103)
2017-09-27 23:10:04,225-0400 INFO (JsonRpc (StompReactor))
[Broker.StompAdapter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:04,622-0400 INFO (monitor/7d18ff2) [storage.SANLock]
Acquiring host id for domain 7d18ff24-57a3-4b4a-9934-0263191fe2e4 (id=1,
async=True) (clusterlock:288)
2017-09-27 23:10:04,623-0400 ERROR (monitor/7d18ff2) [storage.Monitor] Error
acquiring host id 1 for domain 7d18ff24-57a3-4b4a-9934-0263191fe2e4
(monitor:570)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/monitor.py", line 567, in _acquireHostId
self.domain.acquireHostId(self.hostId, async=True)
File "/usr/share/vdsm/storage/sd.py", line 785, in acquireHostId
self._manifest.acquireHostId(hostId, async)
File "/usr/share/vdsm/storage/sd.py", line 449, in acquireHostId
self._domainLock.acquireHostId(hostId, async)
File "/usr/lib/python2.7/site-packages/vdsm/storage/clusterlock.py", line
319, in acquireHostId
raise se.AcquireHostIdFailure(self._sdUUID, e)
AcquireHostIdFailure: Cannot acquire host id:
(u'7d18ff24-57a3-4b4a-9934-0263191fe2e4', SanlockException(22, 'Sanlock
lockspace add failure', 'Invalid argument'))
2017-09-27 23:10:04,647-0400 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call
Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:04,654-0400 INFO (jsonrpc/3) [vdsm.api] START
repoStats(options=None) from=::1,35980,
task_id=9732972d-9a76-4349-9c65-53e57b2472d6 (api:46)
2017-09-27 23:10:04,655-0400 INFO (jsonrpc/3) [vdsm.api] FINISH repoStats
return={u'da001a29-eca5-44d6-a097-129dd9be623f': {'code': 0, 'actual': True,
'version': 0, 'acquired': True, 'delay': '0.000357054', 'lastCheck': '4.1',
'valid': True}, u'f36157cc-b25a-400a-ab0f-a071e8a8eea7': {'code': 0, 'actual':
True, 'version': 4, 'acquired': True, 'delay': '0.000531828', 'lastCheck':
'4.0', 'valid': True}, u'7d18ff24-57a3-4b4a-9934-0263191fe2e4': {'code': 0,
'actual': True, 'version': 4, 'acquired': False, 'delay': '0.000205228',
'lastCheck': '0.0', 'valid': True}, u'795d4a1d-3ceb-4773-99de-8e7cf05112f3':
{'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000352165', 'lastCheck': '4.1', 'valid': True}} from=::1,35980,
task_id=9732972d-9a76-4349-9c65-53e57b2472d6 (api:52)
2017-09-27 23:10:04,656-0400 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call
Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:07,717-0400 INFO (Reactor thread)
[ProtocolDetector.AcceptorImpl] Accepted connection from ::1:35982
(protocoldetector:72)
2017-09-27 23:10:07,728-0400 INFO (Reactor thread) [ProtocolDetector.Detector]
Detected protocol stomp from ::1:35982 (protocoldetector:127)
2017-09-27 23:10:07,729-0400 INFO (Reactor thread) [Broker.StompAdapter]
Processing CONNECT request (stompreactor:103)
2017-09-27 23:10:07,730-0400 INFO (JsonRpc (StompReactor))
[Broker.StompAdapter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:08,152-0400 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call
Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:08,159-0400 INFO (jsonrpc/1) [vdsm.api] START
repoStats(options=None) from=::1,35982,
task_id=80eb282b-f297-4f98-a368-7d5627ac50f3 (api:46)
2017-09-27 23:10:08,160-0400 INFO (jsonrpc/1) [vdsm.api] FINISH repoStats
return={u'da001a29-eca5-44d6-a097-129dd9be623f': {'code': 0, 'actual': True,
'version': 0, 'acquired': True, 'delay': '0.000357054', 'lastCheck': '7.6',
'valid': True}, u'f36157cc-b25a-400a-ab0f-a071e8a8eea7': {'code': 0, 'actual':
True, 'version': 4, 'acquired': True, 'delay': '0.000531828', 'lastCheck':
'7.5', 'valid': True}, u'7d18ff24-57a3-4b4a-9934-0263191fe2e4': {'code': 0,
'actual': True, 'version': 4, 'acquired': False, 'delay': '0.000205228',
'lastCheck': '3.5', 'valid': True}, u'795d4a1d-3ceb-4773-99de-8e7cf05112f3':
{'code': 0, 'actual': True, 'version': 0, 'acquired': True, 'delay':
'0.000352165', 'lastCheck': '7.6', 'valid': True}} from=::1,35982,
task_id=80eb282b-f297-4f98-a368-7d5627ac50f3 (api:52)
2017-09-27 23:10:08,171-0400 INFO (Reactor thread)
[ProtocolDetector.AcceptorImpl] Accepted connection from ::1:35984
(protocoldetector:72)
2017-09-27 23:10:08,198-0400 INFO (Reactor thread) [ProtocolDetector.Detector]
Detected protocol stomp from ::1:35984 (protocoldetector:127)
2017-09-27 23:10:08,199-0400 INFO (Reactor thread) [Broker.StompAdapter]
Processing CONNECT request (stompreactor:103)
2017-09-27 23:10:08,201-0400 INFO (JsonRpc (StompReactor))
[Broker.StompAdapter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:08,374-0400 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call
Host.getStats succeeded in 0.22 seconds (__init__:539)
2017-09-27 23:10:08,728-0400 INFO (Reactor thread)
[ProtocolDetector.AcceptorImpl] Accepted connection from ::1:35987
(protocoldetector:72)
2017-09-27 23:10:08,732-0400 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call
Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:08,738-0400 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call
VM.getStats succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:08,756-0400 INFO (Reactor thread) [ProtocolDetector.Detector]
Detected protocol stomp from ::1:35987 (protocoldetector:127)
2017-09-27 23:10:08,757-0400 INFO (Reactor thread) [Broker.StompAdapter]
Processing CONNECT request (stompreactor:103)
2017-09-27 23:10:08,771-0400 INFO (JsonRpc (StompReactor))
[Broker.StompAdapter] Subscribe command received (stompreactor:132)
2017-09-27 23:10:09,286-0400 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call
Host.getHardwareInfo succeeded in 0.00 seconds (__init__:539)
2017-09-27 23:10:09,395-0400 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call
Host.getCapabilities succeeded in 0.10 seconds (__init__:539)
2017-09-27 23:10:10,590-0400 INFO (itmap/0) [IOProcessClient] Starting client
ioprocess-37 (__init__:330)
ENGINE LOGS:
2017-09-27 23:07:58,154-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Retrieving installation logs to:
'/var/log/ovirt-engine/host-deploy/ovirt-host-mgmt-20170927230758-10.0.0.222-3f898655.log'
2017-09-27 23:07:58,316-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Stage: Termination
2017-09-27 23:07:58,458-04 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(pool-7-thread-1) [3f898655] EVENT_ID: HOST_AVAILABLE_UPDATES_FINISHED(885),
Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1,
Message: Check for available updates on host mennode2 was completed
successfully with message 'no updates found.'.
2017-09-27 23:07:58,874-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Stage: Misc configuration
2017-09-27 23:07:58,888-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Stage: Transaction commit
2017-09-27 23:07:59,029-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Stage: Closing up
2017-09-27 23:07:59,039-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Stage: Pre-termination
2017-09-27 23:07:59,092-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Retrieving installation logs to:
'/var/log/ovirt-engine/host-deploy/ovirt-host-mgmt-20170927230759-menmaster.traindemo.local-3f898655.log'
2017-09-27 23:07:59,262-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [3f898655]
Stage: Termination
2017-09-27 23:07:59,375-04 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(pool-7-thread-2) [3f898655] EVENT_ID: HOST_AVAILABLE_UPDATES_FINISHED(885),
Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1,
Message: Check for available updates on host menmaster was completed
successfully with message 'no updates found.'.
2017-09-27 23:08:38,705-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
(default task-13) [75de01a6-efd2-418f-a6b1-b4ef1f38923f] Lock Acquired to
object
'EngineLock:{exclusiveLocks='[0ef7ec10-af19-4afa-980b-77a772c4e014=VDS]',
sharedLocks=''}'
2017-09-27 23:08:38,776-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
Running command: ActivateVdsCommand internal: false. Entities affected : ID:
0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDSAction group MANIPULATE_HOST with
role type ADMIN
2017-09-27 23:08:38,777-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
Before acquiring lock in order to prevent monitoring for host 'mennode2' from
data-center 'Train1'
2017-09-27 23:08:38,778-04 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
Failed to acquire lock and wait lock
'HostEngineLock:{exclusiveLocks='[0ef7ec10-af19-4afa-980b-77a772c4e014=VDS_INIT]',
sharedLocks=''}'
2017-09-27 23:08:38,995-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f] Lock
acquired, from now a monitoring of host will be skipped for host 'mennode2'
from data-center 'Train1'
2017-09-27 23:08:39,003-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
START, SetVdsStatusVDSCommand(HostName = mennode2,
SetVdsStatusVDSCommandParameters:{runAsync='true',
hostId='0ef7ec10-af19-4afa-980b-77a772c4e014', status='Unassigned',
nonOperationalReason='NONE', stopSpmFailureLogged='false',
maintenanceReason='null'}), log id: 78cc1caf
2017-09-27 23:08:39,027-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
FINISH, SetVdsStatusVDSCommand, log id: 78cc1caf
2017-09-27 23:08:39,040-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
Activate host finished. Lock released. Monitoring can run now for host
'mennode2' from data-center 'Train1'
2017-09-27 23:08:39,050-04 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f]
EVENT_ID: VDS_ACTIVATE(16), Correlation ID:
75de01a6-efd2-418f-a6b1-b4ef1f38923f, Job ID:
2e40097e-e991-45cc-bd49-3176822ebb5a, Call Stack: null, Custom ID: null, Custom
Event ID: -1, Message: Activation of host mennode2 initiated by
admin@internal-authz.
2017-09-27 23:08:39,059-04 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand]
(org.ovirt.thread.pool-6-thread-2) [75de01a6-efd2-418f-a6b1-b4ef1f38923f] Lock
freed to object
'EngineLock:{exclusiveLocks='[0ef7ec10-af19-4afa-980b-77a772c4e014=VDS]',
sharedLocks=''}'
2017-09-27 23:08:43,143-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
(DefaultQuartzScheduler7) [5437c1b7] START, GetHardwareInfoVDSCommand(HostName
= mennode2, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
hostId='0ef7ec10-af19-4afa-980b-77a772c4e014',
vds='Host[mennode2,0ef7ec10-af19-4afa-980b-77a772c4e014]'}), log id: 2bcbe26e
2017-09-27 23:08:44,150-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
(DefaultQuartzScheduler7) [5437c1b7] FINISH, GetHardwareInfoVDSCommand, log id:
2bcbe26e
2017-09-27 23:08:44,186-04 INFO
[org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand]
(DefaultQuartzScheduler7) [12ef27d8] Running command:
HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected :
ID: 0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDS
2017-09-27 23:08:44,214-04 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
(DefaultQuartzScheduler7) [63cdc728] Running command: InitVdsOnUpCommand
internal: true. Entities affected : ID: 59c7f8f3-0063-00a8-02c7-0000000000f3
Type: StoragePool
2017-09-27 23:08:44,234-04 INFO
[org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand]
(DefaultQuartzScheduler7) [7670f89e] Running command:
ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID:
59c7f8f3-0063-00a8-02c7-0000000000f3 Type: StoragePool
2017-09-27 23:08:44,257-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(DefaultQuartzScheduler7) [7670f89e] START,
ConnectStorageServerVDSCommand(HostName = mennode2,
StorageServerConnectionManagementVDSParameters:{runAsync='true',
hostId='0ef7ec10-af19-4afa-980b-77a772c4e014',
storagePoolId='59c7f8f3-0063-00a8-02c7-0000000000f3', storageType='NFS',
connectionList='[StorageServerConnections:{id='aac63433-88fb-4daf-8c83-871a85b3a49d',
connection='menmaster.traindemo.local:/shares/data', iqn='null',
vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'},
StorageServerConnections:{id='5adb7a4b-ebbd-43a4-b5cb-8c8eb3d6f1ee',
connection='localhost:/shares', iqn='null', vfsType='null',
mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null',
iface='null', netIfaceName='null'},
StorageServerConnections:{id='9eb3b1e2-4063-46cd-ba4e-440c4048e92a',
connection='menmaster.traindemo.local:/shares/isos', iqn='null',
vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'},
StorageServerConnections:{id='1c9215a1-4154-4a78-b8ff-565c3a237c58',
connection='menmaster.traindemo.local:/shares/exports', iqn='null',
vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null',
nfsTimeo='null', iface='null', netIfaceName='null'}]'}), log id: 5625b02f
2017-09-27 23:08:48,370-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler7) [7670f89e] EVENT_ID:
VDS_STORAGES_CONNECTION_FAILED(188), Correlation ID: null, Call Stack: null,
Custom ID: null, Custom Event ID: -1, Message: Failed to connect Host mennode2
to the Storage Domains hosted_storage.
2017-09-27 23:08:48,370-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(DefaultQuartzScheduler7) [7670f89e] FINISH, ConnectStorageServerVDSCommand,
return: {aac63433-88fb-4daf-8c83-871a85b3a49d=0,
5adb7a4b-ebbd-43a4-b5cb-8c8eb3d6f1ee=477,
9eb3b1e2-4063-46cd-ba4e-440c4048e92a=0,
1c9215a1-4154-4a78-b8ff-565c3a237c58=0}, log id: 5625b02f
2017-09-27 23:08:48,386-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler7) [7670f89e] EVENT_ID: STORAGE_DOMAIN_ERROR(996),
Correlation ID: null, Call Stack: null, Custom ID: null, Custom Event ID: -1,
Message: The error message for connection localhost:/shares returned by VDSM
was: Problem while trying to mount target
2017-09-27 23:08:48,389-04 ERROR
[org.ovirt.engine.core.bll.storage.connection.BaseFsStorageHelper]
(DefaultQuartzScheduler7) [7670f89e] The connection with details
'localhost:/shares' failed because of error code '477' and error message is:
problem while trying to mount target
2017-09-27 23:08:48,401-04 INFO
[org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand]
(DefaultQuartzScheduler7) [7670f89e] Host 'mennode2' storage connection was
failed
2017-09-27 23:08:48,411-04 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler7) [7670f89e] EVENT_ID:
CONNECT_STORAGE_SERVERS_FAILED(994), Correlation ID: 7670f89e, Call Stack:
null, Custom ID: null, Custom Event ID: -1, Message: Failed to connect Host
mennode2 to Storage Servers
2017-09-27 23:08:48,423-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(org.ovirt.thread.pool-6-thread-8) [] START,
ConnectStoragePoolVDSCommand(HostName = mennode2,
ConnectStoragePoolVDSCommandParameters:{runAsync='true',
hostId='0ef7ec10-af19-4afa-980b-77a772c4e014',
vdsId='0ef7ec10-af19-4afa-980b-77a772c4e014',
storagePoolId='59c7f8f3-0063-00a8-02c7-0000000000f3', masterVersion='1'}), log
id: 3d5c5d79
2017-09-27 23:08:48,423-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(org.ovirt.thread.pool-6-thread-8) [] Executing with domain map:
{7d18ff24-57a3-4b4a-9934-0263191fe2e4=active,
da001a29-eca5-44d6-a097-129dd9be623f=active,
795d4a1d-3ceb-4773-99de-8e7cf05112f3=active,
f36157cc-b25a-400a-ab0f-a071e8a8eea7=active}
2017-09-27 23:08:49,656-04 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(org.ovirt.thread.pool-6-thread-8) [] FINISH, ConnectStoragePoolVDSCommand, log
id: 3d5c5d79
2017-09-27 23:08:50,681-04 ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(org.ovirt.thread.pool-6-thread-8) [] Domain
'7d18ff24-57a3-4b4a-9934-0263191fe2e4:hosted_storage' was reported with error
code '358'
2017-09-27 23:08:50,682-04 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
(org.ovirt.thread.pool-6-thread-8) [] Storage Domain 'hosted_storage' of pool
'Train1' is in problem in host 'mennode2'
2017-09-27 23:08:50,690-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(org.ovirt.thread.pool-6-thread-8) [] EVENT_ID:
VDS_STORAGE_VDS_STATS_FAILED(189), Correlation ID: 63cdc728, Call Stack: null,
Custom ID: null, Custom Event ID: -1, Message: Host mennode2 reports about one
of the Active Storage Domains as Problematic.
2017-09-27 23:08:50,726-04 INFO
[org.ovirt.engine.core.bll.SetNonOperationalVdsCommand]
(DefaultQuartzScheduler7) [748ef73f] Running command:
SetNonOperationalVdsCommand internal: true. Entities affected : ID:
0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDS
2017-09-27 23:08:50,729-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(DefaultQuartzScheduler7) [748ef73f] START, SetVdsStatusVDSCommand(HostName =
mennode2, SetVdsStatusVDSCommandParameters:{runAsync='true',
hostId='0ef7ec10-af19-4afa-980b-77a772c4e014', status='NonOperational',
nonOperationalReason='STORAGE_DOMAIN_UNREACHABLE',
stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 2589acd2
2017-09-27 23:08:50,736-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(DefaultQuartzScheduler7) [748ef73f] FINISH, SetVdsStatusVDSCommand, log id:
2589acd2
2017-09-27 23:08:50,807-04 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler7) [748ef73f] EVENT_ID:
VDS_SET_NONOPERATIONAL_DOMAIN(522), Correlation ID: 748ef73f, Job ID:
e80941e4-3dbf-4159-8421-96d2d90160ca, Call Stack: null, Custom ID: null, Custom
Event ID: -1, Message: Host mennode2 cannot access the Storage Domain(s)
hosted_storage attached to the Data Center Train1. Setting Host state to
Non-Operational.
2017-09-27 23:08:50,820-04 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler7) [748ef73f] EVENT_ID:
VDS_ALERT_FENCE_IS_NOT_CONFIGURED(9,000), Correlation ID: 63cdc728, Call Stack:
null, Custom ID: null, Custom Event ID: -1, Message: Failed to verify Power
Management configuration for Host mennode2.
2017-09-27 23:08:50,826-04 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler7) [748ef73f] EVENT_ID:
CONNECT_STORAGE_POOL_FAILED(995), Correlation ID: 63cdc728, Call Stack: null,
Custom ID: null, Custom Event ID: -1, Message: Failed to connect Host mennode2
to Storage Pool Train1
2017-09-27 23:08:50,856-04 INFO
[org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler7)
[42cbaee1] Running command: HandleVdsVersionCommand internal: true. Entities
affected : ID: 0ef7ec10-af19-4afa-980b-77a772c4e014 Type: VDS
2017-09-27 23:08:50,863-04 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
(DefaultQuartzScheduler7) [42cbaee1] Host
'mennode2'(0ef7ec10-af19-4afa-980b-77a772c4e014) is already in NonOperational
status for reason 'STORAGE_DOMAIN_UNREACHABLE'. SetNonOperationalVds command is
skipped.
/etc/ovirt-hosted-engine/hosted-engine.conf
fqdn=engine.traindemo.local
vm_disk_id=0077cb58-7abe-4bdf-96b1-55cc85030d76
vm_disk_vol_id=d5bd2690-7cfb-4aa2-b898-41579114ec08
vmid=564d290f-ae66-4352-9770-f2e062000358
storage=menmaster.traindemo.local:/shares
mnt_options=
conf=/var/run/ovirt-hosted-engine-ha/vm.conf
host_id=1
console=vnc
domainType=nfs3
spUUID=00000000-0000-0000-0000-000000000000
sdUUID=7d18ff24-57a3-4b4a-9934-0263191fe2e4
connectionUUID=addfd5fc-8668-4d2a-9b0b-f801b21a98d8
ca_cert=/etc/pki/vdsm/libvirt-spice/ca-cert.pem
ca_subject="C=EN, L=Test, O=Test, CN=Test"
vdsm_use_ssl=true
gateway=10.0.0.1
bridge=ovirtmgmt
metadata_volume_UUID=156c7549-8e2e-4580-9825-81c3b13387ce
metadata_image_UUID=c5db06ac-ea37-47bc-9065-b94eb230616a
lockspace_volume_UUID=5f4f4a63-5608-4daf-a034-2b8e83b5de42
lockspace_image_UUID=0d7f8661-3abd-44e8-87a7-3b85f2e29fda
conf_volume_UUID=ddffd499-294c-456f-bb0e-08e366380a22
conf_image_UUID=5f264ea1-4654-4e40-87aa-80fcaacc60d8
Thanks,
Alex Witte
On Sep 27, 2017, at 8:32 PM, Alexander Witte
<alexander.wi...@baicanada.com<mailto:alexander.wi...@baicanada.com>> wrote:
/etc/ovirt-hosted-engine/hosted-engine.conf
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
