Hi Mario,
Please open a bug for this.
Include these logs in the bug for the ovirt engine host, one hypervisor node
that
had no trouble, and one hypervisor node that had trouble (ovirt-node01?).
/var/log/mesages
/var/log/sanlock.log
/var/log/vdsm.log
And of course engine.log for the engine node.
Thanks,
Nir
----- Original Message -----
> From: "Ml Ml" <mliebher...@googlemail.com>
> To: "Sandro Bonazzola" <sbona...@redhat.com>
> Cc: "Matt ." <yamakasi....@gmail.com>, users@ovirt.org, "Dan Kenigsberg"
> <dan...@redhat.com>, "Nir Soffer"
> <nsof...@redhat.com>
> Sent: Wednesday, November 12, 2014 5:18:56 PM
> Subject: Re: [ovirt-users] replace ovirt engine host
>
> Here is the vdsm log of my ovirt-node01:
>
>
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::ERROR::2014-11-12
> 16:13:20,071::sp::330::Storage.StoragePool::(startSpm) failed: Storage
> domain does not exist: ('6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1',)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,071::sp::336::Storage.StoragePool::(_shutDownUpgrade)
> Shutting down upgrade process
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,071::resourceManager::198::ResourceManager.Request::(__init__)
> ResName=`Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d`ReqID=`7ec0dd55-0b56-4d8a-bc21-5aa6fe2ec373`::Request
> was made in '/usr/share/vdsm/storage/sp.py' line '338' at
> '_shutDownUpgrade'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,071::resourceManager::542::ResourceManager::(registerResource)
> Trying to register resource
> 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' for lock type
> 'exclusive'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,072::resourceManager::601::ResourceManager::(registerResource)
> Resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' is
> free. Now locking as 'exclusive' (1 active user)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,072::resourceManager::238::ResourceManager.Request::(grant)
> ResName=`Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d`ReqID=`7ec0dd55-0b56-4d8a-bc21-5aa6fe2ec373`::Granted
> request
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,072::resourceManager::198::ResourceManager.Request::(__init__)
> ResName=`Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1`ReqID=`a6bd57b0-5ac0-459a-a4c2-2a5a58c4b1ea`::Request
> was made in '/usr/share/vdsm/storage/sp.py' line '358' at
> '_shutDownUpgrade'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,073::resourceManager::542::ResourceManager::(registerResource)
> Trying to register resource
> 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' for lock type
> 'exclusive'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,073::resourceManager::601::ResourceManager::(registerResource)
> Resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' is
> free. Now locking as 'exclusive' (1 active user)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,073::resourceManager::238::ResourceManager.Request::(grant)
> ResName=`Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1`ReqID=`a6bd57b0-5ac0-459a-a4c2-2a5a58c4b1ea`::Granted
> request
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,073::resourceManager::616::ResourceManager::(releaseResource)
> Trying to release resource
> 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,073::resourceManager::635::ResourceManager::(releaseResource)
> Released resource
> 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' (0 active
> users)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,074::resourceManager::641::ResourceManager::(releaseResource)
> Resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' is
> free, finding out if anyone is waiting for it.
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,074::resourceManager::649::ResourceManager::(releaseResource)
> No one is waiting for resource
> 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1', Clearing
> records.
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,074::resourceManager::616::ResourceManager::(releaseResource)
> Trying to release resource
> 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,074::resourceManager::635::ResourceManager::(releaseResource)
> Released resource
> 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' (0 active
> users)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,075::resourceManager::641::ResourceManager::(releaseResource)
> Resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' is
> free, finding out if anyone is waiting for it.
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,075::resourceManager::649::ResourceManager::(releaseResource)
> No one is waiting for resource
> 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d', Clearing
> records.
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,075::persistentDict::167::Storage.PersistentDict::(transaction)
> Starting transaction
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,075::persistentDict::173::Storage.PersistentDict::(transaction)
> Flushing changes
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,076::persistentDict::299::Storage.PersistentDict::(flush)
> about to write lines (FileMetadataRW)=['CLASS=Data',
> 'DESCRIPTION=RaidVolBGluster', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3',
> 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5',
> 'MASTER_VERSION=1', 'POOL_DESCRIPTION=HP_Proliant_DL18
> 0G6',
> 'POOL_DOMAINS=6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1:Active,abc51e26-7175-4b38-b3a8-95c6928fbc2b:Active',
> 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0',
> 'POOL_UUID=b384b3da-02a6-44f3-a3f6-56751ce8c26d',
> 'REMOTE_PATH=127.0.0.1:/RaidVolB', 'ROLE=Master',
> 'SDUUID=abc51e26-7175-4b38-b3a8-95c6928fbc2b', 'TYPE=GLUSTERFS',
> 'VERSION=3', '_SHA_CKSUM=9b444340971e2506b55bfe1d4
> a662fde62adbeaa']
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,082::persistentDict::175::Storage.PersistentDict::(transaction)
> Finished transaction
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::INFO::2014-11-12
> 16:13:20,082::clusterlock::279::SANLock::(release) Releasing cluster
> lock for domain abc51e26-7175-4b38-b3a8-95c6928fbc2b
> Thread-28::DEBUG::2014-11-12
> 16:13:20,270::BindingXMLRPC::1067::vds::(wrapper) client
> [192.168.150.8]::call volumesList with () {} flowID [58a6ac1e]
> Thread-28::DEBUG::2014-11-12
> 16:13:20,403::BindingXMLRPC::1074::vds::(wrapper) return volumesList
> with {'status': {'message': 'Done', 'code': 0}, 'volumes':
> {'RaidVolB': {'transportType': ['TCP'], 'uuid':
> 'd46619e9-9368-4e82-bf3a-a2377b6e85e4', 'bricks':
> ['ovirt-node01.foobar.net:/raidvol/volb',
> 'ovirt-node02.foobar.net:/raidvol/volb'], 'volume
> Name': 'RaidVolB', 'volumeType': 'REPLICATE', 'replicaCount': '2',
> 'brickCount': '2', 'distCount': '2', 'volumeStatus': 'ONLINE',
> 'stripeCount': '1', 'bricksInfo': [{'name':
> 'ovirt-node01.foobar.net:/raidvol/volb', 'hostUuid':
> 'de2a515f-c728-449d-b91c-d80cabe0539f'}, {'name':
> 'ovirt-node02.foobar.net:/raidvol/volb', 'hostUuid': '7540f5c0-c4ba-4
> 520-bdf1-3115c10d0eea'}], 'options': {'user.cifs': 'disable',
> 'storage.owner-gid': '36', 'storage.owner-uid': '36', 'nfs.disable':
> 'on', 'auth.allow': '*'}}}}
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,460::clusterlock::289::SANLock::(release) Cluster lock for
> domain abc51e26-7175-4b38-b3a8-95c6928fbc2b successfully released
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::ERROR::2014-11-12
> 16:13:20,460::task::866::TaskManager.Task::(_setError)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Unexpected error
> Traceback (most recent call last):
> File "/usr/share/vdsm/storage/task.py", line 873, in _run
> return fn(*args, **kargs)
> File "/usr/share/vdsm/storage/task.py", line 334, in run
> return self.cmd(*self.argslist, **self.argsdict)
> File "/usr/share/vdsm/storage/sp.py", line 296, in startSpm
> self._updateDomainsRole()
> File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper
> return method(self, *args, **kwargs)
> File "/usr/share/vdsm/storage/sp.py", line 205, in _updateDomainsRole
> domain = sdCache.produce(sdUUID)
> File "/usr/share/vdsm/storage/sdc.py", line 98, in produce
> domain.getRealDomain()
> File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
> return self._cache._realProduce(self._sdUUID)
> File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce
> domain = self._findDomain(sdUUID)
> File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain
> dom = findMethod(sdUUID)
> File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain
> raise se.StorageDomainDoesNotExist(sdUUID)
> StorageDomainDoesNotExist: Storage domain does not exist:
> ('6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1',)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,461::task::885::TaskManager.Task::(_run)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Task._run:
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83 () {} failed - stopping task
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,461::task::1211::TaskManager.Task::(stop)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::stopping in state running
> (force False)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,461::task::990::TaskManager.Task::(_decref)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::ref 1 aborting True
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,461::task::916::TaskManager.Task::(_runJobs)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::aborting: Task is
> aborted: 'Storage domain does not exist' - code 358
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,462::task::990::TaskManager.Task::(_decref)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::ref 0 aborting True
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,462::task::925::TaskManager.Task::(_doAbort)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Task._doAbort: force
> False
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,462::resourceManager::977::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,462::task::595::TaskManager.Task::(_updateState)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state running
> -> state aborting
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,462::task::550::TaskManager.Task::(__state_aborting)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::_aborting: recover policy
> auto
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,463::task::595::TaskManager.Task::(_updateState)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state
> aborting -> state racquiring
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,463::task::595::TaskManager.Task::(_updateState)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state
> racquiring -> state recovering
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,463::task::798::TaskManager.Task::(_recover)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::_recover
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,463::task::805::TaskManager.Task::(_recover)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::running recovery None
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,463::task::786::TaskManager.Task::(_recoverDone)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Recover Done: state
> recovering
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,463::task::595::TaskManager.Task::(_updateState)
> Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state
> recovering -> state recovered
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,464::resourceManager::940::ResourceManager.Owner::(releaseAll)
> Owner.releaseAll requests {} resources
> {'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d': < ResourceRef
> 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d', isValid: 'True' obj:
> 'None'>}
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,464::resourceManager::977::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,464::resourceManager::616::ResourceManager::(releaseResource)
> Trying to release resource
> 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d'
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,464::resourceManager::635::ResourceManager::(releaseResource)
> Released resource 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d' (0
> active users)
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,465::resourceManager::641::ResourceManager::(releaseResource)
> Resource 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d' is free,
> finding out if anyone is waiting for it.
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,465::resourceManager::649::ResourceManager::(releaseResource)
> No one is waiting for resource
> 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d', Clearing records.
> fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12
> 16:13:20,465::threadPool::57::Misc.ThreadPool::(setRunningTask) Number
> of running tasks: 0
> Thread-28::DEBUG::2014-11-12
> 16:13:20,940::BindingXMLRPC::251::vds::(wrapper) client
> [192.168.150.8] flowID [4c2997b8]
> Thread-28::DEBUG::2014-11-12
> 16:13:20,941::task::595::TaskManager.Task::(_updateState)
> Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::moving from state init ->
> state preparing
> Thread-28::INFO::2014-11-12
> 16:13:20,941::logUtils::44::dispatcher::(wrapper) Run and protect:
> getTaskStatus(taskID='fda6e0ee-33e9-4eb2-b724-34f7a5492e83',
> spUUID=None, options=None)
> Thread-28::DEBUG::2014-11-12
> 16:13:20,941::taskManager::93::TaskManager::(getTaskStatus) Entry.
> taskID: fda6e0ee-33e9-4eb2-b724-34f7a5492e83
> Thread-28::DEBUG::2014-11-12
> 16:13:20,941::taskManager::96::TaskManager::(getTaskStatus) Return.
> Response: {'code': 358, 'message': 'Storage domain does not exist',
> 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID':
> 'fda6e0ee-33e9-4eb2-b724-34f7a5492e83'}
> Thread-28::INFO::2014-11-12
> 16:13:20,941::logUtils::47::dispatcher::(wrapper) Run and protect:
> getTaskStatus, Return response: {'taskStatus': {'code': 358,
> 'message': 'Storage domain does not exist', 'taskState': 'finished',
> 'taskResult': 'cleanSuccess', 'taskID':
> 'fda6e0ee-33e9-4eb2-b724-34f7a5492e83'}}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,942::task::1185::TaskManager.Task::(prepare)
> Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::finished: {'taskStatus':
> {'code': 358, 'message': 'Storage domain does not exist', 'taskState':
> 'finished', 'taskResult': 'cleanSuccess', 'taskID':
> 'fda6e0ee-33e9-4eb2-b724-34f7a5492e83'}}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,942::task::595::TaskManager.Task::(_updateState)
> Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::moving from state
> preparing -> state finished
> Thread-28::DEBUG::2014-11-12
> 16:13:20,942::resourceManager::940::ResourceManager.Owner::(releaseAll)
> Owner.releaseAll requests {} resources {}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,942::resourceManager::977::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,942::task::990::TaskManager.Task::(_decref)
> Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::ref 0 aborting False
> Thread-28::DEBUG::2014-11-12
> 16:13:20,951::BindingXMLRPC::251::vds::(wrapper) client
> [192.168.150.8] flowID [4c2997b8]
> Thread-28::DEBUG::2014-11-12
> 16:13:20,952::task::595::TaskManager.Task::(_updateState)
> Task=`a421f847-c259-4bdf-929a-b2df3568e881`::moving from state init ->
> state preparing
> Thread-28::INFO::2014-11-12
> 16:13:20,952::logUtils::44::dispatcher::(wrapper) Run and protect:
> getSpmStatus(spUUID='b384b3da-02a6-44f3-a3f6-56751ce8c26d',
> options=None)
> Thread-28::INFO::2014-11-12
> 16:13:20,956::logUtils::47::dispatcher::(wrapper) Run and protect:
> getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus':
> 'Free', 'spmLver': -1}}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,957::task::1185::TaskManager.Task::(prepare)
> Task=`a421f847-c259-4bdf-929a-b2df3568e881`::finished: {'spm_st':
> {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,957::task::595::TaskManager.Task::(_updateState)
> Task=`a421f847-c259-4bdf-929a-b2df3568e881`::moving from state
> preparing -> state finished
> Thread-28::DEBUG::2014-11-12
> 16:13:20,957::resourceManager::940::ResourceManager.Owner::(releaseAll)
> Owner.releaseAll requests {} resources {}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,957::resourceManager::977::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-28::DEBUG::2014-11-12
> 16:13:20,957::task::990::TaskManager.Task::(_decref)
> Task=`a421f847-c259-4bdf-929a-b2df3568e881`::ref 0 aborting False
> Thread-28::DEBUG::2014-11-12
> 16:13:21,006::BindingXMLRPC::251::vds::(wrapper) client
> [192.168.150.8] flowID [4c2997b8]
> Thread-28::DEBUG::2014-11-12
> 16:13:21,006::task::595::TaskManager.Task::(_updateState)
> Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::moving from state init ->
> state preparing
> Thread-28::INFO::2014-11-12
> 16:13:21,006::logUtils::44::dispatcher::(wrapper) Run and protect:
> clearTask(taskID='fda6e0ee-33e9-4eb2-b724-34f7a5492e83', spUUID=None,
> options=None)
> Thread-28::DEBUG::2014-11-12
> 16:13:21,007::taskManager::161::TaskManager::(clearTask) Entry.
> taskID: fda6e0ee-33e9-4eb2-b724-34f7a5492e83
> Thread-28::DEBUG::2014-11-12
> 16:13:21,007::taskManager::166::TaskManager::(clearTask) Return.
> Thread-28::INFO::2014-11-12
> 16:13:21,007::logUtils::47::dispatcher::(wrapper) Run and protect:
> clearTask, Return response: None
> Thread-28::DEBUG::2014-11-12
> 16:13:21,007::task::1185::TaskManager.Task::(prepare)
> Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::finished: None
> Thread-28::DEBUG::2014-11-12
> 16:13:21,007::task::595::TaskManager.Task::(_updateState)
> Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::moving from state
> preparing -> state finished
> Thread-28::DEBUG::2014-11-12
> 16:13:21,007::resourceManager::940::ResourceManager.Owner::(releaseAll)
> Owner.releaseAll requests {} resources {}
> Thread-28::DEBUG::2014-11-12
> 16:13:21,008::resourceManager::977::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-28::DEBUG::2014-11-12
> 16:13:21,008::task::990::TaskManager.Task::(_decref)
> Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::ref 0 aborting False
>
>
> Again: i only replaced my ovirt-engine host by a backup restore.
>
> What could cause this problem?
>
> Thanks,
> Mario
>
>
>
> On Wed, Nov 12, 2014 at 2:16 PM, Sandro Bonazzola <sbona...@redhat.com>
> wrote:
> > Il 12/11/2014 14:06, Ml Ml ha scritto:
> >> Anyone? :-(
> >
> > Dan, Nir, can you take a look?
> >
> >
> >>
> >> On Tue, Nov 11, 2014 at 6:39 PM, Ml Ml <mliebher...@googlemail.com> wrote:
> >>> I dunno why this is all so simple for you.
> >>>
> >>> I just replaced the ovirt-engine like described in the docs.
> >>>
> >>> I ejected the CD ISOs on every vm so i was able to delete the ISO_DOMAIN.
> >>>
> >>> But i have still problems with my storage. Its a replicated glusterfs.
> >>> It looks healthy on the nodes itself. But somehow my ovirt-engine gets
> >>> confused. Can someone explain me what the actual error is?
> >>>
> >>> Note: i only replaced the ovirt-engine host and delete the ISO_DOMAIN:
> >>>
> >>> 2014-11-11 18:32:37,832 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] Failed in
> >>> HSMGetTaskStatusVDS method
> >>> 2014-11-11 18:32:37,833 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] spmStart polling ended:
> >>> taskId = 8c5fae2c-0ddb-41cd-ac54-c404c943e00f task status = finished
> >>> 2014-11-11 18:32:37,834 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] Start SPM Task failed -
> >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException:
> >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist,
> >>> code = 358
> >>> 2014-11-11 18:32:37,888 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] spmStart polling ended,
> >>> spm status: Free
> >>> 2014-11-11 18:32:37,889 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] START,
> >>> HSMClearTaskVDSCommand(HostName = ovirt-node01.foobar.net, HostId =
> >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c,
> >>> taskId=8c5fae2c-0ddb-41cd-ac54-c404c943e00f), log id: 547e26fd
> >>> 2014-11-11 18:32:37,937 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] FINISH,
> >>> HSMClearTaskVDSCommand, log id: 547e26fd
> >>> 2014-11-11 18:32:37,938 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] FINISH,
> >>> SpmStartVDSCommand, return:
> >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@5027ed97,
> >>> log id: 461eb5b5
> >>> 2014-11-11 18:32:37,941 INFO
> >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Running command:
> >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID:
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool
> >>> 2014-11-11 18:32:37,948 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d]
> >>> IrsBroker::Failed::ActivateStorageDomainVDS due to:
> >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException:
> >>> SpmStart failed
> >>> 2014-11-11 18:32:38,006 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Irs placed on server
> >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c failed. Proceed Failover
> >>> 2014-11-11 18:32:38,044 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-29) START,
> >>> GlusterVolumesListVDSCommand(HostName = ovirt-node01.foobar.net,
> >>> HostId = 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c), log id: 7a110756
> >>> 2014-11-11 18:32:38,045 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d]
> >>> hostFromVds::selectedVds - ovirt-node02.foobar.net, spmStatus Free,
> >>> storage pool HP_Proliant_DL180G6
> >>> 2014-11-11 18:32:38,048 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] starting spm on vds
> >>> ovirt-node02.foobar.net, storage pool HP_Proliant_DL180G6, prevId -1,
> >>> LVER -1
> >>> 2014-11-11 18:32:38,050 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] START,
> >>> SpmStartVDSCommand(HostName = ovirt-node02.foobar.net, HostId =
> >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3, storagePoolId =
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, prevId=-1, prevLVER=-1,
> >>> storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log
> >>> id: 1a6ccb9c
> >>> 2014-11-11 18:32:38,108 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] spmStart polling
> >>> started: taskId = 78d31638-70a5-46aa-89e7-1d1e8126bdba
> >>> 2014-11-11 18:32:38,193 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-29) FINISH,
> >>> GlusterVolumesListVDSCommand, return:
> >>> {d46619e9-9368-4e82-bf3a-a2377b6e85e4=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9746ef53},
> >>> log id: 7a110756
> >>> 2014-11-11 18:32:38,352 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-29) START,
> >>> GlusterVolumesListVDSCommand(HostName = ovirt-node04.foobar.net,
> >>> HostId = 073c24e1-003f-412a-be56-0c41a435829a), log id: 2f25d56e
> >>> 2014-11-11 18:32:38,433 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-29) FINISH,
> >>> GlusterVolumesListVDSCommand, return:
> >>> {660ca9ef-46fc-47b0-9b6b-61ccfd74016c=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cd3b51c4},
> >>> log id: 2f25d56e
> >>> 2014-11-11 18:32:39,117 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Failed in
> >>> HSMGetTaskStatusVDS method
> >>> 2014-11-11 18:32:39,118 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] spmStart polling ended:
> >>> taskId = 78d31638-70a5-46aa-89e7-1d1e8126bdba task status = finished
> >>> 2014-11-11 18:32:39,119 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Start SPM Task failed -
> >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException:
> >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist,
> >>> code = 358
> >>> 2014-11-11 18:32:39,171 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] spmStart polling ended,
> >>> spm status: Free
> >>> 2014-11-11 18:32:39,173 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] START,
> >>> HSMClearTaskVDSCommand(HostName = ovirt-node02.foobar.net, HostId =
> >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3,
> >>> taskId=78d31638-70a5-46aa-89e7-1d1e8126bdba), log id: 46abf4a0
> >>> 2014-11-11 18:32:39,220 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] FINISH,
> >>> HSMClearTaskVDSCommand, log id: 46abf4a0
> >>> 2014-11-11 18:32:39,221 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] FINISH,
> >>> SpmStartVDSCommand, return:
> >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@7d3782f7,
> >>> log id: 1a6ccb9c
> >>> 2014-11-11 18:32:39,224 INFO
> >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Running command:
> >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID:
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool
> >>> 2014-11-11 18:32:39,232 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [4777665a]
> >>> IrsBroker::Failed::ActivateStorageDomainVDS due to:
> >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException:
> >>> SpmStart failed
> >>> 2014-11-11 18:32:39,235 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] FINISH,
> >>> ActivateStorageDomainVDSCommand, log id: 75877740
> >>> 2014-11-11 18:32:39,236 ERROR
> >>> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Command
> >>> org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw
> >>> Vdc Bll exception. With error message VdcBLLException:
> >>> org.ovirt.engine.core.vdsbroker.irsbroker.IrsSpmStartFailedException:
> >>> IRSGenericException: IRSErrorException: SpmStart failed (Failed with
> >>> error ENGINE and code 5001)
> >>> 2014-11-11 18:32:39,239 INFO
> >>> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand]
> >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Command
> >>> [id=c5315de2-0817-4da2-a13e-50c8cfa93a6a]: Compensating
> >>> CHANGED_STATUS_ONLY of
> >>> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap;
> >>> snapshot: EntityStatusSnapshot [id=storagePoolId =
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, storageId =
> >>> abc51e26-7175-4b38-b3a8-95c6928fbc2b, status=Unknown].
> >>> 2014-11-11 18:32:39,243 INFO
> >>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Correlation ID:
> >>> 71891fe3, Job ID: 239d4ac0-aa7d-486a-a70f-55a9d1b910f4, Call Stack:
> >>> null, Custom Event ID: -1, Message: Failed to activate Storage Domain
> >>> RaidVolBGluster (Data Center HP_Proliant_DL180G6) by admin
> >>> 2014-11-11 18:32:40,566 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] Command
> >>> org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand
> >>> return value
> >>>
> >>> TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654,
> >>> mMessage=Not SPM]]
> >>>
> >>> 2014-11-11 18:32:40,569 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] HostName =
> >>> ovirt-node02.foobar.net
> >>> 2014-11-11 18:32:40,570 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] Command
> >>> HSMGetAllTasksStatusesVDSCommand(HostName = ovirt-node02.foobar.net,
> >>> HostId = 6948da12-0b8a-4b6d-a9af-162e6c25dad3) execution failed.
> >>> Exception: IRSNonOperationalException: IRSGenericException:
> >>> IRSErrorException: IRSNonOperationalException: Not SPM
> >>> 2014-11-11 18:32:40,625 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] hostFromVds::selectedVds
> >>> - ovirt-node02.foobar.net, spmStatus Free, storage pool
> >>> HP_Proliant_DL180G6
> >>> 2014-11-11 18:32:40,628 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] starting spm on vds
> >>> ovirt-node02.foobar.net, storage pool HP_Proliant_DL180G6, prevId -1,
> >>> LVER -1
> >>> 2014-11-11 18:32:40,630 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] START,
> >>> SpmStartVDSCommand(HostName = ovirt-node02.foobar.net, HostId =
> >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3, storagePoolId =
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, prevId=-1, prevLVER=-1,
> >>> storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log
> >>> id: 1f3ac280
> >>> 2014-11-11 18:32:40,687 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] spmStart polling
> >>> started: taskId = 50ab033e-76cd-44d5-b661-a1c2b8c312ef
> >>> 2014-11-11 18:32:41,735 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] Failed in
> >>> HSMGetTaskStatusVDS method
> >>> 2014-11-11 18:32:41,736 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] spmStart polling ended:
> >>> taskId = 50ab033e-76cd-44d5-b661-a1c2b8c312ef task status = finished
> >>> 2014-11-11 18:32:41,737 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] Start SPM Task failed -
> >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException:
> >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist,
> >>> code = 358
> >>> 2014-11-11 18:32:41,790 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] spmStart polling ended,
> >>> spm status: Free
> >>> 2014-11-11 18:32:41,791 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] START,
> >>> HSMClearTaskVDSCommand(HostName = ovirt-node02.foobar.net, HostId =
> >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3,
> >>> taskId=50ab033e-76cd-44d5-b661-a1c2b8c312ef), log id: 852d287
> >>> 2014-11-11 18:32:41,839 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] FINISH,
> >>> HSMClearTaskVDSCommand, log id: 852d287
> >>> 2014-11-11 18:32:41,840 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [47871083] FINISH,
> >>> SpmStartVDSCommand, return:
> >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@32b92b73,
> >>> log id: 1f3ac280
> >>> 2014-11-11 18:32:41,843 INFO
> >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Running command:
> >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID:
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool
> >>> 2014-11-11 18:32:41,851 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509]
> >>> IrsBroker::Failed::GetStoragePoolInfoVDS due to:
> >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException:
> >>> SpmStart failed
> >>> 2014-11-11 18:32:41,909 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Irs placed on server
> >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3 failed. Proceed Failover
> >>> 2014-11-11 18:32:41,928 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] hostFromVds::selectedVds
> >>> - ovirt-node01.foobar.net, spmStatus Free, storage pool
> >>> HP_Proliant_DL180G6
> >>> 2014-11-11 18:32:41,930 INFO
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] starting spm on vds
> >>> ovirt-node01.foobar.net, storage pool HP_Proliant_DL180G6, prevId -1,
> >>> LVER -1
> >>> 2014-11-11 18:32:41,932 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] START,
> >>> SpmStartVDSCommand(HostName = ovirt-node01.foobar.net, HostId =
> >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c, storagePoolId =
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, prevId=-1, prevLVER=-1,
> >>> storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log
> >>> id: 56dfcc3c
> >>> 2014-11-11 18:32:41,984 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] spmStart polling
> >>> started: taskId = 84ac9f17-d5ec-4e43-8fcc-8ca9065a8492
> >>> 2014-11-11 18:32:42,993 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Failed in
> >>> HSMGetTaskStatusVDS method
> >>> 2014-11-11 18:32:42,994 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] spmStart polling ended:
> >>> taskId = 84ac9f17-d5ec-4e43-8fcc-8ca9065a8492 task status = finished
> >>> 2014-11-11 18:32:42,995 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Start SPM Task failed -
> >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException:
> >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist,
> >>> code = 358
> >>> 2014-11-11 18:32:43,048 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] spmStart polling ended,
> >>> spm status: Free
> >>> 2014-11-11 18:32:43,049 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] START,
> >>> HSMClearTaskVDSCommand(HostName = ovirt-node01.foobar.net, HostId =
> >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c,
> >>> taskId=84ac9f17-d5ec-4e43-8fcc-8ca9065a8492), log id: 5abaa4ce
> >>> 2014-11-11 18:32:43,098 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] FINISH,
> >>> HSMClearTaskVDSCommand, log id: 5abaa4ce
> >>> 2014-11-11 18:32:43,098 INFO
> >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] FINISH,
> >>> SpmStartVDSCommand, return:
> >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@7d9b9905,
> >>> log id: 56dfcc3c
> >>> 2014-11-11 18:32:43,101 INFO
> >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [725b57af] Running command:
> >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID:
> >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool
> >>> 2014-11-11 18:32:43,108 ERROR
> >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> >>> (DefaultQuartzScheduler_Worker-28) [725b57af]
> >>> IrsBroker::Failed::GetStoragePoolInfoVDS due to:
> >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException:
> >>> SpmStart failed
> >>> 2014-11-11 18:32:43,444 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] START,
> >>> GlusterVolumesListVDSCommand(HostName = ovirt-node01.foobar.net,
> >>> HostId = 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c), log id: 12ae9c47
> >>> 2014-11-11 18:32:43,585 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] FINISH,
> >>> GlusterVolumesListVDSCommand, return:
> >>> {d46619e9-9368-4e82-bf3a-a2377b6e85e4=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a5d949dc},
> >>> log id: 12ae9c47
> >>> 2014-11-11 18:32:43,745 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] START,
> >>> GlusterVolumesListVDSCommand(HostName = ovirt-node04.foobar.net,
> >>> HostId = 073c24e1-003f-412a-be56-0c41a435829a), log id: 4b994fd9
> >>> 2014-11-11 18:32:43,826 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] FINISH,
> >>> GlusterVolumesListVDSCommand, return:
> >>> {660ca9ef-46fc-47b0-9b6b-61ccfd74016c=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@10521f1b},
> >>> log id: 4b994fd9
> >>> 2014-11-11 18:32:48,838 INFO
> >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
> >>> (DefaultQuartzScheduler_Worker-71) START,
> >>> GlusterVolumesListVDSCommand(HostName = ovirt-node01.foobar.net,
> >>> HostId = 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c), log id: 3b036a37
> >>>
> >>>
> >>>
> >>> Thanks,
> >>> Mario
> >>>
> >>> On Fri, Nov 7, 2014 at 11:49 PM, Matt . <yamakasi....@gmail.com> wrote:
> >>>> Hi,
> >>>>
> >>>> Actually it's very simple as described in the docs.
> >>>>
> >>>> Just stop the engine, make a backup, copy it over, place it back and
> >>>> start it. You can do this in a several of ways.
> >>>>
> >>>> ISO domains is which I would remove and recreate again. ISO domains
> >>>> are actually dumb domains, so nothing can go wrong.
> >>>>
> >>>> Did it some time ago because I needed more performance.
> >>>>
> >>>> VDSM can run without the engine, it doesn't need it as the egine
> >>>> monitors and does the commands, so when it's not there... VM's just
> >>>> run (until you make them die yourself :))
> >>>>
> >>>> I would give it 15-30 min/
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Matt
> >>>>
> >>>>
> >>>> 2014-11-07 18:36 GMT+01:00 Daniel Helgenberger
> >>>> <daniel.helgenber...@m-box.de>:
> >>>>>
> >>>>> Daniel Helgenberger
> >>>>> m box bewegtbild GmbH
> >>>>>
> >>>>> ACKERSTR. 19 P: +49/30/2408781-22
> >>>>> D-10115 BERLIN F: +49/30/2408781-10
> >>>>>
> >>>>> www.m-box.de
> >>>>> www.monkeymen.tv
> >>>>>
> >>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner
> >>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767
> >>>>> On 07.11.2014, at 15:24, Koen Vanoppen <vanoppen.k...@gmail.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> We had a consulting partner who did the same for our company. This is
> >>>>> his
> >>>>> procedure and worked great:
> >>>>>
> >>>>> How to migrate ovirt management engine
> >>>>> Packages
> >>>>> Ensure you have the same packages & versions installed on the
> >>>>> destination
> >>>>> hostas on the source, using 'rpm -qa | grep ovirt'. Make sure versions
> >>>>> are
> >>>>> 100%identical.
> >>>>> Default setup
> >>>>>
> >>>>> Run 'engine-setup' on the destination host after installing the
> >>>>> packages.
> >>>>> Use
> >>>>> the following configuration:
> >>>>> 1. Backup existing configuration
> >>>>> 2. On the source host, do:
> >>>>>
> >>>>> You might want your consultant take a look on [1]...
> >>>>> Steps a-3d:
> >>>>> engine-backup mode=backup --file=~/ovirt-engine-source --log=backup.log
> >>>>>
> >>>>> a. service ovirt-engine stop
> >>>>> b. service ovirt-engine-dwhd stop
> >>>>> c. mkdir ~/backup
> >>>>> d. tar -C /etc/pki/ovirt-engine -czpf
> >>>>> ~/backup/ovirt-engine-pki.tar.gz .
> >>>>> e. tar -C /etc/ovirt-engine -czpf ~/backup/ovirt-engine-conf.tar.gz
> >>>>> .
> >>>>> f. cd /usr/share/ovirt-engine/dbscripts
> >>>>> g. ./backup.sh
> >>>>> h. mv engine_*.sql ~/backup/engine.sql
> >>>>> 3. You may also want to backup dwh & reports:
> >>>>> a. cd /usr/share/ovirt-engine/bin/
> >>>>> b. ./engine-backup.sh --mode=backup --scope=db --db-user=engine
> >>>>> --db-password=XXX --file=/usr/tmp/rhevm-backups/engine-backup
> >>>>> --log=/tmp/engine-backup.log
> >>>>> c. ./engine-backup.sh --mode=backup --scope=dwhdb --db-user=engine
> >>>>> --db-password=XXX --file=/usr/tmp/rhevm-backups/dwh-backup
> >>>>> --log=/tmp/engine-backup.log
> >>>>> d. ./engine-backup.sh --mode=backup --scope=reportsdb
> >>>>> --db-user=engine
> >>>>> --db-password=XXX --file=/usr/tmp/rhevm-backups/reports-backup
> >>>>> --log=/tmp/engine-backup.log
> >>>>> 4. Download these backup files, and copy them to the destination
> >>>>> host.
> >>>>> Restore configuration
> >>>>> 1. On the destination host, do:
> >>>>>
> >>>>> Again, steps a-h, basically
> >>>>> engine-setup
> >>>>> engine-cleanup
> >>>>> engine-backup mode=restore --file=~/ovirt-engine-source
> >>>>> --log=backup.log
> >>>>>
> >>>>> also, I would run a second
> >>>>> engine-setup
> >>>>> After that, you should be good to go..
> >>>>>
> >>>>> Of course, depending on your previous engine setup this could be a
> >>>>> little
> >>>>> more complicated. Still, quite strait forward.
> >>>>> [1] http://www.ovirt.org/Ovirt-engine-backup
> >>>>>
> >>>>> a. service ovirt-engine stop
> >>>>> b. service ovirt-engine-dwhd stop
> >>>>> c. cd backup
> >>>>> d. tar -C /etc/pki/ovirt-engine -xzpf ovirt-engine-pki.tar.gz
> >>>>> e. tar -C /etc/ovirt-engine -xzpf ovirt-engine-conf.tar.gz
> >>>>> f. tar -xvjf engine-backup
> >>>>> g. tar -xvjf dwh-backup
> >>>>> h. tar -xvjf reports-backup
> >>>>>
> >>>>> Restore Database
> >>>>> 1. On the destination host do:
> >>>>> a. su - postgres -c "psql -d template1 -c 'drop database engine;'"
> >>>>> b. su - postgres -c "psql -d template1 -c 'create database engine
> >>>>> owner
> >>>>> engine;'"
> >>>>> c. su - postgres
> >>>>> d. psql
> >>>>> e. \c engine
> >>>>> f. \i /path/to/backup/engine.sql
> >>>>> NOTE: in case you have issues logging in to the database, add the
> >>>>> following
> >>>>> line to the pg_hba.conf file:
> >>>>>
> >>>>> host all engine 127.0.0.1/32 trust
> >>>>>
> >>>>> 2. Fix engine password:
> >>>>> a. su - postgres
> >>>>> b. psql
> >>>>> c. alter user engine with password 'XXXXXXX';
> >>>>> Change ovirt hostname
> >>>>> On the destination host, run:
> >>>>>
> >>>>> /usr/share/ovirt-engine/setup/bin/ovirt-engine-rename
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> NB:
> >>>>> Restoring the dwh/reports database is similar to steps 5-7, but omitted
> >>>>> from
> >>>>> this document due to problems starting the reporting service.
> >>>>>
> >>>>>
> >>>>> 2014-11-07 10:28 GMT+01:00 Sven Kieske <s.kie...@mittwald.de>:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 07/11/14 10:10, Ml Ml wrote:
> >>>>>>> anyone? :)
> >>>>>>>
> >>>>>>> Or are you only doing backups, no restore? :-P
> >>>>>>
> >>>>>> gladly I just had to test disaster recovery and not actually
> >>>>>> perform it (yet) :D
> >>>>>>
> >>>>>> To be honest: I never have restored ovirt-engine with running vdsm
> >>>>>> hosts connected to it, sounds like a lot of fun, I see if I can
> >>>>>> grab some time and try this out myself :)
> >>>>>>
> >>>>>> By your description I guess you have nfs/iso domain on your engine
> >>>>>> host?
> >>>>>> why don't you just seperate it, so no need for remounts
> >>>>>> if your engine is destroyed.
> >>>>>>
> >>>>>> HTH
> >>>>>>
> >>>>>> --
> >>>>>> Mit freundlichen Grüßen / Regards
> >>>>>>
> >>>>>> Sven Kieske
> >>>>>>
> >>>>>> Systemadministrator
> >>>>>> Mittwald CM Service GmbH & Co. KG
> >>>>>> Königsberger Straße 6
> >>>>>> 32339 Espelkamp
> >>>>>> T: +49-5772-293-100
> >>>>>> F: +49-5772-293-333
> >>>>>> https://www.mittwald.de
> >>>>>> Geschäftsführer: Robert Meyer
> >>>>>> St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad
> >>>>>> Oeynhausen
> >>>>>> Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad
> >>>>>> Oeynhausen
> >>>>>> _______________________________________________
> >>>>>> Users mailing list
> >>>>>> Users@ovirt.org
> >>>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Users mailing list
> >>>>> Users@ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Users mailing list
> >>>>> Users@ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/users
> >>>>>
> >>>> _______________________________________________
> >>>> Users mailing list
> >>>> Users@ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/users
> >> _______________________________________________
> >> Users mailing list
> >> Users@ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/users
> >>
> >
> >
> > --
> > Sandro Bonazzola
> > Better technology. Faster innovation. Powered by community collaboration.
> > See how it works at redhat.com
>
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
