[ovirt-users] Re: Hosted -engine is down and cannot be restarted

2019-03-19 Thread ada per
should i still upgrade it ? I am currently in version 4.3.1.1-1.el7

On Tue, Mar 19, 2019 at 4:17 PM ada per  wrote:

> After changing the ownership the engine is up!!
>
> thanks for your help!!!:)
>
> On Tue, Mar 19, 2019 at 3:25 PM Simone Tiraboschi 
> wrote:
>
>>
>>
>> On Tue, Mar 19, 2019 at 2:21 PM ada per  wrote:
>>
>>> Thanks for you reply.
>>>
>>> Can you please provide step by step instructions on how to upgrade the
>>> vdsm from a node command line?
>>>
>>
>> Can you please report the version of vdsm you are using?
>>
>> then check the ownership of
>>
>> /rhev/data-center/----/05b2b2d5-a80e-4622-9410-8e1e9d362f3f/images/bb890447-f1f7-46af-8e57-543d61f0bd08/81685d19-0060-4f5d-a4cd-c5efa24aecfe
>>
>> if it's not vdsm:kvm, change it and then try again with hosted-engine
>> --vm-start
>>
>>
>>>
>>> On Tue, Mar 19, 2019 at 2:49 PM Simone Tiraboschi 
>>> wrote:
>>>
 Hi Ada,
 here the error:

 2019-03-19 14:08:25,833+0200 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer]
 RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312)
 2019-03-19 14:08:25,839+0200 INFO  (vm/a492d2eb) [vdsm.api] FINISH
 prepareImage error=Volume does not exist:
 (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) from=internal,
 task_id=dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257 (api:52)
 2019-03-19 14:08:25,839+0200 ERROR (vm/a492d2eb)
 [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
 Unexpected error (task:875)
 Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line
 882, in _run
 return fn(*args, **kargs)
   File "", line 2, in prepareImage
   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50,
 in method
 ret = func(*args, **kwargs)
   File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line
 3199, in prepareImage
 legality = dom.produceVolume(imgUUID, volUUID).getLegality()
   File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 822,
 in produceVolume
 volUUID)
   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
 801, in __init__
 self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID,
 volUUID)
   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
 line 71, in __init__
 volUUID)
   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
 86, in __init__
 self.validate()
   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
 112, in validate
 self.validateVolumePath()
   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
 line 131, in validateVolumePath
 raise se.VolumeDoesNotExist(self.volUUID)
 VolumeDoesNotExist: Volume does not exist:
 (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)
 2019-03-19 14:08:25,840+0200 INFO  (vm/a492d2eb)
 [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
 aborting: Task is aborted: "Volume does not exist:
 (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)" - code 201 (task:1181)
 2019-03-19 14:08:25,840+0200 ERROR (vm/a492d2eb) [storage.Dispatcher]
 FINISH prepareImage error=Volume does not exist:
 (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) (dispatcher:83)

 I think it's still https://bugzilla.redhat.com/1666795
 

 Can you please try updating vdsm to vdsm-4.30.10 since the bug is
 reported as solved in that version?




 On Tue, Mar 19, 2019 at 12:30 PM ada per  wrote:

> an vdsm:
>
>
>
>
>
> On Tue, Mar 19, 2019 at 1:24 PM ada per  wrote:
>
>> Thank you! please see attached files:
>>
>> On Tue, Mar 19, 2019 at 12:52 PM Simone Tiraboschi <
>> stira...@redhat.com> wrote:
>>
>>> Can you please check/attach also
>>> /var/log/ovirt-hosted-engine-ha/broker.log and /var/log/vdsm/vdsm.log ?
>>>
>>> On Tue, Mar 19, 2019 at 11:36 AM ada per  wrote:
>>>
 Hello everyone,

 For a strange reason the hosted engine went down and I cannot
 restart it.  I tried manually restarting it without any success can you
 please advice?

 For all the nodes the engine status is the same as the one below.
 --== Host nodex. (id: 6) status ==--
 conf_on_shared_storage : True
 Status up-to-date  : True
 Hostname   : nodex
 Host ID: 6
 Engine status  : {"reason": "bad vm status",
 "health": "bad", "vm": "down_unexpected", "detail": "Down"}
 Score  : 3400
 stopped: False
 Local maintenance 

[ovirt-users] Re: Hosted -engine is down and cannot be restarted

2019-03-19 Thread ada per
After changing the ownership the engine is up!!

thanks for your help!!!:)

On Tue, Mar 19, 2019 at 3:25 PM Simone Tiraboschi 
wrote:

>
>
> On Tue, Mar 19, 2019 at 2:21 PM ada per  wrote:
>
>> Thanks for you reply.
>>
>> Can you please provide step by step instructions on how to upgrade the
>> vdsm from a node command line?
>>
>
> Can you please report the version of vdsm you are using?
>
> then check the ownership of
>
> /rhev/data-center/----/05b2b2d5-a80e-4622-9410-8e1e9d362f3f/images/bb890447-f1f7-46af-8e57-543d61f0bd08/81685d19-0060-4f5d-a4cd-c5efa24aecfe
>
> if it's not vdsm:kvm, change it and then try again with hosted-engine
> --vm-start
>
>
>>
>> On Tue, Mar 19, 2019 at 2:49 PM Simone Tiraboschi 
>> wrote:
>>
>>> Hi Ada,
>>> here the error:
>>>
>>> 2019-03-19 14:08:25,833+0200 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer]
>>> RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312)
>>> 2019-03-19 14:08:25,839+0200 INFO  (vm/a492d2eb) [vdsm.api] FINISH
>>> prepareImage error=Volume does not exist:
>>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) from=internal,
>>> task_id=dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257 (api:52)
>>> 2019-03-19 14:08:25,839+0200 ERROR (vm/a492d2eb)
>>> [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
>>> Unexpected error (task:875)
>>> Traceback (most recent call last):
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line
>>> 882, in _run
>>> return fn(*args, **kargs)
>>>   File "", line 2, in prepareImage
>>>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50,
>>> in method
>>> ret = func(*args, **kwargs)
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line
>>> 3199, in prepareImage
>>> legality = dom.produceVolume(imgUUID, volUUID).getLegality()
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 822,
>>> in produceVolume
>>> volUUID)
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
>>> 801, in __init__
>>> self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID,
>>> volUUID)
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
>>> line 71, in __init__
>>> volUUID)
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
>>> 86, in __init__
>>> self.validate()
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
>>> 112, in validate
>>> self.validateVolumePath()
>>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
>>> line 131, in validateVolumePath
>>> raise se.VolumeDoesNotExist(self.volUUID)
>>> VolumeDoesNotExist: Volume does not exist:
>>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)
>>> 2019-03-19 14:08:25,840+0200 INFO  (vm/a492d2eb)
>>> [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
>>> aborting: Task is aborted: "Volume does not exist:
>>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)" - code 201 (task:1181)
>>> 2019-03-19 14:08:25,840+0200 ERROR (vm/a492d2eb) [storage.Dispatcher]
>>> FINISH prepareImage error=Volume does not exist:
>>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) (dispatcher:83)
>>>
>>> I think it's still https://bugzilla.redhat.com/1666795
>>> 
>>>
>>> Can you please try updating vdsm to vdsm-4.30.10 since the bug is
>>> reported as solved in that version?
>>>
>>>
>>>
>>>
>>> On Tue, Mar 19, 2019 at 12:30 PM ada per  wrote:
>>>
 an vdsm:





 On Tue, Mar 19, 2019 at 1:24 PM ada per  wrote:

> Thank you! please see attached files:
>
> On Tue, Mar 19, 2019 at 12:52 PM Simone Tiraboschi <
> stira...@redhat.com> wrote:
>
>> Can you please check/attach also
>> /var/log/ovirt-hosted-engine-ha/broker.log and /var/log/vdsm/vdsm.log ?
>>
>> On Tue, Mar 19, 2019 at 11:36 AM ada per  wrote:
>>
>>> Hello everyone,
>>>
>>> For a strange reason the hosted engine went down and I cannot
>>> restart it.  I tried manually restarting it without any success can you
>>> please advice?
>>>
>>> For all the nodes the engine status is the same as the one below.
>>> --== Host nodex. (id: 6) status ==--
>>> conf_on_shared_storage : True
>>> Status up-to-date  : True
>>> Hostname   : nodex
>>> Host ID: 6
>>> Engine status  : {"reason": "bad vm status",
>>> "health": "bad", "vm": "down_unexpected", "detail": "Down"}
>>> Score  : 3400
>>> stopped: False
>>> Local maintenance  : False
>>> crc32  : 323a9f45
>>> local_conf_timestamp   : 2648874
>>> Host timestamp : 2648874
>>> Extra metadata (valid at timestamp):
>>> 

[ovirt-users] Re: Hosted -engine is down and cannot be restarted

2019-03-19 Thread Simone Tiraboschi
On Tue, Mar 19, 2019 at 2:21 PM ada per  wrote:

> Thanks for you reply.
>
> Can you please provide step by step instructions on how to upgrade the
> vdsm from a node command line?
>

Can you please report the version of vdsm you are using?

then check the ownership of
/rhev/data-center/----/05b2b2d5-a80e-4622-9410-8e1e9d362f3f/images/bb890447-f1f7-46af-8e57-543d61f0bd08/81685d19-0060-4f5d-a4cd-c5efa24aecfe

if it's not vdsm:kvm, change it and then try again with hosted-engine
--vm-start


>
> On Tue, Mar 19, 2019 at 2:49 PM Simone Tiraboschi 
> wrote:
>
>> Hi Ada,
>> here the error:
>>
>> 2019-03-19 14:08:25,833+0200 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer]
>> RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312)
>> 2019-03-19 14:08:25,839+0200 INFO  (vm/a492d2eb) [vdsm.api] FINISH
>> prepareImage error=Volume does not exist:
>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) from=internal,
>> task_id=dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257 (api:52)
>> 2019-03-19 14:08:25,839+0200 ERROR (vm/a492d2eb)
>> [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
>> Unexpected error (task:875)
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
>> in _run
>> return fn(*args, **kargs)
>>   File "", line 2, in prepareImage
>>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in
>> method
>> ret = func(*args, **kwargs)
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3199,
>> in prepareImage
>> legality = dom.produceVolume(imgUUID, volUUID).getLegality()
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 822,
>> in produceVolume
>> volUUID)
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
>> 801, in __init__
>> self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID,
>> volUUID)
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
>> line 71, in __init__
>> volUUID)
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
>> 86, in __init__
>> self.validate()
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
>> 112, in validate
>> self.validateVolumePath()
>>   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py",
>> line 131, in validateVolumePath
>> raise se.VolumeDoesNotExist(self.volUUID)
>> VolumeDoesNotExist: Volume does not exist:
>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)
>> 2019-03-19 14:08:25,840+0200 INFO  (vm/a492d2eb)
>> [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
>> aborting: Task is aborted: "Volume does not exist:
>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)" - code 201 (task:1181)
>> 2019-03-19 14:08:25,840+0200 ERROR (vm/a492d2eb) [storage.Dispatcher]
>> FINISH prepareImage error=Volume does not exist:
>> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) (dispatcher:83)
>>
>> I think it's still https://bugzilla.redhat.com/1666795
>> 
>>
>> Can you please try updating vdsm to vdsm-4.30.10 since the bug is
>> reported as solved in that version?
>>
>>
>>
>>
>> On Tue, Mar 19, 2019 at 12:30 PM ada per  wrote:
>>
>>> an vdsm:
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Mar 19, 2019 at 1:24 PM ada per  wrote:
>>>
 Thank you! please see attached files:

 On Tue, Mar 19, 2019 at 12:52 PM Simone Tiraboschi 
 wrote:

> Can you please check/attach also
> /var/log/ovirt-hosted-engine-ha/broker.log and /var/log/vdsm/vdsm.log ?
>
> On Tue, Mar 19, 2019 at 11:36 AM ada per  wrote:
>
>> Hello everyone,
>>
>> For a strange reason the hosted engine went down and I cannot restart
>> it.  I tried manually restarting it without any success can you please
>> advice?
>>
>> For all the nodes the engine status is the same as the one below.
>> --== Host nodex. (id: 6) status ==--
>> conf_on_shared_storage : True
>> Status up-to-date  : True
>> Hostname   : nodex
>> Host ID: 6
>> Engine status  : {"reason": "bad vm status",
>> "health": "bad", "vm": "down_unexpected", "detail": "Down"}
>> Score  : 3400
>> stopped: False
>> Local maintenance  : False
>> crc32  : 323a9f45
>> local_conf_timestamp   : 2648874
>> Host timestamp : 2648874
>> Extra metadata (valid at timestamp):
>> metadata_parse_version=1
>> metadata_feature_version=1
>> timestamp=2648874 (Tue Mar 19 12:25:44 2019)
>> host-id=6
>> score=3400
>> vm_conf_refresh_time=2648874 (Tue Mar 19 12:25:44 2019)
>> conf_on_shared_storage=True
>>

[ovirt-users] Re: Hosted -engine is down and cannot be restarted

2019-03-19 Thread ada per
Thanks for you reply.

Can you please provide step by step instructions on how to upgrade the vdsm
from a node command line?

On Tue, Mar 19, 2019 at 2:49 PM Simone Tiraboschi 
wrote:

> Hi Ada,
> here the error:
>
> 2019-03-19 14:08:25,833+0200 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
> call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312)
> 2019-03-19 14:08:25,839+0200 INFO  (vm/a492d2eb) [vdsm.api] FINISH
> prepareImage error=Volume does not exist:
> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) from=internal,
> task_id=dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257 (api:52)
> 2019-03-19 14:08:25,839+0200 ERROR (vm/a492d2eb)
> [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
> Unexpected error (task:875)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
> in _run
> return fn(*args, **kargs)
>   File "", line 2, in prepareImage
>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in
> method
> ret = func(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3199,
> in prepareImage
> legality = dom.produceVolume(imgUUID, volUUID).getLegality()
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 822, in
> produceVolume
> volUUID)
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
> 801, in __init__
> self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
> 71, in __init__
> volUUID)
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86,
> in __init__
> self.validate()
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line
> 112, in validate
> self.validateVolumePath()
>   File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
> 131, in validateVolumePath
> raise se.VolumeDoesNotExist(self.volUUID)
> VolumeDoesNotExist: Volume does not exist:
> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)
> 2019-03-19 14:08:25,840+0200 INFO  (vm/a492d2eb)
> [storage.TaskManager.Task] (Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257')
> aborting: Task is aborted: "Volume does not exist:
> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)" - code 201 (task:1181)
> 2019-03-19 14:08:25,840+0200 ERROR (vm/a492d2eb) [storage.Dispatcher]
> FINISH prepareImage error=Volume does not exist:
> (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) (dispatcher:83)
>
> I think it's still https://bugzilla.redhat.com/1666795
> 
>
> Can you please try updating vdsm to vdsm-4.30.10 since the bug is reported
> as solved in that version?
>
>
>
>
> On Tue, Mar 19, 2019 at 12:30 PM ada per  wrote:
>
>> an vdsm:
>>
>>
>>
>>
>>
>> On Tue, Mar 19, 2019 at 1:24 PM ada per  wrote:
>>
>>> Thank you! please see attached files:
>>>
>>> On Tue, Mar 19, 2019 at 12:52 PM Simone Tiraboschi 
>>> wrote:
>>>
 Can you please check/attach also
 /var/log/ovirt-hosted-engine-ha/broker.log and /var/log/vdsm/vdsm.log ?

 On Tue, Mar 19, 2019 at 11:36 AM ada per  wrote:

> Hello everyone,
>
> For a strange reason the hosted engine went down and I cannot restart
> it.  I tried manually restarting it without any success can you please
> advice?
>
> For all the nodes the engine status is the same as the one below.
> --== Host nodex. (id: 6) status ==--
> conf_on_shared_storage : True
> Status up-to-date  : True
> Hostname   : nodex
> Host ID: 6
> Engine status  : {"reason": "bad vm status",
> "health": "bad", "vm": "down_unexpected", "detail": "Down"}
> Score  : 3400
> stopped: False
> Local maintenance  : False
> crc32  : 323a9f45
> local_conf_timestamp   : 2648874
> Host timestamp : 2648874
> Extra metadata (valid at timestamp):
> metadata_parse_version=1
> metadata_feature_version=1
> timestamp=2648874 (Tue Mar 19 12:25:44 2019)
> host-id=6
> score=3400
> vm_conf_refresh_time=2648874 (Tue Mar 19 12:25:44 2019)
> conf_on_shared_storage=True
> maintenance=False
> state=GlobalMaintenance
> stopped=False
>
> When I try the commands
> root@node5# hosted-engine --vm-shutdown
> I ge the response:
> root@node5# Command VM.shutdown with args {'delay': '120', 'message':
> 'VM is shutting down!', 'vmID': 'a492d2eb-1dfd-470d-a141-3e55d2189275'}
> failed:(code=1, message=Virtual machine does not exist)
>
> But when I run  : hosted-engine --vm-start
> I get the response: VM exists and is down, 

[ovirt-users] Re: Hosted -engine is down and cannot be restarted

2019-03-19 Thread Simone Tiraboschi
Hi Ada,
here the error:

2019-03-19 14:08:25,833+0200 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312)
2019-03-19 14:08:25,839+0200 INFO  (vm/a492d2eb) [vdsm.api] FINISH
prepareImage error=Volume does not exist:
(u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) from=internal,
task_id=dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257 (api:52)
2019-03-19 14:08:25,839+0200 ERROR (vm/a492d2eb) [storage.TaskManager.Task]
(Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257') Unexpected error (task:875)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
in _run
return fn(*args, **kargs)
  File "", line 2, in prepareImage
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in
method
ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3199,
in prepareImage
legality = dom.produceVolume(imgUUID, volUUID).getLegality()
  File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 822, in
produceVolume
volUUID)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 801,
in __init__
self._manifest = self.manifestClass(repoPath, sdUUID, imgUUID, volUUID)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
71, in __init__
volUUID)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 86,
in __init__
self.validate()
  File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 112,
in validate
self.validateVolumePath()
  File "/usr/lib/python2.7/site-packages/vdsm/storage/fileVolume.py", line
131, in validateVolumePath
raise se.VolumeDoesNotExist(self.volUUID)
VolumeDoesNotExist: Volume does not exist:
(u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)
2019-03-19 14:08:25,840+0200 INFO  (vm/a492d2eb) [storage.TaskManager.Task]
(Task='dc8fbf34-8d7e-47a3-8e02-0a5e5cb90257') aborting: Task is aborted:
"Volume does not exist: (u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',)" - code
201 (task:1181)
2019-03-19 14:08:25,840+0200 ERROR (vm/a492d2eb) [storage.Dispatcher]
FINISH prepareImage error=Volume does not exist:
(u'81685d19-0060-4f5d-a4cd-c5efa24aecfe',) (dispatcher:83)

I think it's still https://bugzilla.redhat.com/1666795


Can you please try updating vdsm to vdsm-4.30.10 since the bug is reported
as solved in that version?




On Tue, Mar 19, 2019 at 12:30 PM ada per  wrote:

> an vdsm:
>
>
>
>
>
> On Tue, Mar 19, 2019 at 1:24 PM ada per  wrote:
>
>> Thank you! please see attached files:
>>
>> On Tue, Mar 19, 2019 at 12:52 PM Simone Tiraboschi 
>> wrote:
>>
>>> Can you please check/attach also
>>> /var/log/ovirt-hosted-engine-ha/broker.log and /var/log/vdsm/vdsm.log ?
>>>
>>> On Tue, Mar 19, 2019 at 11:36 AM ada per  wrote:
>>>
 Hello everyone,

 For a strange reason the hosted engine went down and I cannot restart
 it.  I tried manually restarting it without any success can you please
 advice?

 For all the nodes the engine status is the same as the one below.
 --== Host nodex. (id: 6) status ==--
 conf_on_shared_storage : True
 Status up-to-date  : True
 Hostname   : nodex
 Host ID: 6
 Engine status  : {"reason": "bad vm status",
 "health": "bad", "vm": "down_unexpected", "detail": "Down"}
 Score  : 3400
 stopped: False
 Local maintenance  : False
 crc32  : 323a9f45
 local_conf_timestamp   : 2648874
 Host timestamp : 2648874
 Extra metadata (valid at timestamp):
 metadata_parse_version=1
 metadata_feature_version=1
 timestamp=2648874 (Tue Mar 19 12:25:44 2019)
 host-id=6
 score=3400
 vm_conf_refresh_time=2648874 (Tue Mar 19 12:25:44 2019)
 conf_on_shared_storage=True
 maintenance=False
 state=GlobalMaintenance
 stopped=False

 When I try the commands
 root@node5# hosted-engine --vm-shutdown
 I ge the response:
 root@node5# Command VM.shutdown with args {'delay': '120', 'message':
 'VM is shutting down!', 'vmID': 'a492d2eb-1dfd-470d-a141-3e55d2189275'}
 failed:(code=1, message=Virtual machine does not exist)

 But when I run  : hosted-engine --vm-start
 I get the response: VM exists and is down, cleaning up and restarting



 Below you can see the # journalctl -u ovirt-ha-agent logs

 Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent
 ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Unhandled
 monitoring loop exception
   Traceback
 

[ovirt-users] Re: Hosted -engine is down and cannot be restarted

2019-03-19 Thread Simone Tiraboschi
Can you please check/attach also /var/log/ovirt-hosted-engine-ha/broker.log
and /var/log/vdsm/vdsm.log ?

On Tue, Mar 19, 2019 at 11:36 AM ada per  wrote:

> Hello everyone,
>
> For a strange reason the hosted engine went down and I cannot restart it.
> I tried manually restarting it without any success can you please advice?
>
> For all the nodes the engine status is the same as the one below.
> --== Host nodex. (id: 6) status ==--
> conf_on_shared_storage : True
> Status up-to-date  : True
> Hostname   : nodex
> Host ID: 6
> Engine status  : {"reason": "bad vm status", "health":
> "bad", "vm": "down_unexpected", "detail": "Down"}
> Score  : 3400
> stopped: False
> Local maintenance  : False
> crc32  : 323a9f45
> local_conf_timestamp   : 2648874
> Host timestamp : 2648874
> Extra metadata (valid at timestamp):
> metadata_parse_version=1
> metadata_feature_version=1
> timestamp=2648874 (Tue Mar 19 12:25:44 2019)
> host-id=6
> score=3400
> vm_conf_refresh_time=2648874 (Tue Mar 19 12:25:44 2019)
> conf_on_shared_storage=True
> maintenance=False
> state=GlobalMaintenance
> stopped=False
>
> When I try the commands
> root@node5# hosted-engine --vm-shutdown
> I ge the response:
> root@node5# Command VM.shutdown with args {'delay': '120', 'message': 'VM
> is shutting down!', 'vmID': 'a492d2eb-1dfd-470d-a141-3e55d2189275'}
> failed:(code=1, message=Virtual machine does not exist)
>
> But when I run  : hosted-engine --vm-start
> I get the response: VM exists and is down, cleaning up and restarting
>
>
>
> Below you can see the # journalctl -u ovirt-ha-agent logs
>
> Mar 14 12:04:42 node7. ovirt-ha-agent[4134]: ovirt-ha-agent
> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Unhandled
> monitoring loop exception
>   Traceback (most
> recent call last):
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 430, in start_monitoring
>
> self._monitoring_loop()
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 449, in _monitoring_loop
>   for
> old_state, state, delay in self.fsm:
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py",
> line 127, in next
>   new_data =
> self.refresh(self._state.data)
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py",
> line 81, in refresh
>
> stats.update(self.hosted_engine.collect_stats())
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
> line 737, in collect_stats
>   all_stats =
> self._broker.get_stats_from_storage()
> File
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
> line 143, in get_stats_from_storage
>   result =
> self._proxy.get_stats()
> File
> "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
>   return
> self.__send(self.__name, args)
> File
> "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
>
> verbose=self.__verbose
> File
> "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
>   return
> self.single_request(host, handler, request_body, verbose)
> File
> "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in single_request
>
> self.send_content(h, request_body)
> File
> "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in send_content
>
> connection.endheaders(request_body)
> File
> "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
>
> self._send_output(message_body)
> File
>