Re: [ovirt-users] Cloning VM on NFS Leads to Locked Disks

[Charles Tassell]

Hi Fred,

  Unfortunately we reinstalled the cluster a while ago, so I don't know 
the exact version we were running at the time.  It would have been the 
latest version in the 4.1 production repos though if that helps.



On 2017-07-02 11:32 AM, Fred Rolland wrote:

Seems you hit [1].
Can you tell me what is the exact version of the engine ?

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1395793

On Thu, May 11, 2017 at 10:29 PM, Charles Tassell 
<char...@islandadmin.ca <mailto:char...@islandadmin.ca>> wrote:


Sure, it's pretty big so I've put it online for download at
http://krissy.islandadmin.ca/public/engine.log.txt
<http://krissy.islandadmin.ca/public/engine.log.txt>


On 2017-05-11 04:08 PM, Fred Rolland wrote:

The locking is on the engine side and restarting the vdsm will
not affect it .
Can you send the whole engine log ?
Which exact version are you using ?


On Thu, May 11, 2017 at 9:30 PM, Charles Tassell
<char...@islandadmin.ca <mailto:char...@islandadmin.ca>> wrote:

Just as an update, I created a new VM and had the same issue:
the disk remains locked.  So I then added a new data store
(this one iSCSI not NFS) and create a new VM on that.  Again,
the disk remains locked.  So the problem seems to be that any
action that sets to modify a disk image on my cluster locks
the disk and keeps it locked permanently.

I tried restarting the vdsm daemon, but that didn't make a
difference.  I'm seeing this in my sanlock.log file though,
which worries me:


017-05-07 07:51:41-0300 1738538 [13575]: s2 renewal error
-202 delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [11513]: s1 renewal error
-202 delta_length 10 last_success 1738508

Here's the last 20 lines:
2017-05-07 07:51:41-0300 1738538 [13580]: s3 renewal error
-202 delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [13575]: 20423d5e aio
timeout RD 0x7fe1440008c0:0x7fe1440008d0:0x7fe160255000 ioto
10 to_count 67
2017-05-07 07:51:41-0300 1738538 [13575]: s2 delta_renew read
timeout 10 sec offset 0

/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/ids
2017-05-07 07:51:41-0300 1738538 [13575]: s2 renewal error
-202 delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [11513]: hosted-e aio
timeout RD 0x7fe1480008c0:0x7fe1480008d0:0x7fe14e6fc000 ioto
10 to_count 65
2017-05-07 07:51:41-0300 1738538 [11513]: s1 delta_renew read
timeout 10 sec offset 0

/var/run/vdsm/storage/5dccd07d-a923-4d4b-9cb1-3b51ebfdca4d/5a9c284f-0faa-4a25-94ce-c9efdae07484/ab2443f1-95ed-475d-886c-c1653257cf04
2017-05-07 07:51:41-0300 1738538 [11513]: s1 renewal error
-202 delta_length 10 last_success 1738508
2017-05-07 07:51:47-0300 1738544 [13575]: 20423d5e aio
collect RD 0x7fe1440008c0:0x7fe1440008d0:0x7fe160255000
result 1048576:0 match reap
2017-05-07 07:51:47-0300 1738544 [13580]: 5dccd07d aio
collect RD 0x7fe13c0008c0:0x7fe13c0008d0:0x7fe14e5fa000
result 1048576:0 match reap
2017-05-07 07:51:47-0300 1738544 [11513]: hosted-e aio
collect RD 0x7fe1480008c0:0x7fe1480008d0:0x7fe14e6fc000
result 1048576:0 match reap
2017-05-07 07:53:57-0300 1738674 [13590]: s2:r15 resource

20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576
for 7,21,78395
2017-05-07 07:59:49-0300 1739027 [13575]: s2 delta_renew long
write time 10 sec
2017-05-09 08:38:34-0300 1914151 [13590]: s2:r16 resource

20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576
for 7,21,78395
2017-05-11 15:07:45-0300 2110302 [13590]: s2:r17 resource

20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576
for 7,21,112346
2017-05-11 15:17:24-0300 2110881 [13590]: s4 lockspace

b010093e-1924-46e1-bd57-2cf2b2445087:1:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/ids:0
2017-05-11 15:17:45-0300 2110902 [1395]: s4 host 1 1 2110881
44ae07eb-3371-4750-8728-ab3b049dbae2.ovirt730-0
2017-05-11 15:17:45-0300 2110902 [1400]: s4:r18 resource

b010093e-1924-46e1-bd57-2cf2b2445087:SDM:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/leases:1048576
for 7,21,112346
2017-05-11 15:17:52-0300 2110909 [1399]: s5 lockspace

b010093e-1924-46e1-bd57-2cf2b2445087:1:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/ids:0
2017-05-

Re: [ovirt-users] Cloning VM on NFS Leads to Locked Disks

[Charles Tassell]

Sure, it's pretty big so I've put it online for download at 
http://krissy.islandadmin.ca/public/engine.log.txt


On 2017-05-11 04:08 PM, Fred Rolland wrote:
The locking is on the engine side and restarting the vdsm will not 
affect it .

Can you send the whole engine log ?
Which exact version are you using ?


On Thu, May 11, 2017 at 9:30 PM, Charles Tassell 
<char...@islandadmin.ca <mailto:char...@islandadmin.ca>> wrote:


Just as an update, I created a new VM and had the same issue: the
disk remains locked.  So I then added a new data store (this one
iSCSI not NFS) and create a new VM on that.  Again, the disk
remains locked.  So the problem seems to be that any action that
sets to modify a disk image on my cluster locks the disk and keeps
it locked permanently.

I tried restarting the vdsm daemon, but that didn't make a
difference.  I'm seeing this in my sanlock.log file though, which
worries me:


017-05-07 07:51:41-0300 1738538 [13575]: s2 renewal error -202
delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [11513]: s1 renewal error -202
delta_length 10 last_success 1738508

Here's the last 20 lines:
2017-05-07 07:51:41-0300 1738538 [13580]: s3 renewal error -202
delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [13575]: 20423d5e aio timeout RD
0x7fe1440008c0:0x7fe1440008d0:0x7fe160255000 ioto 10 to_count 67
2017-05-07 07:51:41-0300 1738538 [13575]: s2 delta_renew read
timeout 10 sec offset 0

/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/ids
2017-05-07 07:51:41-0300 1738538 [13575]: s2 renewal error -202
delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [11513]: hosted-e aio timeout RD
0x7fe1480008c0:0x7fe1480008d0:0x7fe14e6fc000 ioto 10 to_count 65
2017-05-07 07:51:41-0300 1738538 [11513]: s1 delta_renew read
timeout 10 sec offset 0

/var/run/vdsm/storage/5dccd07d-a923-4d4b-9cb1-3b51ebfdca4d/5a9c284f-0faa-4a25-94ce-c9efdae07484/ab2443f1-95ed-475d-886c-c1653257cf04
2017-05-07 07:51:41-0300 1738538 [11513]: s1 renewal error -202
delta_length 10 last_success 1738508
2017-05-07 07:51:47-0300 1738544 [13575]: 20423d5e aio collect RD
0x7fe1440008c0:0x7fe1440008d0:0x7fe160255000 result 1048576:0
match reap
2017-05-07 07:51:47-0300 1738544 [13580]: 5dccd07d aio collect RD
0x7fe13c0008c0:0x7fe13c0008d0:0x7fe14e5fa000 result 1048576:0
match reap
2017-05-07 07:51:47-0300 1738544 [11513]: hosted-e aio collect RD
0x7fe1480008c0:0x7fe1480008d0:0x7fe14e6fc000 result 1048576:0
match reap
2017-05-07 07:53:57-0300 1738674 [13590]: s2:r15 resource

20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576
for 7,21,78395
2017-05-07 07:59:49-0300 1739027 [13575]: s2 delta_renew long
write time 10 sec
2017-05-09 08:38:34-0300 1914151 [13590]: s2:r16 resource

20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576
for 7,21,78395
2017-05-11 15:07:45-0300 2110302 [13590]: s2:r17 resource

20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576
for 7,21,112346
2017-05-11 15:17:24-0300 2110881 [13590]: s4 lockspace

b010093e-1924-46e1-bd57-2cf2b2445087:1:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/ids:0
2017-05-11 15:17:45-0300 2110902 [1395]: s4 host 1 1 2110881
44ae07eb-3371-4750-8728-ab3b049dbae2.ovirt730-0
2017-05-11 15:17:45-0300 2110902 [1400]: s4:r18 resource

b010093e-1924-46e1-bd57-2cf2b2445087:SDM:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/leases:1048576
for 7,21,112346
2017-05-11 15:17:52-0300 2110909 [1399]: s5 lockspace

b010093e-1924-46e1-bd57-2cf2b2445087:1:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/ids:0
2017-05-11 15:18:13-0300 2110930 [1395]: s5 host 1 2 2110909
44ae07eb-3371-4750-8728-ab3b049dbae2.ovirt730-0
2017-05-11 15:18:13-0300 2110930 [1395]: s5 host 2 1 2110065
4d31313f-b2dd-4368-bf31-d39835e10afb.ovirt730-0


On 2017-05-11 10:09 AM, Charles Tassell wrote:


Hi Freddy,

  Sure, thanks for looking into this.  Here you go:

2017-05-10 11:35:30,249-03 INFO
[org.ovirt.engine.core.bll.aaa.SessionDataContainer]
(DefaultQuartzScheduler8) [1c84abac] Not removing session

'vZyqrcCljPC7hQtcILsk4uDug3QsiinZBOyoGDiQKkYYT2znGyWe4fasrPbjYxdjbfyR3DBnp+UZ9/k20dGsMA==',
session has running commands for user 'admin@internal-authz'.[snip]

On 2017-05-11 04:30 AM, Fred Rolland wrote:

Hi,

Can you provide the engine log ?

Thanks,

Freddy

On Wed, May 10, 2017 at 5:57 PM, Charles Tassell
<char...@islandadmin.

Re: [ovirt-users] Cloning VM on NFS Leads to Locked Disks

[Charles Tassell]

Just as an update, I created a new VM and had the same issue: the disk 
remains locked.  So I then added a new data store (this one iSCSI not 
NFS) and create a new VM on that.  Again, the disk remains locked.  So 
the problem seems to be that any action that sets to modify a disk image 
on my cluster locks the disk and keeps it locked permanently.


I tried restarting the vdsm daemon, but that didn't make a difference.  
I'm seeing this in my sanlock.log file though, which worries me:



017-05-07 07:51:41-0300 1738538 [13575]: s2 renewal error -202 
delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [11513]: s1 renewal error -202 
delta_length 10 last_success 1738508


Here's the last 20 lines:
2017-05-07 07:51:41-0300 1738538 [13580]: s3 renewal error -202 
delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [13575]: 20423d5e aio timeout RD 
0x7fe1440008c0:0x7fe1440008d0:0x7fe160255000 ioto 10 to_count 67
2017-05-07 07:51:41-0300 1738538 [13575]: s2 delta_renew read timeout 10 
sec offset 0 
/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/ids
2017-05-07 07:51:41-0300 1738538 [13575]: s2 renewal error -202 
delta_length 10 last_success 1738508
2017-05-07 07:51:41-0300 1738538 [11513]: hosted-e aio timeout RD 
0x7fe1480008c0:0x7fe1480008d0:0x7fe14e6fc000 ioto 10 to_count 65
2017-05-07 07:51:41-0300 1738538 [11513]: s1 delta_renew read timeout 10 
sec offset 0 
/var/run/vdsm/storage/5dccd07d-a923-4d4b-9cb1-3b51ebfdca4d/5a9c284f-0faa-4a25-94ce-c9efdae07484/ab2443f1-95ed-475d-886c-c1653257cf04
2017-05-07 07:51:41-0300 1738538 [11513]: s1 renewal error -202 
delta_length 10 last_success 1738508
2017-05-07 07:51:47-0300 1738544 [13575]: 20423d5e aio collect RD 
0x7fe1440008c0:0x7fe1440008d0:0x7fe160255000 result 1048576:0 match reap
2017-05-07 07:51:47-0300 1738544 [13580]: 5dccd07d aio collect RD 
0x7fe13c0008c0:0x7fe13c0008d0:0x7fe14e5fa000 result 1048576:0 match reap
2017-05-07 07:51:47-0300 1738544 [11513]: hosted-e aio collect RD 
0x7fe1480008c0:0x7fe1480008d0:0x7fe14e6fc000 result 1048576:0 match reap
2017-05-07 07:53:57-0300 1738674 [13590]: s2:r15 resource 
20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576 
for 7,21,78395
2017-05-07 07:59:49-0300 1739027 [13575]: s2 delta_renew long write time 
10 sec
2017-05-09 08:38:34-0300 1914151 [13590]: s2:r16 resource 
20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576 
for 7,21,78395
2017-05-11 15:07:45-0300 2110302 [13590]: s2:r17 resource 
20423d5e-188c-4e10-9893-588ceb81b354:SDM:/rhev/data-center/mnt/192.168.130.217:_media_ovirt/20423d5e-188c-4e10-9893-588ceb81b354/dom_md/leases:1048576 
for 7,21,112346
2017-05-11 15:17:24-0300 2110881 [13590]: s4 lockspace 
b010093e-1924-46e1-bd57-2cf2b2445087:1:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/ids:0
2017-05-11 15:17:45-0300 2110902 [1395]: s4 host 1 1 2110881 
44ae07eb-3371-4750-8728-ab3b049dbae2.ovirt730-0
2017-05-11 15:17:45-0300 2110902 [1400]: s4:r18 resource 
b010093e-1924-46e1-bd57-2cf2b2445087:SDM:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/leases:1048576 
for 7,21,112346
2017-05-11 15:17:52-0300 2110909 [1399]: s5 lockspace 
b010093e-1924-46e1-bd57-2cf2b2445087:1:/dev/b010093e-1924-46e1-bd57-2cf2b2445087/ids:0
2017-05-11 15:18:13-0300 2110930 [1395]: s5 host 1 2 2110909 
44ae07eb-3371-4750-8728-ab3b049dbae2.ovirt730-0
2017-05-11 15:18:13-0300 2110930 [1395]: s5 host 2 1 2110065 
4d31313f-b2dd-4368-bf31-d39835e10afb.ovirt730-0



On 2017-05-11 10:09 AM, Charles Tassell wrote:


Hi Freddy,

  Sure, thanks for looking into this.  Here you go:

2017-05-10 11:35:30,249-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler8) [1c84abac] Not removing session 
'vZyqrcCljPC7hQtcILsk4uDug3QsiinZBOyoGDiQKkYYT2znGyWe4fasrPbjYxdjbfyR3DBnp+UZ9/k20dGsMA==', 
session has running commands for user 'admin@internal-authz'.[snip]


On 2017-05-11 04:30 AM, Fred Rolland wrote:

Hi,

Can you provide the engine log ?

Thanks,

Freddy

On Wed, May 10, 2017 at 5:57 PM, Charles Tassell 
<char...@islandadmin.ca <mailto:char...@islandadmin.ca>> wrote:


Hi Everyone,

  I'm having some issues with my oVirt 4.1 (fully updated to
latest release as of yesterday) cluster.  When I clone a VM the
disks of both the original and the clone stay in the locked
state, and the only way I can resolve it is to go into the
database on the engine and run "update images set imagestatus=1
where imagestatus=2;"

  I'm using NFS4 as a datastore and the disks seem to copy fine
(file sizes match and everything), but the locking worries me. 
To clone the VM I just shut the source VM down and then right

click on it and select "Clone"

  I've attached the full VDSM lo

Re: [ovirt-users] Cloning VM on NFS Leads to Locked Disks

[Charles Tassell]

sult [within thread]: endAction for 
action type 'CopyImageGroup' succeeded, clearing tasks.
2017-05-10 11:38:09,325-03 INFO 
[org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] 
SPMAsyncTask::ClearAsyncTask: Attempting to clear task 
'de21abbb-5ec9-4f03-886d-c853abcc80ec'
2017-05-10 11:38:09,326-03 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] START, 
SPMClearTaskVDSCommand( 
SPMTaskGuidBaseVDSCommandParameters:{runAsync='true', 
storagePoolId='0001-0001-0001-0001-0311', 
ignoreFailoverLimit='false', 
taskId='de21abbb-5ec9-4f03-886d-c853abcc80ec'}), log id: 3ae0372a
2017-05-10 11:38:09,326-03 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] START, 
HSMClearTaskVDSCommand(HostName = ovirt730-01, 
HSMTaskGuidBaseVDSCommandParameters:{runAsync='true', 
hostId='f9519bb1-7b11-4a7b-98d8-72f606288e7c', 
taskId='de21abbb-5ec9-4f03-886d-c853abcc80ec'}), log id: 26278f86
2017-05-10 11:38:10,313-03 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] FINISH, 
HSMClearTaskVDSCommand, log id: 26278f86
2017-05-10 11:38:10,313-03 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] FINISH, 
SPMClearTaskVDSCommand, log id: 3ae0372a
2017-05-10 11:38:10,316-03 INFO 
[org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] 
BaseAsyncTask::removeTaskFromDB: Removed task 
'de21abbb-5ec9-4f03-886d-c853abcc80ec' from DataBase
2017-05-10 11:38:10,316-03 INFO 
[org.ovirt.engine.core.bll.tasks.CommandAsyncTask] 
(org.ovirt.thread.pool-6-thread-34) [78d33cee] 
CommandAsyncTask::HandleEndActionResult [within thread]: Removing 
CommandMultiAsyncTasks object for entity 
'f273e5fe-0d77-47c0-a5b1-8dc8f16dde9f'
2017-05-10 11:38:30,304-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler9) [29c890f1] Not removing session 
'vZyqrcCljPC7hQtcILsk4uDug3QsiinZBOyoGDiQKkYYT2znGyWe4fasrPbjYxdjbfyR3DBnp+UZ9/k20dGsMA==', 
session has running commands for user 'admin@internal-authz'.
2017-05-10 11:38:30,304-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler9) [29c890f1] Not removing session 
'nLzKb6P4Th3A9zUFNNZYuZgWaL/DraYRfQjOvjmXisa9c3SiI+Sv8xvLZ+n2+9ptKPHqaiw0oNCOG6kDWOaszA==', 
session has running commands for user 'admin@internal-authz'.
2017-05-10 11:39:26,248-03 INFO 
[org.ovirt.engine.core.bll.tasks.AsyncTaskManager] 
(DefaultQuartzScheduler9) [29c890f1] Setting new tasks map. The map 
contains now 0 tasks
2017-05-10 11:39:26,248-03 INFO 
[org.ovirt.engine.core.bll.tasks.AsyncTaskManager] 
(DefaultQuartzScheduler9) [29c890f1] Cleared all tasks of pool 
'0001-0001-0001-0001-0311'.
2017-05-10 11:39:30,318-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler9) [29c890f1] Not removing session 
'vZyqrcCljPC7hQtcILsk4uDug3QsiinZBOyoGDiQKkYYT2znGyWe4fasrPbjYxdjbfyR3DBnp+UZ9/k20dGsMA==', 
session has running commands for user 'admin@internal-authz'.
2017-05-10 11:39:30,318-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler9) [29c890f1] Not removing session 
'nLzKb6P4Th3A9zUFNNZYuZgWaL/DraYRfQjOvjmXisa9c3SiI+Sv8xvLZ+n2+9ptKPHqaiw0oNCOG6kDWOaszA==', 
session has running commands for user 'admin@internal-authz'.
2017-05-10 11:40:30,333-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler3) [5bdc6552] Not removing session 
'vZyqrcCljPC7hQtcILsk4uDug3QsiinZBOyoGDiQKkYYT2znGyWe4fasrPbjYxdjbfyR3DBnp+UZ9/k20dGsMA==', 
session has running commands for user 'admin@internal-authz'.
2017-05-10 11:40:30,333-03 INFO 
[org.ovirt.engine.core.bll.aaa.SessionDataContainer] 
(DefaultQuartzScheduler3) [5bdc6552] Not removing session 
'nLzKb6P4Th3A9zUFNNZYuZgWaL/DraYRfQjOvjmXisa9c3SiI+Sv8xvLZ+n2+9ptKPHqaiw0oNCOG6kDWOaszA==', 
session has running commands for user 'admin@internal-authz'.



On 2017-05-11 04:30 AM, Fred Rolland wrote:

Hi,

Can you provide the engine log ?

Thanks,

Freddy

On Wed, May 10, 2017 at 5:57 PM, Charles Tassell 
<char...@islandadmin.ca <mailto:char...@islandadmin.ca>> wrote:


Hi Everyone,

  I'm having some issues with my oVirt 4.1 (fully updated to
latest release as of yesterday) cluster.  When I clone a VM the
disks of both the original and the clone stay in the locked state,
and the only way I can resolve it is to go into the database on
the engine and run "update images set imagestatus=1 where
imagestatus=2;"

  I'm using NFS4 as a datastore and the disks seem to copy fine
(file sizes match and everything), but the locking worries me.  To
clone the VM I just shut the source VM down and then right click
on it and select "C

Re: [ovirt-users] how does ovirt deal with multiple networks, with multiple gateways

[Charles Tassell]

Hi Martin,

  oVirt 4.1 (and maybe earlier versions that I just didn't notice) use 
policy based routing.  You can see these routes by typing "ip rule show" 
on one of the hosts.


  In situations such as yours, where you are connected to multiple 
networks and need to specify which one has the gateway, there is a 
little trick you need to use on the admin box (usually the hosted engine 
VM, it' sthe same box that runs the GUI)  ssh into that box and then run 
the following two commands:


  engine-config -g UserDefinedNetworkCustomProperties
  engine-config -s 
UserDefinedNetworkCustomProperties='default_route=^(true|false)$'


Next, go into the GUI and onto the Hosts tab.  Click on your first host 
then on the Network Interfaces tab, and "Setup Host Networks"  Click on 
the little pencil icon next to your VM network (the one that should have 
the default route) and then on the Custom Properties tab.  Add the 
property "default_route" and set it to true and hit OK.  Then click on 
all of the other networks you have and add the same property with the 
value of false.  Then hit OK to save the network config.  Repeat this 
process for all hosts.


Note that I think when I did this I had a problem getting it to save for 
the host that was running the HostedEngine.  You may have to put each 
host into Maintenance Mode in order to apply the setting.


This bug has some more details on the setup I think: 
https://bugzilla.redhat.com/show_bug.cgi?id=1400366




On 2017-04-30 09:00 AM, users-requ...@ovirt.org wrote:

Message: 1
Date: Sun, 30 Apr 2017 13:03:28 +0200
From: martin chamambo 
To: Yaniv Kaul 
Cc: users 
Subject: Re: [ovirt-users] how does ovirt deal with multiple networks
with multiple gateways
Message-ID:

Re: [ovirt-users] Communicate between the guest vm's hosted on, different hosts on different data centres using OVS

[Charles Tassell]

Hmm, so you have 2 10G ports on each host.  1 port is currently in use 
with oVirt, and the other isn't?  Then I think you just add a new 
network in the Networks tab called OVNLink, then go into the Hosts tab, 
click on each host and then on the Network Interfaces tab in the bottom 
of the window and assign the unused 10G network device to that link.


I haven't used OVN with oVirt, but that would work with regular 
networking so I think it should work with OVN.


If the link is already in use, you should be able to do the same thing 
only when you add the new network set it up with a VLAN ID. If you are 
going through a switch then the switch will have to be setup to handle 
the VLANs.



On 2017-04-20 10:07 PM, Linov Suresh wrote:

Hi Charles,

We use an Ethernet cable to connect between the hosts through 10G port.

We want to use the guest VM's hosted on different hosts 10G network. 
oVirt already supports SR-IOV. So we should guest 10G speed for our VM's.


In KVM we can create a second bridge and add the 10G interface to the 
bridge would do the trick.


But in oVirt how do we do this? We want to use our OVs network, which 
was created using OVN Provider.


Linov Suresh


On Thu, Apr 20, 2017 at 6:59 PM, Charles Tassell <ctass...@gmail.com 
<mailto:ctass...@gmail.com>> wrote:


Hi Suresh,

  You would need to connect the two OVN instances somehow.  If
it's just two single hosts, I think the easiest way would be to
create a VPN connection between the two hosts with OpenVPN or the
like and then add the tun/tap interfaces into the OVN on each
box.  You might run into problems if you start adding more hosts
though, as if the host with the VPN goes down it would disconnect
the two datacenters.

  If the two datacenters are on the same physical network (ie, you
just mean oVirt datacenter and not different colocation providers)
then adding a VLAN to the NICs connected to the OVN interface
would work.  You would probably have to setup some sort of channel
bonding/LACP as you add more hosts, but OVN should be able to
handle that simply enough.

On 2017-04-20 07:33 PM, users-requ...@ovirt.org
<mailto:users-requ...@ovirt.org> wrote:

Date: Thu, 20 Apr 2017 14:43:26 -0400
From: Linov Suresh <linov.sur...@gmail.com
<mailto:linov.sur...@gmail.com>>
To: users@ovirt.org <mailto:users@ovirt.org>
Subject: [ovirt-users] Communicate between the guest vm's
hosted on
different hosts on different data centres using OVS
Message-ID:
   

Re: [ovirt-users] Communicate between the guest vm's hosted on, different hosts on different data centres using OVS

[Charles Tassell]

Hi Suresh,

  You would need to connect the two OVN instances somehow.  If it's 
just two single hosts, I think the easiest way would be to create a VPN 
connection between the two hosts with OpenVPN or the like and then add 
the tun/tap interfaces into the OVN on each box.  You might run into 
problems if you start adding more hosts though, as if the host with the 
VPN goes down it would disconnect the two datacenters.


  If the two datacenters are on the same physical network (ie, you just 
mean oVirt datacenter and not different colocation providers) then 
adding a VLAN to the NICs connected to the OVN interface would work.  
You would probably have to setup some sort of channel bonding/LACP as 
you add more hosts, but OVN should be able to handle that simply enough.


On 2017-04-20 07:33 PM, users-requ...@ovirt.org wrote:

Date: Thu, 20 Apr 2017 14:43:26 -0400
From: Linov Suresh 
To: users@ovirt.org
Subject: [ovirt-users] Communicate between the guest vm's hosted on
different hosts on different data centres using OVS
Message-ID:

Re: [ovirt-users] More 4.1 Networking Questions

[Charles Tassell]

And bingo!  I totally spaced on the fact that the MAC pool would be the 
same on the two clusters.  I changed the pool range (Configure->MAC 
Address Pools for anyone interested) on the 4.1 cluster and things are 
looking good again. Thank you, this was driving me nuts!


On 2017-04-11 05:22 AM, users-requ...@ovirt.org wrote:

Message: 1
Date: Mon, 10 Apr 2017 21:07:44 -0400
From: Jeff Bailey <bai...@cs.kent.edu>
To: users@ovirt.org
Subject: Re: [ovirt-users] More 4.1 Networking Questions
Message-ID: <31d1821c-9172-5a62-4f7d-b43fa4608...@cs.kent.edu>
Content-Type: text/plain; charset=windows-1252; format=flowed

On 4/10/2017 6:59 AM, Charles Tassell wrote:


Ah, spoke to soon.  30 seconds later the network went down with IPv6
disabled.  So it does appear to be a host forwarding problem, not a VM
problem.  I have an oVirt 4.0 cluster on the same network that doesn't
have these issues, so it must be a configuration issue somewhere.
Here is a dump of my ip config on the host:


The same L2 network?  Using the same range of MAC addresses?

[snip]
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] More 4.1 Networking Questions

[Charles Tassell]

Hi Everyone,

  Just an update, I installed a new Ubuntu guest VM and it was doing 
the same thing regarding the network going down, then I disabled IPv6 
and it's been fine for the past 10-15 minutes.  So the issue seems to be 
IPv6 related, and I don't need IPv6 so I can just turn it off.  The eth1 
NIC disappearing is still worrisome though.



On 2017-04-10 07:13 AM, Charles Tassell wrote:

Hi Everyone,

  Thanks for the help, answers below.

On 2017-04-10 05:27 AM, Sandro Bonazzola wrote:

Adding Simone and Martin, replying inline.

On Mon, Apr 10, 2017 at 10:16 AM, Ondrej Svoboda <osvob...@redhat.com 
<mailto:osvob...@redhat.com>> wrote:


Hello Charles,

First, can you give us more information regarding the duplicated
IPv6 addresses? Since you are going to reinstall the hosted
engine, could you make sure that NetworkManager is disabled
before adding the second vNIC (and perhaps even disable IPv6 and
reboot as well, so we have a solid base and see what makes the
difference)?

I disabled NetworkManager on the hosts (systemctl disable 
NetworkManager ; service NetworkManager stop) before doing the oVirt 
setup and rebooted to make sure that it didn't come back up.  Or are 
you referring to on the hosted engine VM?  I just removed and re-added 
the eth1 NIC in the hosted engine, and this is what showed up in dmesg:
[Mon Apr 10 06:46:43 2017] pci :00:08.0: [1af4:1000] type 00 class 
0x02

[Mon Apr 10 06:46:43 2017] pci :00:08.0: reg 0x10: [io 0x-0x001f]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: reg 0x14: [mem 
0x-0x0fff]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: reg 0x20: [mem 
0x-0x3fff 64bit pref]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: reg 0x30: [mem 
0x-0x0003 pref]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: BAR 6: assigned [mem 
0xc000-0xc003 pref]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: BAR 4: assigned [mem 
0xc004-0xc0043fff 64bit pref]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: BAR 1: assigned [mem 
0xc0044000-0xc0044fff]
[Mon Apr 10 06:46:43 2017] pci :00:08.0: BAR 0: assigned [io 
0x1000-0x101f]
[Mon Apr 10 06:46:43 2017] virtio-pci :00:08.0: enabling device 
( -> 0003)

[Mon Apr 10 06:46:43 2017] virtio-pci :00:08.0: irq 35 for MSI/MSI-X
[Mon Apr 10 06:46:43 2017] virtio-pci :00:08.0: irq 36 for MSI/MSI-X
[Mon Apr 10 06:46:43 2017] virtio-pci :00:08.0: irq 37 for MSI/MSI-X
[Mon Apr 10 06:46:43 2017] IPv6: ADDRCONF(NETDEV_UP): eth1: link is 
not ready
[Mon Apr 10 06:46:43 2017] IPv6: eth1: IPv6 duplicate address 
fe80::21a:4aff:fe16:151 detected!


Then when the network dropped I started getting these:

[Mon Apr 10 06:48:00 2017] IPv6: eth1: IPv6 duplicate address 
2001:410:e000:902:21a:4aff:fe16:151 detected!
[Mon Apr 10 06:48:00 2017] IPv6: eth1: IPv6 duplicate address 
2001:410:e000:902:21a:4aff:fe16:151 detected!
[Mon Apr 10 06:49:51 2017] IPv6: eth1: IPv6 duplicate address 
2001:410:e000:902:21a:4aff:fe16:151 detected!
[Mon Apr 10 06:51:40 2017] IPv6: eth1: IPv6 duplicate address 
2001:410:e000:902:21a:4aff:fe16:151 detected!


The network on eth1 would go down for a few seconds then come back up, 
but networking stays solid on eth0.  I disabled NetworkManager on the 
HE VM as well to see if that makes a difference.  I also disabled IPv6 
with sysctl to see if that helps.  I'll install a Ubuntu VM on the 
cluster later today and see if it has a similar issue.





What kind of documentation did you follow to install the hosted
engine? Was it this page?
https://www.ovirt.org/documentation/how-to/hosted-engine/
<https://www.ovirt.org/documentation/how-to/hosted-engine/> If
so, could you file a bug against VDSM networking and attach
/var/log/vdsm/vdsm.log and supervdsm.log, and make sure they
include the time period from adding the second vNIC to rebooting?

Second, even the vNIC going missing after reboot looks like a bug
to me. Even though eth1 does not exist in the VM, can you see it
defined for the VM in the engine web GUI?


If the HE vm configuration wasn't flushed to the OVF_STORE yet, it 
make sense it disappeared on restart.


The docs I used were 
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html/self-hosted_engine_guide/chap-deploying_self-hosted_engine#Deploying_Self-Hosted_Engine_on_RHEL 
which someone on the list pointed me to last week as being more 
up-to-date than what was on the website (the docs on the website don't 
seem to mention that you need to put the HE on it's own datastore and 
look to be more geared towards bare-metal engine rather than the VM 
self hosted option.)


When I went back into the GUI and looked at the hosted engine config 
the second NIC was listed there, but it wasn't showing up in lspci on 
the VM.  I removed the NIC in the GUI and re-added it, and the device 
appeared again on the VM.  What is the proper way to "sav

Re: [ovirt-users] More 4.1 Networking Questions

[Charles Tassell]

th regards to
networking look good to me, but I believe Sandro (CC'ed) would be
able to give more advice.

Sandro, since we want to configure bonding, would you recommend to
install the engine physically first, move it to a VM, according to
the following method, and only then reconfigure networking?

https://www.ovirt.org/documentation/self-hosted/chap-Migrating_from_Bare_Metal_to_an_EL-Based_Self-Hosted_Environment/

<https://www.ovirt.org/documentation/self-hosted/chap-Migrating_from_Bare_Metal_to_an_EL-Based_Self-Hosted_Environment/>



I don't see why a diret HE deployment couldn't be done. Simone, Martin 
can you help here?




Thank you,
Ondra

On Mon, Apr 10, 2017 at 8:51 AM, Charles Tassell
<ctass...@gmail.com <mailto:ctass...@gmail.com>> wrote:

Hi Everyone,

  Okay, I'm again having problems with getting basic
networking setup with oVirt 4.1  Here is my situation.  I have
two servers I want to use to create an oVirt cluster, with two
different networks.  My "public" network is a 1G link on
device em1 connected to my Internet feed, and my "storage"
network is a 10G link connected on device p5p1 to my file
server.  Since I need to connect to my storage network in
order to do the install, I selected p5p1 has the ovirtmgmt
interface when installing the hosted engine. That worked fine,
I got everything installed, so I used some ssh-proxy magic to
connect to the web console and completed the install (setup a
Storage domain and create a new network vmNet for VM
networking and added em1 to it.)

  The problem was that when I added a second network device to
the HostedEngine VM (so that I can connect to it from my
public network) it would intermittently go down.  I did some
digging and found some IPV6 errors in the dmesg (IPv6: eth1:
IPv6 duplicate address 2001:410:e000:902:21a:4aff:fe16:151
detected!) so I disabled IPv6 on both eth0 and eth1 in the
HostedEngine and rebooted it.  The problem is that when I
restarted the VM, the eth1 device was missing.

  So, my question is: Can I add a second NIC to the
HostedEngine VM and make it stick, or will it be deleted
whenever the engine VM is restarted? 



When you change something in the HE Vm using the web ui, it has to be 
saved also on the OVF_STORE to make it permanent for further reboot.

Martin can you please elaborate here?


Is there a better way to do what I'm trying to do, ie, should
I setup ovirtmgmt on the public em1 interface, and then create
the "storage" network after the fact for connecting to the
datastores and such.  Is that even possible, or required?  I
was thinking that it would be better for migrations and other
management functions to happen on the faster 10G network, but
if the HostedEngine doesn't need to be able to connect to the
storage network maybe it's not worth the effort?

  Eventually I want to setup LACP on the storage network, but
I had to wipe the servers and reinstall from scratch the last
time I tried to set that up.  I was thinking that it was
because I setup the bonding before installing oVirt, so I
didn't do that this time.

  Here are my /etc/sysconfig/network-scripts/ifcfg-* files in
case I did something wrong there (I'm more familiar with
Debian/Ubuntu network setup than CentOS)

ifcfg-eth0: (ovirtmgmt aka storage)

BROADCAST=192.168.130.255
NETMASK=255.255.255.0
BOOTPROTO=static
DEVICE=eth0
IPADDR=192.168.130.179
ONBOOT=yes
DOMAIN=public.net <http://public.net>
ZONE=public
IPV6INIT=no


ifcfg-eth1: (vmNet aka Internet)

BROADCAST=192.168.1.255
NETMASK=255.255.255.0
BOOTPROTO=static
DEVICE=eth1
IPADDR=192.168.1.179
GATEWAY=192.168.1.254
ONBOOT=yes
DNS1=192.168.1.1
DNS2=192.168.1.2
DOMAIN=public.net <http://public.net>
ZONE=public
IPV6INIT=no

___
Users mailing list
Users@ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.org/mailman/listinfo/users>





--

SANDRO BONAZZOLA

ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R

Red Hat EMEA <https://www.redhat.com/>

<https://red.ht/sig>  
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] More 4.1 Networking Questions

[Charles Tassell]

Hi Everyone,

  Okay, I'm again having problems with getting basic networking setup 
with oVirt 4.1  Here is my situation.  I have two servers I want to use 
to create an oVirt cluster, with two different networks.  My "public" 
network is a 1G link on device em1 connected to my Internet feed, and my 
"storage" network is a 10G link connected on device p5p1 to my file 
server.  Since I need to connect to my storage network in order to do 
the install, I selected p5p1 has the ovirtmgmt interface when installing 
the hosted engine.  That worked fine, I got everything installed, so I 
used some ssh-proxy magic to connect to the web console and completed 
the install (setup a Storage domain and create a new network vmNet for 
VM networking and added em1 to it.)


  The problem was that when I added a second network device to the 
HostedEngine VM (so that I can connect to it from my public network) it 
would intermittently go down.  I did some digging and found some IPV6 
errors in the dmesg (IPv6: eth1: IPv6 duplicate address 
2001:410:e000:902:21a:4aff:fe16:151 detected!) so I disabled IPv6 on 
both eth0 and eth1 in the HostedEngine and rebooted it.  The problem is 
that when I restarted the VM, the eth1 device was missing.


  So, my question is: Can I add a second NIC to the HostedEngine VM and 
make it stick, or will it be deleted whenever the engine VM is 
restarted?  Is there a better way to do what I'm trying to do, ie, 
should I setup ovirtmgmt on the public em1 interface, and then create 
the "storage" network after the fact for connecting to the datastores 
and such.  Is that even possible, or required?  I was thinking that it 
would be better for migrations and other management functions to happen 
on the faster 10G network, but if the HostedEngine doesn't need to be 
able to connect to the storage network maybe it's not worth the effort?


  Eventually I want to setup LACP on the storage network, but I had to 
wipe the servers and reinstall from scratch the last time I tried to set 
that up.  I was thinking that it was because I setup the bonding before 
installing oVirt, so I didn't do that this time.


  Here are my /etc/sysconfig/network-scripts/ifcfg-* files in case I 
did something wrong there (I'm more familiar with Debian/Ubuntu network 
setup than CentOS)


ifcfg-eth0: (ovirtmgmt aka storage)

BROADCAST=192.168.130.255
NETMASK=255.255.255.0
BOOTPROTO=static
DEVICE=eth0
IPADDR=192.168.130.179
ONBOOT=yes
DOMAIN=public.net
ZONE=public
IPV6INIT=no


ifcfg-eth1: (vmNet aka Internet)

BROADCAST=192.168.1.255
NETMASK=255.255.255.0
BOOTPROTO=static
DEVICE=eth1
IPADDR=192.168.1.179
GATEWAY=192.168.1.254
ONBOOT=yes
DNS1=192.168.1.1
DNS2=192.168.1.2
DOMAIN=public.net
ZONE=public
IPV6INIT=no

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosts Network Management

[Charles Tassell]

Hey Kai,

  Go into the Hosts tab, click on the host you want to add the bonded 
interface to then on the Network Interfaces tab in the bottom pane.  
Click on Setup Host Networks and drag an unused network adaptor over the 
one that you want to bond it with.  A box will come up and let you 
configure bonding parameters like which mode to use.


On 2017-04-09 04:17 AM, users-requ...@ovirt.org wrote:

Date: Sat, 8 Apr 2017 23:46:27 +0200
From: Kai Wagner 
To: Ondrej Svoboda , users@ovirt.org
Subject: Re: [ovirt-users] Hosts Network Management
Message-ID: 
Content-Type: text/plain; charset="utf-8"

I need to create a bonded device with lacp. How can I do this through
the UI?

Sorry, maybe it's quite easy, but I don't know how to configure it.


On 04/07/2017 07:10 PM, Ondrej Svoboda wrote:

Hello Kai,

I'd like to know ? what is your initial problem that you cannot solve
through the GUI?

VDSM does use a custom comment in ifcfg files, but only to recognize
them ? if they are unknown, it acquires the relevant network devices
from NetworkManager. But VDSM is in charge of networking
configuration, exclusively.

If you need to add extra configuration parameters in ifcfg files, or
modify them, you could write or use a VDSM hook to alter the ifcfg
files as they are written.

But perhaps it is better for us to know what problems you are having
and whether they can be solved by available methods.

Thank you,
Ondra

On Thu, Apr 6, 2017 at 12:14 PM, Kai Wagner > wrote:

 Hi all,

 is it possible to set a comment at the beginning of a file for network
 files -> something like don't touch this anymore please?

 I want to configure my network stuff on the cli and not via the UI
 because I failed now a few times.

 Thx

 Kai


 --
 SUSE Linux GmbH, GF: Felix Imend?rffer, Jane Smithard, Graham
 Norton, HRB 21284 (AG N?rnberg)



 ___
 Users mailing list
 Users@ovirt.org 
 http://lists.ovirt.org/mailman/listinfo/users
 




___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Customized Network Setup?

[Charles Tassell]

Hi Petr,

  That looks to be exactly what I want to do, but when I try to set it 
up I get two error messages.  On the box running the hosted engine I get 
"Error while executing action HostSetupNetworks: Network is currently 
being used" and on the new box I'm trying to get to join the cluster I 
get "Cannot setup Networks. Operation can be performed only when Host 
status is Maintenance, Up, NonOperational."




On 2017-04-05 10:00 AM, Petr Horacek wrote:

Hello Charles,

I think you can get your desired network with oVirt.

Create second network external_network, you can uncheck "VM network"
there. Then in Setup Networks dialog you can create a bonding, add
ovirtmgmt on top of that and external_net on top of a 1G interface.

During the configuration you can specify which network will handle
default route. From
https://bugzilla.redhat.com/show_bug.cgi?id=1400366:

1. In the Manager, open a console and add the custom property.
   engine-config -g UserDefinedNetworkCustomProperties
   engine-config -s
UserDefinedNetworkCustomProperties='default_route=^(true|false)$'
   systemctl restart ovirt-engine

2. On each host:
2.1 Set the property to False on the management (ovirtmgmt|rhevm) network
2.2 Set the property to True on the desired new default route network
2.3 Set a next-hop IP address (Gateway) in the new default route network

3. Apply configurations
Two important notes that must be added to the docs:
* This configuration is PER HOST, and needs to be done on each host.
* If the gataway for the new default route network is not set, it will
be IGNORED. Leaving the gateway blank will NOT set device based route
(such as for proxy-arp scenarios).

Regards,
Petr

2017-04-05 6:22 GMT+02:00 Charles Tassell <char...@islandadmin.ca>:

Hi Guys,

   I'm wondering, is it possible to override VDSM and setup my network
interfaces manually?  I've got two Dell servers with dual-10G networking
that I want to use bonded (LACP) for ovirtmgmt and then some 1G interfaces
that I want to use for the VM network/Internet connection.  I've got it
working on the box I installed the hosted engine on, but it won't properly
setup the networking on additional hosts.  I've tried setting up bonding
before doing the install (worked fine, ran the New host setup through the
admin GUI, network died); doing the install with bonding turned off and then
adding it in later (didn't create the ovirtmgmt bridge and the policy
routing rules that it uses (I'd REALLY like to be able to turn that garbage
off) didn't get added to bond0...)  In all situations it removes the default
route from my standard routing table so I can't access the machines from
outside the subnet...

   I can setup the bonding and bridges manually fairly in 10 minutes, but I'm
worried that VDSM is going to come in and overwrite my config with the
broken stuff if I do that.


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Customized Network Setup?

[Charles Tassell]

Hi Guys,

  I'm wondering, is it possible to override VDSM and setup my network 
interfaces manually?  I've got two Dell servers with dual-10G networking 
that I want to use bonded (LACP) for ovirtmgmt and then some 1G 
interfaces that I want to use for the VM network/Internet connection.  
I've got it working on the box I installed the hosted engine on, but it 
won't properly setup the networking on additional hosts.  I've tried 
setting up bonding before doing the install (worked fine, ran the New 
host setup through the admin GUI, network died); doing the install with 
bonding turned off and then adding it in later (didn't create the 
ovirtmgmt bridge and the policy routing rules that it uses (I'd REALLY 
like to be able to turn that garbage off) didn't get added to bond0...)  
In all situations it removes the default route from my standard routing 
table so I can't access the machines from outside the subnet...


  I can setup the bonding and bridges manually fairly in 10 minutes, 
but I'm worried that VDSM is going to come in and overwrite my config 
with the broken stuff if I do that.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Recent Install Docs?

[Charles Tassell]

Hi Alan,

  Thanks, that's good to know.  But without the storage being imported 
into the cluster, how do I manage the HostedEngine VM? IE, if I want to 
migrate it to another host in order to do maintenance?  I see in my 
event log an entry "Failed to import the Hosted Engine Storage Domain" 
but there are no extra details as to why.  And my Virtual Machines tab 
is empty, where in my other oVirt clusters I've always been able to see 
the HostedEngine there.



On 2017-03-30 03:20 PM, Alan Griffiths wrote:
I believe that is the required setup - dedicated storage domain for 
HE. It's certainly the case on 4.0, I haven't played with 4.1 yet.


For docs you can always check the RHV set, they are generally 
applicable to oVirt.


https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1-beta/html/self-hosted_engine_guide/ 
<https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1-beta/html/self-hosted_engine_guide/>



On 30 March 2017 at 18:17, Charles Tassell <char...@islandadmin.ca 
<mailto:char...@islandadmin.ca>> wrote:


Hello,

  Are there any more recent install docs than what's on the
website?  Those all seem to be back from the 3.x days and don't
really deal with the modern setup of using a hosted engine.

  More specifically, I've noticed that when deploying a hosted
engine I can't import the storage domain that the engine is on.
When I try I get errors like "Can't import the storage domain,
connection already exists"  Because I can't import my storage
domain, I can't initialize the cluster, so I can't really get
anywhere with the setup.  I had this same problem on 4.0 (using
4.1 now) and I "fixed" it by wiping the cluster and creating two
sets of storage: a small 50G one that holds the hosted engine, and
then another one that holds all my VMs.  I don't want to do that
with my new cluster if I can at all avoid it.


___
Users mailing list
Users@ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.org/mailman/listinfo/users>




___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Recent Install Docs?

[Charles Tassell]

Hello,

  Are there any more recent install docs than what's on the website?  
Those all seem to be back from the 3.x days and don't really deal with 
the modern setup of using a hosted engine.


  More specifically, I've noticed that when deploying a hosted engine I 
can't import the storage domain that the engine is on. When I try I get 
errors like "Can't import the storage domain, connection already 
exists"  Because I can't import my storage domain, I can't initialize 
the cluster, so I can't really get anywhere with the setup.  I had this 
same problem on 4.0 (using 4.1 now) and I "fixed" it by wiping the 
cluster and creating two sets of storage: a small 50G one that holds the 
hosted engine, and then another one that holds all my VMs.  I don't want 
to do that with my new cluster if I can at all avoid it.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Best Storage Option: iSCSI/NFS/GlusterFS?

[Charles Tassell]

Hi Marcin,

  Hmm, so if you are using multipath with VDSM you have to manually 
edit the vdsm.conf file to put the right IP in every time the active 
controller switches?  That sort of defeats the purpose of multipath  
That was the issue we were having: we'd spin up another host, it would 
connect to the SAN which would then reballance the disks among 
controllers, and all our other hosts would lose their connection to the 
active controller and pause all of the VMs.  It's the "Device is not on 
preferred path" issue that is common on the MD3x00 line.  We had the 
same errors with VMWare, but VMWare was able to automatically switch to 
the active path.


On 2017-03-26 05:42 PM, Marcin Kruk wrote:
But on the Dell MD32x00 you have got two controllers. The trick is 
that you have to sustain link to both controllers, so the best option 
is to use multipath as Yaniv said. Otherwise you get an error 
notifications from the array.

The problem is with iSCSI target.
After server reboot, VDSM tries to connect to target which was 
previously set, but it could be inactive.
So in that case you have to remember to edit configuration in 
vdsm.conf, because vdsm.conf do not accept target with multi IP addresses.


2017-03-26 9:40 GMT+02:00 Yaniv Kaul <yk...@redhat.com 
<mailto:yk...@redhat.com>>:




On Sat, Mar 25, 2017 at 9:20 AM, Charles Tassell
<ctass...@gmail.com <mailto:ctass...@gmail.com>> wrote:

Hi Everyone,

  I'm about to setup an oVirt cluster with two hosts hitting a
Linux storage server.  Since the Linux box can provide the
storage in pretty much any form, I'm wondering which option is
"best." Our primary focus is on reliability, with performance
being a close second.  Since we will only be using a single
storage server I was thinking NFS would probably beat out
GlusterFS, and that NFSv4 would be a better choice than
NFSv3.  I had assumed that that iSCSI would be better
performance wise, but from what I'm seeing online that might
not be the case.


NFS 4.2 is better than NFS 3 in the sense that you'll get DISCARD
support, which is nice.
Gluster probably requires 3 servers.
In most cases, I don't think people see the difference in
performance between NFS and iSCSI. The theory is that block
storage is faster, but in practice, most don't get to those limits
where it matters really.


  Our servers will be using a 1G network backbone for regular
traffic and a dedicated 10G backbone with LACP for redundancy
and extra bandwidth for storage traffic if that makes a
difference.


LCAP many times (especially on NFS) does not provide extra
bandwidth, as the (single) NFS connection tends to be sticky to a
single physical link.
It's one of the reasons I personally prefer iSCSI with multipathing.


  I'll probably try to do some performance benchmarks with 2-3
options, but the reliability issue is a little harder to test
for.  Has anyone had any particularly bad experiences with a
particular storage option?  We have been using iSCSI with a
Dell MD3x00 SAN and have run into a bunch of issues with the
multipath setup, but that won't be a problem with the new SAN
since it's only got a single controller interface.


A single controller is not very reliable. If reliability is your
primary concern, I suggest ensuring there is no single point of
failure - or at least you are aware of all of them (does the
storage server have redundant power supply? to two power sources?
Of course in some scenarios it's an overkill and perhaps not
practical, but you should be aware of your weak spots).

I'd stick with what you are most comfortable managing - creating,
backing up, extending, verifying health, etc.
Y.



___
Users mailing list
Users@ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.org/mailman/listinfo/users>



___
Users mailing list
Users@ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.org/mailman/listinfo/users>




___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Best Storage Option: iSCSI/NFS/GlusterFS?

[Charles Tassell]

Hi Everyone,

  I'm about to setup an oVirt cluster with two hosts hitting a Linux 
storage server.  Since the Linux box can provide the storage in pretty 
much any form, I'm wondering which option is "best." Our primary focus 
is on reliability, with performance being a close second.  Since we will 
only be using a single storage server I was thinking NFS would probably 
beat out GlusterFS, and that NFSv4 would be a better choice than NFSv3.  
I had assumed that that iSCSI would be better performance wise, but from 
what I'm seeing online that might not be the case.


  Our servers will be using a 1G network backbone for regular traffic 
and a dedicated 10G backbone with LACP for redundancy and extra 
bandwidth for storage traffic if that makes a difference.


  I'll probably try to do some performance benchmarks with 2-3 options, 
but the reliability issue is a little harder to test for.  Has anyone 
had any particularly bad experiences with a particular storage option?  
We have been using iSCSI with a Dell MD3x00 SAN and have run into a 
bunch of issues with the multipath setup, but that won't be a problem 
with the new SAN since it's only got a single controller interface.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Two Problems On New Install

[Charles Tassell]

tion::(emit) Sending event 
{"params": {"notify_time": 5250206560, 
"3390a529-e0c3-4f1f-9c8f-7595b2b3f38f": {"status": "Down", "timeOf
Thread-351015::DEBUG::2017-02-24 
01:01:37,660::vm::777::virt.vm::(_startUnderlyingVm) 
vmId=`3390a529-e0c3-4f1f-9c8f-7595b2b3f38f`::Releasing incoming 
migration semaphore



On 2017-02-15 05:16 AM, Martin Sivak wrote:

Hi Charles,

I am not sure about the migration issue, we need more logs [1] to be
able to tell, but the SSL issue is probably just annoying and not a
functionality bug (see
https://bugzilla.redhat.com/show_bug.cgi?id=1412583).

Best regards

[1] libvirt log for the domain, the VM process might be crashing for some reason

--
Martin Sivak
SLA / oVirt

On Wed, Feb 15, 2017 at 3:45 AM, Charles Tassell <ctass...@gmail.com> wrote:

Hi Everyone,

   I recently installed oVirt 4.0 on two servers with an iSCSI backstore, and
I'm having two issues I'm having trouble tracking down. First, the initial
datastore that the hosted engine is running on won't connect.  It just says
"Unattached" in the GUI, and I can't find an error message anywhere.  Not a
huge deal, it just means I can't see the hosted engine itself in the GUI.

   More importantly: I can't migrate VMs.  They fail with errors like this in
the vdsm.log of the destination host:

Thread-60381::ERROR::2017-02-14
21:55:47,921::vm::767::virt.vm::(_startUnderlyingVm)
vmId=`59f235cc-3ab2-4dae-b460-62be850e3d5d`::Failed to start a migration
destination vm
Traceback (most recent call last):
   File "/usr/share/vdsm/virt/vm.py", line 733, in _startUnderlyingVm
 self._completeIncomingMigration()
   File "/usr/share/vdsm/virt/vm.py", line 3113, in
_completeIncomingMigration
 self._incomingMigrationFinished.isSet(), usedTimeout)
   File "/usr/share/vdsm/virt/vm.py", line 3196, in
_attachLibvirtDomainAfterMigration
 raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid
'59f235cc-3ab2-4dae-b460-62be850e3d5d'

   If I shut the VM down and start it on the host it works fine, it just
won't migrate.  I notice I'm also seeing a lot (like every 7 seconds) of
this error:

JsonRpc (StompReactor)::ERROR::2017-02-14
21:57:33,984::betterAsyncore::113::vds.dispatcher::(recv) SSL error during
reading data: unexpected eof

   Which I can't seem to diagnose.  Any ideas?


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Two Problems On New Install

[Charles Tassell]

Hi Everyone,

  I recently installed oVirt 4.0 on two servers with an iSCSI 
backstore, and I'm having two issues I'm having trouble tracking down. 
First, the initial datastore that the hosted engine is running on won't 
connect.  It just says "Unattached" in the GUI, and I can't find an 
error message anywhere.  Not a huge deal, it just means I can't see the 
hosted engine itself in the GUI.


  More importantly: I can't migrate VMs.  They fail with errors like 
this in the vdsm.log of the destination host:


Thread-60381::ERROR::2017-02-14 
21:55:47,921::vm::767::virt.vm::(_startUnderlyingVm) 
vmId=`59f235cc-3ab2-4dae-b460-62be850e3d5d`::Failed to start a migration 
destination vm

Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 733, in _startUnderlyingVm
self._completeIncomingMigration()
  File "/usr/share/vdsm/virt/vm.py", line 3113, in 
_completeIncomingMigration

self._incomingMigrationFinished.isSet(), usedTimeout)
  File "/usr/share/vdsm/virt/vm.py", line 3196, in 
_attachLibvirtDomainAfterMigration

raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid 
'59f235cc-3ab2-4dae-b460-62be850e3d5d'


  If I shut the VM down and start it on the host it works fine, it just 
won't migrate.  I notice I'm also seeing a lot (like every 7 seconds) of 
this error:


JsonRpc (StompReactor)::ERROR::2017-02-14 
21:57:33,984::betterAsyncore::113::vds.dispatcher::(recv) SSL error 
during reading data: unexpected eof


  Which I can't seem to diagnose.  Any ideas?


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Two Problems On New Install

[Charles Tassell]

Hi Everyone,

  I recently installed oVirt 4.0 on two servers with an iSCSI 
backstore, and I'm having two issues I'm having trouble tracking down.  
First, the initial datastore that the hosted engine is running on won't 
connect.  It just says "Unattached" in the GUI, and I can't find an 
error message anywhere.  Not a huge deal, it just means I can't see the 
hosted engine itself in the GUI.


  More importantly: I can't migrate VMs.  They fail with errors like 
this in the vdsm.log of the destination host:


Thread-60381::ERROR::2017-02-14 
21:55:47,921::vm::767::virt.vm::(_startUnderlyingVm) 
vmId=`59f235cc-3ab2-4dae-b460-62be850e3d5d`::Failed to start a migration 
destination vm

Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 733, in _startUnderlyingVm
self._completeIncomingMigration()
  File "/usr/share/vdsm/virt/vm.py", line 3113, in 
_completeIncomingMigration

self._incomingMigrationFinished.isSet(), usedTimeout)
  File "/usr/share/vdsm/virt/vm.py", line 3196, in 
_attachLibvirtDomainAfterMigration

raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid 
'59f235cc-3ab2-4dae-b460-62be850e3d5d'


  If I shut the VM down and start it on the host it works fine, it just 
won't migrate.  I notice I'm also seeing a lot (like every 7 seconds) of 
this error:


JsonRpc (StompReactor)::ERROR::2017-02-14 
21:57:33,984::betterAsyncore::113::vds.dispatcher::(recv) SSL error 
during reading data: unexpected eof


  Which I can't seem to diagnose.  Any ideas?


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Storage Problem After Host Update (v3.6)

[Charles Tassell]

Hi Pavel,

  Thanks, I added the 3.6-snapshot repo to upgrade to vdsm 4.17.36 and 
then diagnosed a bunch of LVM problems (PV's set as missing in pvs -v) 
and that got us fixed back up!



On 2017-01-04 06:13 PM, Pavel Gashev wrote:

https://bugzilla.redhat.com/1363734
https://bugzilla.redhat.com/1364339
https://bugzilla.redhat.com/1366176

Downgrade the lvm2 package.
Or upgrade vdsm to 4.17.35

-Original Message-
From: <users-boun...@ovirt.org> on behalf of Charles Tassell 
<char...@islandadmin.ca>
Date: Wednesday 4 January 2017 at 23:22
To: "users@ovirt.org" <users@ovirt.org>
Subject: [ovirt-users] Storage Problem After Host Update (v3.6)

Hello,

One of our admins did a package update on one of our 3.6 hosts
through the web GUI and then rebooted the host.  Since then, it hasn't
been able to properly mount storage and thus we can't move any VMs to
it.  I rebooted it myself this afternoon and it took our whole cluster
down for about a half hour, which was kind of concerning.

We use multipath fibre channel for storage and when I do a vgscan or
vgdisplay -s everything shows up correctly but I get a lot of "Duplicate
of PV ..." warning messages.  I also get the log entries below
constantly repeating in vdsm.log.  Any ideas what's going on and how I
fix it?

CannotActivateLogicalVolumes: Cannot activate Logical Volumes: ('General
Storage Exception: (\'5 [] [\\\'  WARNING: lvmetad is running but
disabled. Restart lvmetad before enabling it!\\\', " Refusing activation
of partial LV 8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/metadata.  Use
\\\'--activationmode partial\\\' to
override."]\\n8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/[\\\'metadata\\\']\',)',)
Thread-1844::ERROR::2017-01-04
16:03:43,649::dispatcher::76::Storage.Dispatcher::(wrapper) {'status':
{'message': 'Cannot activate Logical Volumes: (\'General Storage
Exception: (\\\'5 [] [\\\'  WARNING: lvmetad is running but
disabled. Restart lvmetad before enabling it!\\\', " Refusing
activation of partial LV 8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/metadata.
Use \\\'--activationmode partial\\\' to
override."]n8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/[\\\'metadata\\\']\\\',)\',)',
'code': 570}}
Thread-101::ERROR::2017-01-04
16:03:44,303::monitor::276::Storage.Monitor::(_monitorDomain) Error
monitoring domain 8dfcc83b-64d7-4ff7-9800-02ba3430ea7b
Traceback (most recent call last):
File "/usr/share/vdsm/storage/monitor.py", line 264, in _monitorDomain
  self._produceDomain()
File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 769, in
wrapper
  value = meth(self, *a, **kw)
File "/usr/share/vdsm/storage/monitor.py", line 323, in _produceDomain
  self.domain = sdCache.produce(self.sdUUID)
File "/usr/share/vdsm/storage/sdc.py", line 100, in produce
  domain.getRealDomain()
File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
  return self._cache._realProduce(self._sdUUID)
File "/usr/share/vdsm/storage/sdc.py", line 124, in _realProduce
  domain = self._findDomain(sdUUID)
File "/usr/share/vdsm/storage/sdc.py", line 143, in _findDomain
  dom = findMethod(sdUUID)
File "/usr/share/vdsm/storage/blockSD.py", line 1402, in findDomain
  return BlockStorageDomain(BlockStorageDomain.findDomainPath(sdUUID))
File "/usr/share/vdsm/storage/blockSD.py", line 455, in __init__
  manifest = self.manifestClass(sdUUID)
   File "/usr/share/vdsm/storage/blockSD.py", line 407, in __init__
  metadata = selectMetadata(self.sdUUID)
File "/usr/share/vdsm/storage/blockSD.py", line 375, in selectMetadata
  if len(mdProvider) > 0:
File "/usr/share/vdsm/storage/persistentDict.py", line 65, in __len__
  return len(self.keys())
File "/usr/share/vdsm/storage/persistentDict.py", line 110, in keys
  return list(self.__iter__())
File "/usr/share/vdsm/storage/persistentDict.py", line 107, in __iter__
  self._dict.__iter__())
File "/usr/share/vdsm/storage/persistentDict.py", line 225, in __iter__
  with self._accessWrapper():
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
  return self.gen.next()
File "/usr/share/vdsm/storage/persistentDict.py", line 152, in
_accessWrapper
  self.refresh()
File "/usr/share/vdsm/storage/persistentDict.py", line 230, in refresh
  lines = self._metaRW.readlines()
File "/usr/share/vdsm/storage/blockSD.py", line 335, in readlines
  lvm.activateLVs(self._vgName, self._lvName)
File "/usr/share/vdsm/storage/lvm.py", line 1191, in activateLVs
  _setLVAvailability(vgName, toActivate, "y")
File "/usr/share/vdsm/storage/lvm.py", line 826, in _setLVAvailability
  raise error(str(e

[ovirt-users] Storage Problem After Host Update (v3.6)

[Charles Tassell]

Hello,

  One of our admins did a package update on one of our 3.6 hosts 
through the web GUI and then rebooted the host.  Since then, it hasn't 
been able to properly mount storage and thus we can't move any VMs to 
it.  I rebooted it myself this afternoon and it took our whole cluster 
down for about a half hour, which was kind of concerning.


  We use multipath fibre channel for storage and when I do a vgscan or 
vgdisplay -s everything shows up correctly but I get a lot of "Duplicate 
of PV ..." warning messages.  I also get the log entries below 
constantly repeating in vdsm.log.  Any ideas what's going on and how I 
fix it?


CannotActivateLogicalVolumes: Cannot activate Logical Volumes: ('General 
Storage Exception: (\'5 [] [\\\'  WARNING: lvmetad is running but 
disabled. Restart lvmetad before enabling it!\\\', " Refusing activation 
of partial LV 8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/metadata.  Use 
\\\'--activationmode partial\\\' to 
override."]\\n8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/[\\\'metadata\\\']\',)',)
Thread-1844::ERROR::2017-01-04 
16:03:43,649::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': 
{'message': 'Cannot activate Logical Volumes: (\'General Storage 
Exception: (\\\'5 [] [\\\'  WARNING: lvmetad is running but 
disabled. Restart lvmetad before enabling it!\\\', " Refusing 
activation of partial LV 8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/metadata.  
Use \\\'--activationmode partial\\\' to 
override."]n8dfcc83b-64d7-4ff7-9800-02ba3430ea7b/[\\\'metadata\\\']\\\',)\',)', 
'code': 570}}
Thread-101::ERROR::2017-01-04 
16:03:44,303::monitor::276::Storage.Monitor::(_monitorDomain) Error 
monitoring domain 8dfcc83b-64d7-4ff7-9800-02ba3430ea7b

Traceback (most recent call last):
  File "/usr/share/vdsm/storage/monitor.py", line 264, in _monitorDomain
self._produceDomain()
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 769, in 
wrapper

value = meth(self, *a, **kw)
  File "/usr/share/vdsm/storage/monitor.py", line 323, in _produceDomain
self.domain = sdCache.produce(self.sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 100, in produce
domain.getRealDomain()
  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
return self._cache._realProduce(self._sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 124, in _realProduce
domain = self._findDomain(sdUUID)
  File "/usr/share/vdsm/storage/sdc.py", line 143, in _findDomain
dom = findMethod(sdUUID)
  File "/usr/share/vdsm/storage/blockSD.py", line 1402, in findDomain
return BlockStorageDomain(BlockStorageDomain.findDomainPath(sdUUID))
  File "/usr/share/vdsm/storage/blockSD.py", line 455, in __init__
manifest = self.manifestClass(sdUUID)
 File "/usr/share/vdsm/storage/blockSD.py", line 407, in __init__
metadata = selectMetadata(self.sdUUID)
  File "/usr/share/vdsm/storage/blockSD.py", line 375, in selectMetadata
if len(mdProvider) > 0:
  File "/usr/share/vdsm/storage/persistentDict.py", line 65, in __len__
return len(self.keys())
  File "/usr/share/vdsm/storage/persistentDict.py", line 110, in keys
return list(self.__iter__())
  File "/usr/share/vdsm/storage/persistentDict.py", line 107, in __iter__
self._dict.__iter__())
  File "/usr/share/vdsm/storage/persistentDict.py", line 225, in __iter__
with self._accessWrapper():
  File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
  File "/usr/share/vdsm/storage/persistentDict.py", line 152, in 
_accessWrapper

self.refresh()
  File "/usr/share/vdsm/storage/persistentDict.py", line 230, in refresh
lines = self._metaRW.readlines()
  File "/usr/share/vdsm/storage/blockSD.py", line 335, in readlines
lvm.activateLVs(self._vgName, self._lvName)
  File "/usr/share/vdsm/storage/lvm.py", line 1191, in activateLVs
_setLVAvailability(vgName, toActivate, "y")
  File "/usr/share/vdsm/storage/lvm.py", line 826, in _setLVAvailability
raise error(str(e))

LVM output:
[root@ovirt-01 mapper]# vgchange -a y
  WARNING: duplicate PV 1oDZGSqoQ4ueh0XEMzILNleRoNY8ZDAt is being used 
from both devices /dev/sdac and /dev/sdk
  Found duplicate PV 1oDZGSqoQ4ueh0XEMzILNleRoNY8ZDAt: using /dev/sdk 
not /dev/sdac

  Using duplicate PV /dev/sdk which is last seen, replacing /dev/sdac
  Duplicate of PV EMfxl6-8coB-iUdZ-Dx4P-e5LZ-UcIF-kvIfXx dev 
/dev/mapper/360060160ec732300fa5fdde41c66e111p1 exists on unknown device 
65:177
  Duplicate of PV EMfxl6-8coB-iUdZ-Dx4P-e5LZ-UcIF-kvIfXx dev 
/dev/mapper/360060160ec732300fa5fdde41c66e111p1 exists on unknown device 
8:145
  Duplicate of PV i8z5fn-iYa0-ElZg-nL8h-Abul-2HKU-Oh3F8w dev 
/dev/mapper/360060160ec7323000c61c43b3a66e111p1 exists on unknown device 
65:225
  Duplicate of PV i8z5fn-iYa0-ElZg-nL8h-Abul-2HKU-Oh3F8w dev 
/dev/mapper/360060160ec7323000c61c43b3a66e111p1 exists on unknown device 
8:193
  Duplicate of PV f1IMBI-f2kl-vr1b-BbO1-Ydfw-n5Ok-p1zPDd dev 

Re: [ovirt-users] Hosted Engine Spamming Transition E-Mails

[Charles Tassell]

Hi Didi,

  Unfortunately the box got reinstalled before I could grab the logs.  
I think I have an idea as to what caused the issue though. When I was 
originally registering the oVirt node one of the fiber channel volumes 
wouldn't detect so it just kept waiting to register with the hosted 
engine, saying it couldn't connect to the storage pool.  I rebooted the 
host which caused the FC volume to show up and it looked like everything 
was fine.  We were able to host VMs on the box but then it started 
spitting out the error E-Mails.  I migrated the VMs off to other hosts 
(2 went fine, 2 went into a paused state citing storage error) and then 
rebooted the host.  When the host rebooted I shutdown the VDSM service 
and started it manually from the shell prompt.  I'm not sure if it was 
right when the box came up or when I started VDSM manually, but that 
messed up our storage pool and crashed all the VMs in the cluster, 
putting them into a paused state.


  Sorry I couldn't grab the logs, one of the other techs had some spare 
time so he reinstalled the OS the same afternoon.


On 2016-06-21 04:34 PM, Yedidyah Bar David wrote:

On Tue, Jun 21, 2016 at 7:53 PM, Charles Tassell <ctass...@gmail.com> wrote:

Hi Didi,

   Okay, I looked at the logs as you suggested and found one of the hosts was
showing that it couldn't connect to the local VDSM.  I restarted the host
and then tried running VDSM through the console so that I could see the
debugging output, and it crashed my storage domain taking down all the VMs
in the cluster. So that wasn't good... ;-)

   I'm going to leave that host down and reinstall it.  I think the issue was
that when I originally installed it there was a problem with attaching to
the FC storage.  I rebooted it and it seemed to be working fine, joined the
cluster et all, but when I ran VDSM manually it looked like it wanted to
reinitialize the storage or something.

Any chance you can open a bug about this and attach all the logs you can
get from this host before reinstalling it? Thanks!



On 2016-06-19 04:54 AM, Yedidyah Bar David wrote:

On Sat, Jun 18, 2016 at 6:04 PM, Charles Tassell <ctass...@gmail.com>
wrote:

Hi Folks,

I'm having a strange issue with my 3.6 setup.  For the past few days
the
system has been spamming me with "The state machine changed state"
E-Mails.
First I get the "EngineUnexpectedlyDown-EngineDown", then
"EngineDown-EngineStart" then "EngineStart-EngineStarting"  I checked the
hosted engine VM and it's been up for 16 days and the system looks to be
running fine, so I'm wondering what's going on.

   And when I say spamming, I mean I'm getting 200+ E-Mails a day.  It
seems
to trigger every 20 minutes or so with about 2 minutes between each of
the 3
messages in the set.

Please check/share /var/log/ovirt-hosted-engine-ha/agent.log on your
hosts.

Thanks,







___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted Engine Spamming Transition E-Mails

[Charles Tassell]

Hi Didi,

  Okay, I looked at the logs as you suggested and found one of the 
hosts was showing that it couldn't connect to the local VDSM.  I 
restarted the host and then tried running VDSM through the console so 
that I could see the debugging output, and it crashed my storage domain 
taking down all the VMs in the cluster. So that wasn't good... ;-)


  I'm going to leave that host down and reinstall it.  I think the 
issue was that when I originally installed it there was a problem with 
attaching to the FC storage.  I rebooted it and it seemed to be working 
fine, joined the cluster et all, but when I ran VDSM manually it looked 
like it wanted to reinitialize the storage or something.


On 2016-06-19 04:54 AM, Yedidyah Bar David wrote:

On Sat, Jun 18, 2016 at 6:04 PM, Charles Tassell <ctass...@gmail.com> wrote:

Hi Folks,

   I'm having a strange issue with my 3.6 setup.  For the past few days the
system has been spamming me with "The state machine changed state" E-Mails.
First I get the "EngineUnexpectedlyDown-EngineDown", then
"EngineDown-EngineStart" then "EngineStart-EngineStarting"  I checked the
hosted engine VM and it's been up for 16 days and the system looks to be
running fine, so I'm wondering what's going on.

  And when I say spamming, I mean I'm getting 200+ E-Mails a day.  It seems
to trigger every 20 minutes or so with about 2 minutes between each of the 3
messages in the set.

Please check/share /var/log/ovirt-hosted-engine-ha/agent.log on your hosts.

Thanks,


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Hosted Engine Spamming Transition E-Mails

[Charles Tassell]

Hi Folks,

  I'm having a strange issue with my 3.6 setup.  For the past few days 
the system has been spamming me with "The state machine changed state" 
E-Mails.  First I get the "EngineUnexpectedlyDown-EngineDown", then 
"EngineDown-EngineStart" then "EngineStart-EngineStarting"  I checked 
the hosted engine VM and it's been up for 16 days and the system looks 
to be running fine, so I'm wondering what's going on.


 And when I say spamming, I mean I'm getting 200+ E-Mails a day.  It 
seems to trigger every 20 minutes or so with about 2 minutes between 
each of the 3 messages in the set.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Adding another host to my cluster

[Charles Tassell]

quot;, line
173, in Client
c = SocketClient(address)
  File "/usr/lib64/python2.7/multiprocessing/connection.py", line
308, in SocketClient
s.connect(address)
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
    error: [Errno 2] No such file or directory
Killed


Thanks for the help. It's really appreciated.

Cheers,
Gervais

On Fri, May 13, 2016 at 12:55 AM, Charles Tassell <ctass...@gmail.com 
<mailto:ctass...@gmail.com>> wrote:


Hi Gervais,

  Hmm, can you tar up the logfiles (/var/log/vdsm/* on the host
you are installing on) and put them somewhere to look at?  Also, I
found that starting VDSM from the command line is useful as it
sometimes spits out error messages that don't show up in the
logs.  I think the command I used was:
su -s /bin/bash vdsm
python /usr/share/vdsm/vdsm

My problem was that I customized the logging settings in
/etc/vdsm/*conf to try and tone down the debugging stuff and had a
syntax error.


On 16-05-12 10:24 PM, Gervais de Montbrun wrote:

Hi Charles,

Thanks for the suggestion.

I cleaned up again using the bash script from the
recoving-from-failed-install link below, then reinstalled (yum
install ovirt-hosted-engine-setup).

I enabled NetworkManager and firewalld as you suggested. The
install stops very early on with an error:
[ ERROR ] Failed to execute stage 'Programs detection':
hosted-engine cannot be deployed while NetworkManager is running,
please stop and disable it before proceeding

I disabled and stopped NetworkManager and tried again. Same
result. :(

Any more guesses?

Cheers,
Gervais




On May 12, 2016, at 9:08 PM, Charles Tassell <ctass...@gmail.com
<mailto:ctass...@gmail.com>> wrote:

Hey Gervais,

Try enabling NetworkManager and firewalld before doing the
hosted-engine --deploy.  I have run into problems with oVirt
trying to perform tasks on hosts where firewalld is disabled, so
maybe you are running into a similar problem.  Also, I think the
setup script will disable NetworkManager if it needs to.  I know
I didn't manually disable it on any of the boxes I installed on.

On 16-05-12 04:49 PM, users-requ...@ovirt.org
<mailto:users-requ...@ovirt.org> wrote:

Message: 1
Date: Thu, 12 May 2016 14:22:12 -0300
From: Gervais de Montbrun <gerv...@demontbrun.com
<mailto:gerv...@demontbrun.com>>
To: Wee Sritippho <we...@forest.go.th <mailto:we...@forest.go.th>>
Cc: users <users@ovirt.org <mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] Adding another host to my cluster
Message-ID:
<28b7fc74-5c52-4f60-b9f3-39a36621a...@demontbrun.com
<mailto:28b7fc74-5c52-4f60-b9f3-39a36621a...@demontbrun.com>>
Content-Type: text/plain; charset="utf-8"

Hi Wee
(and others)

Thanks for the reply. I tried what you suggested, but I am in
the exact same state. :-(

I don't want to completely remove my hosted engine setup as it
is working on the two other hosts in my cluster. I did not run
the rm -rf stes listed here

(https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install

<https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install>)
that would wipe my hosted_engine nfs mount. If you know that
this is 100% necessary, please let me know.

I did:
hosted-engine --clean-metadata --force-cleanup --host-id=3
run the bash script to remove all of the ovirt packages and
config files
reinstalled ovirt-hosted-engine-setup
ran "hosted-engine --deploy"

I'm back exactly where I started. Is there a way to run just
the network configuration part of the deploy?

Since the last attempt, I did upgrade my hosted engine and my
cluster is now running oVirt 3.6.5.

Cheers,
Gervais




On May 12, 2016, at 11:50 AM, Wee Sritippho
<we...@forest.go.th <mailto:we...@forest.go.th>> wrote:

Hi,

I used to have a similar problem where one of my host can't be
deployed due to the absence of ovirtmgmt bridge. Simone said
it's a bug ( https://bugzilla.redhat.com/1323465
<https://bugzilla.redhat.com/1323465> ) which would be fixed
in 3.6.6.

This is what I've done to solve it:

1. In the web UI, set the failed host to maintenance.
2. Remove it.
3. In that host, run a script from

https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install

<https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install>
4. Install ovirt-hosted-engine-setup again.
5. Redeploy again.

Hope that helps

On 11 ??? 2016 22 ?? 48  

Re: [ovirt-users] Adding another host to my cluster

[Charles Tassell]

Hi Gervais,

  Hmm, can you tar up the logfiles (/var/log/vdsm/* on the host you are 
installing on) and put them somewhere to look at?  Also, I found that 
starting VDSM from the command line is useful as it sometimes spits out 
error messages that don't show up in the logs.  I think the command I 
used was:

su -s /bin/bash vdsm
python /usr/share/vdsm/vdsm

My problem was that I customized the logging settings in /etc/vdsm/*conf 
to try and tone down the debugging stuff and had a syntax error.


On 16-05-12 10:24 PM, Gervais de Montbrun wrote:

Hi Charles,

Thanks for the suggestion.

I cleaned up again using the bash script from the 
recoving-from-failed-install link below, then reinstalled (yum install 
ovirt-hosted-engine-setup).


I enabled NetworkManager and firewalld as you suggested. The install 
stops very early on with an error:
[ ERROR ] Failed to execute stage 'Programs detection': hosted-engine 
cannot be deployed while NetworkManager is running, please stop and 
disable it before proceeding


I disabled and stopped NetworkManager and tried again. Same result. :(

Any more guesses?

Cheers,
Gervais



On May 12, 2016, at 9:08 PM, Charles Tassell <ctass...@gmail.com 
<mailto:ctass...@gmail.com>> wrote:


Hey Gervais,

Try enabling NetworkManager and firewalld before doing the 
hosted-engine --deploy.  I have run into problems with oVirt trying 
to perform tasks on hosts where firewalld is disabled, so maybe you 
are running into a similar problem. Also, I think the setup script 
will disable NetworkManager if it needs to.  I know I didn't manually 
disable it on any of the boxes I installed on.


On 16-05-12 04:49 PM, users-requ...@ovirt.org 
<mailto:users-requ...@ovirt.org> wrote:

Message: 1
Date: Thu, 12 May 2016 14:22:12 -0300
From: Gervais de Montbrun <gerv...@demontbrun.com 
<mailto:gerv...@demontbrun.com>>

To: Wee Sritippho <we...@forest.go.th <mailto:we...@forest.go.th>>
Cc: users <users@ovirt.org <mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] Adding another host to my cluster
Message-ID: <28b7fc74-5c52-4f60-b9f3-39a36621a...@demontbrun.com 
<mailto:28b7fc74-5c52-4f60-b9f3-39a36621a...@demontbrun.com>>

Content-Type: text/plain; charset="utf-8"

Hi Wee
(and others)

Thanks for the reply. I tried what you suggested, but I am in the 
exact same state. :-(


I don't want to completely remove my hosted engine setup as it is 
working on the two other hosts in my cluster. I did not run the rm 
-rf stes listed here 
(https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install 
<https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install>) 
that would wipe my hosted_engine nfs mount. If you know that this is 
100% necessary, please let me know.


I did:
hosted-engine --clean-metadata --force-cleanup --host-id=3
run the bash script to remove all of the ovirt packages and config files
reinstalled ovirt-hosted-engine-setup
ran "hosted-engine --deploy"

I'm back exactly where I started. Is there a way to run just the 
network configuration part of the deploy?


Since the last attempt, I did upgrade my hosted engine and my 
cluster is now running oVirt 3.6.5.


Cheers,
Gervais



On May 12, 2016, at 11:50 AM, Wee Sritippho <we...@forest.go.th 
<mailto:we...@forest.go.th>> wrote:


Hi,

I used to have a similar problem where one of my host can't be 
deployed due to the absence of ovirtmgmt bridge. Simone said it's a 
bug ( https://bugzilla.redhat.com/1323465 
<https://bugzilla.redhat.com/1323465> ) which would be fixed in 3.6.6.


This is what I've done to solve it:

1. In the web UI, set the failed host to maintenance.
2. Remove it.
3. In that host, run a script from 
https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install 
<https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install>

4. Install ovirt-hosted-engine-setup again.
5. Redeploy again.

Hope that helps

On 11 ??? 2016 22 ?? 48  58 ?? GMT+07:00, Gervais 
de Montbrun <gerv...@demontbrun.com 
<mailto:gerv...@demontbrun.com>> wrote:

Hi Folks,

I hate to reply to my own message, but I'm really hoping someone 
can help me with my issue
http://lists.ovirt.org/pipermail/users/2016-May/039690.html 
<http://lists.ovirt.org/pipermail/users/2016-May/039690.html>


Does anyone have a suggestion for me? If there is any more 
information that I can provide that would help you to help me, 
please advise.


Cheers,
Gervais



On May 9, 2016, at 1:42 PM, Gervais de Montbrun 
<gerv...@demontbrun.com <mailto:gerv...@demontbrun.com> 
<mailto:gerv...@demontbrun.com <mailto:gerv...@demontbrun.com>>> 
wrote:


Hi All,

I'm trying to add a third host into my oVirt cluster. I have 
hosted engine setup on the first two. It's failing to finish the 
hosted-engine --deplo

Re: [ovirt-users] Adding another host to my cluster

[Charles Tassell]

Hey Gervais,

  Try enabling NetworkManager and firewalld before doing the 
hosted-engine --deploy.  I have run into problems with oVirt trying to 
perform tasks on hosts where firewalld is disabled, so maybe you are 
running into a similar problem.  Also, I think the setup script will 
disable NetworkManager if it needs to.  I know I didn't manually disable 
it on any of the boxes I installed on.


On 16-05-12 04:49 PM, users-requ...@ovirt.org wrote:

Message: 1
Date: Thu, 12 May 2016 14:22:12 -0300
From: Gervais de Montbrun 
To: Wee Sritippho 
Cc: users 
Subject: Re: [ovirt-users] Adding another host to my cluster
Message-ID: <28b7fc74-5c52-4f60-b9f3-39a36621a...@demontbrun.com>
Content-Type: text/plain; charset="utf-8"

Hi Wee
(and others)

Thanks for the reply. I tried what you suggested, but I am in the exact same 
state. :-(

I don't want to completely remove my hosted engine setup as it is working on the two 
other hosts in my cluster. I did not run the rm -rf stes listed here 
(https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install
 
)
 that would wipe my hosted_engine nfs mount. If you know that this is 100% necessary, 
please let me know.

I did:
hosted-engine --clean-metadata --force-cleanup --host-id=3
run the bash script to remove all of the ovirt packages and config files
reinstalled ovirt-hosted-engine-setup
ran "hosted-engine --deploy"

I'm back exactly where I started. Is there a way to run just the network 
configuration part of the deploy?

Since the last attempt, I did upgrade my hosted engine and my cluster is now 
running oVirt 3.6.5.

Cheers,
Gervais




On May 12, 2016, at 11:50 AM, Wee Sritippho  wrote:

Hi,

I used to have a similar problem where one of my host can't be deployed due to the 
absence of ovirtmgmt bridge. Simone said it's a bug ( 
https://bugzilla.redhat.com/1323465  ) 
which would be fixed in 3.6.6.

This is what I've done to solve it:

1. In the web UI, set the failed host to maintenance.
2. Remove it.
3. In that host, run a script from 
https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install
 

4. Install ovirt-hosted-engine-setup again.
5. Redeploy again.

Hope that helps

On 11 ??? 2016 22 ?? 48  58 ?? GMT+07:00, Gervais de Montbrun 
 wrote:
Hi Folks,

I hate to reply to my own message, but I'm really hoping someone can help me 
with my issue
http://lists.ovirt.org/pipermail/users/2016-May/039690.html 


Does anyone have a suggestion for me? If there is any more information that I 
can provide that would help you to help me, please advise.

Cheers,
Gervais




On May 9, 2016, at 1:42 PM, Gervais de Montbrun > wrote:

Hi All,

I'm trying to add a third host into my oVirt cluster. I have hosted engine 
setup on the first two. It's failing to finish the hosted-engine --deploy on 
this third host. I wiped the server and did a CentOS 7 minimum install and ran 
it again to have a clean machine.

My setup:
CentOS 7 clean install
yum install -y http://resources.ovirt.org/pub/yum-repo/ovirt-release36.rpm 

yum install -y ovirt-hosted-engine-setup
yum upgrade -y && reboot
systemctl disable NetworkManager ; systemctl stop NetworkManager ; systemctl 
disable firewalld ; systemctl stop firewalld
hosted-engine --deploy

hosted-engine --deploy always throws an error:
[ ERROR ] The VDSM host was found in a failed state. Please check engine and 
bootstrap installation logs.
[ ERROR ] Unable to add Cultivar2 to the manager
and then echo's
[ INFO  ] Waiting for VDSM hardware info
...
[ ERROR ] Failed to execute stage 'Closing up': VDSM did not start within 120 
seconds
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file 
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20160509131103.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed: this system is not reliable, please 
check the issue, fix and redeploy
  Log file is located at 
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160509130658-qb8ev0.log

Full output of hosted-engine --deploy included in the attached zip file.
I've also included vdsm.log (There is more than one tries worth of tries in 
there).
You'll also find the ovirt-hosted-engine-setup-20160509130658-qb8ev0.log listed 
above.

This is my "test" setup. Cultivar0 is my first host and my nfs server for storage. I have 
two hosts in the setup already and everything is working fine. The host does show up in the 

Re: [ovirt-users] Users Digest, Vol 55, Issue 124

[Charles Tassell]

Hi Nick,

Try running "ls -l 
/rhev/data-center/0001-0001-0001-0001-037d/88dfff37-4ad5-446c-a957-1cf67829c9a6/images/da93350c-8926-42bf-99fa-8a7aca043fcd/22e7803d-e266-4e3e-ae34-0181f8acef8b" 
and check the ownership of the file.  It should be owned by vdsm:kvm (or 
user and group 36:36 I think) Another thing to check would be that the 
parent directories are accessible.  Try running "sudo -s /bin/bash vdsm" 
and then "cd 
/rhev/data-center/0001-0001-0001-0001-037d/88dfff37-4ad5-446c-a957-1cf67829c9a6/images" 
to make sure you can access all the parent folders.  If you can, try 
creating a temp file ("echo hello >hello") in that same directory to 
make sure that the iSCSI mount hasn't gone read only.


If none of that helps, I'd say take a look at the output of vgdisplay to 
make sure that the volume group still has free physical extents (Free 
PE/Size near the bottom)  If you are using thin provisioning that might 
be an issue.


This is all based on the issue being disk permission errors.  It might 
be an error with the libvirt authentication, but I have no idea how to 
diagnose that.



On 16-04-18 05:32 AM, users-requ...@ovirt.org wrote:

Message: 2
Date: Mon, 18 Apr 2016 10:32:40 +0200
From: "nicola.gentile.to"
To:Users@ovirt.org
Subject: [ovirt-users] any virtual machine does not start
Message-ID:<57149ba8.90...@gmail.com>
Content-Type: text/plain; charset="iso-8859-15"; Format="flowed"

hi,
I have a problem with ovirt 3.6.
When I click on the button "run" to start vm, it not start and terminate
immediatly.

This is the error in the admin portale:

VM ubuexp is down with error. Exit message: internal error: process
exited while connecting to monitor: 2016-04-18T08:18:22.499782Z
qemu-kvm: -drive
file=/rhev/data-center/0001-0001-0001-0001-037d/88dfff37-4ad5-446c-a957-1cf67829c9a6/images/da93350c-8926-42bf-99fa-8a7aca043fcd/22e7803d-e266-4e3e-ae34-0181f8acef8b,if=none,id=drive-virtio-disk0,format=qcow2,serial=da93350c-8926-42bf-99fa-8a7aca043fcd,cache=none,werror=stop,rerror=stop,aio=native:
Could not open
'/rhev/data-center/0001-0001-0001-0001-037d/88dfff37-4ad5-446c-a957-1cf67829c9a6/images/da93350c-8926-42bf-99fa-8a7aca043fcd/22e7803d-e266-4e3e-ae34-0181f8acef8b':
Permission denied
.

My ovirt installation is composed of thefollowing:

- 1 manager (virtualized)
- 1 host
- storage iscsi

Everything seems to work well from the web administration portal, but
all vm do not start.

I attach a log file.

Please help me!

Nick


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Live migration fails

[Charles Tassell]

Hi Nick,

  I had this problem myself a while ago and it turned out the issue was 
DNS related (one of the hosts couldn't do a DNS lookup on the name 
registered to the other host so it failed with a strange error.)  The 
best way to diagnose a migration failure is probably with the 
/var/log/vdsm/vdsm.log file (might be vdsmd instead of vdsm)  I'd 
recommend ssh'ing into both hosts and run the following command:


 tail -f /var/log/vdsm/vdsm.log |egrep -v 'DEBUG|INFO' |tee 
/tmp/migrate.log


Then attempt the migration.  When the GUI says the migration has failed 
hit Control-C in both windows to stop capturing the log. You can then go 
through the logfiles (stored in /tmp/migrate.log) to find the actual 
error message and post it to the list.  If you can't find the error you 
might want to upload the logfiles somewhere and post the URLs to the 
list so some of the devs or power users can better diagnose the problem.


On 16-04-14 01:00 PM, users-requ...@ovirt.org wrote:

Date: Thu, 14 Apr 2016 16:35:34 +0200
From: Sandro Bonazzola 
To: Nick Vercampt 
Cc: users 
Subject: Re: [ovirt-users] Live migration fails
Message-ID:

[ovirt-users] Importing An Ubuntu Disk

[Charles Tassell]

Hello,

  I was wondering if there is any way to import a disk image 
(qcow/qcow2) into an oVirt storage domain?  I've tried v2v but it won't 
work because the image customization parts of it won't deal with Ubuntu, 
and I tried import-to-ovirt.pl but the disks it creates seem to be 
broken in some way that prevents them from booting when attached to a VM.


  I've seen some references to creating a pre-allocated disk of the 
same size and then using dd to overwrite the contents of it, but is 
there a better method?  Or should I just import my existing VMs by 
booting off a system rescue CD and restoring backups over the network?



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] VDSM Won't Start

[Charles Tassell]

Hi Guys,

  Thanks, I found the problem.  I had edited the /etc/vdsm logger 
config files to cut down on all the debug logging and the process failed 
(silently in any system log I could find) due to a parsing error.  
Restored the logger files to defaults and it worked.


For anyone going through the list logs in the future, try running:
sudo -s /bin/bash vdsm
python /usr/share/vdsm/vdsm

to get an error message.

On 16-04-13 09:46 AM, Martin Sivak wrote:

Hi Charles,

can you please make sure you are not in local maintenance mode and the
vdsmd systemd service is enabled?

The line


Apr 13 07:57:09 ovirt-01 ovirt-ha-agent: 
ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Service vdsmd is not running and 
the admin is responsible for starting it. Waiting...

seems to indicate that hosted engine agent refused to start vdsm on
its own and that usually happens when the host is in maintenance.

Regards

--
Martin Sivak
SLA / oVirt


On Wed, Apr 13, 2016 at 2:25 PM, Charles Tassell <ctass...@gmail.com> wrote:

Hi Simon,

   I tried that, but the same problem: vdsm won't start, so the deploy can't
continue.  There is some sort of bug with running VDSMd, but I can't find
any error logging to say what that bug is so I'm at a loss as to how to fix
it.

On 16-04-13 08:47 AM, Simone Tiraboschi wrote:

[snip] You can run hosted-engine --deploy again and choose to redeploy
using the same host-id: it will reconfigure VDSM for you.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] VDSM Won't Start

[Charles Tassell]

Hi Simon,

  I tried that, but the same problem: vdsm won't start, so the deploy 
can't continue.  There is some sort of bug with running VDSMd, but I 
can't find any error logging to say what that bug is so I'm at a loss as 
to how to fix it.


On 16-04-13 08:47 AM, Simone Tiraboschi wrote:
[snip] You can run hosted-engine --deploy again and choose to redeploy 
using the same host-id: it will reconfigure VDSM for you.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] VDSM Won't Start

[Charles Tassell]

Hi Everyone,

  I'm having an issue with one of my hosts.  I was performing a yum 
update when the box crashed and rebooted, which left a lot of packages 
broken.  I resolved the issue by running "yum reinstall" on all the 
packages that were in the update, but since then VDSM won't start.  I 
can't find any error message as to why it won't start, just that it 
won't.  IE, I see this in /var/log/messages:


Apr 13 07:57:09 ovirt-01 ovirt-ha-agent: 
WARNING:ovirt_hosted_engine_ha.agent.agent.Agent:Restarting agent, 
attempt '5'
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent: 
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found 
certificate common name: ovirt-01.local.domain
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent: 
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Initializing VDSM
Apr 13 07:57:09 ovirt-01 journal: ovirt-ha-agent 
ovirt_hosted_engine_ha.agent.agent.Agent ERROR Service vdsmd is not 
running and the admin is responsible for starting it. Waiting...
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent: 
ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Service vdsmd is not 
running and the admin is responsible for starting it. Waiting...


And this in /var/log/vdsmd/mom.log (vdsmd.log is a 0 byte file)

2016-04-13 07:58:07,958 - mom - INFO - MOM starting
2016-04-13 07:58:07,992 - mom.HostMonitor - INFO - Host Monitor starting
2016-04-13 07:58:07,992 - mom - INFO - hypervisor interface vdsmxmlrpc
2016-04-13 07:58:08,089 - mom.vdsmInterface - ERROR - Cannot connect to 
VDSM! [Errno 111] Connection refused
2016-04-13 07:58:08,090 - mom.GuestManager - INFO - Guest Manager 
starting: multi-thread

2016-04-13 07:58:08,105 - mom.Policy - INFO - Loaded policy '00-defines'
2016-04-13 07:58:08,109 - mom.vdsmInterface - ERROR - Cannot connect to 
VDSM! [Errno 111] Connection refused

2016-04-13 07:58:08,169 - mom.Policy - INFO - Loaded policy '02-balloon'
2016-04-13 07:58:08,263 - mom.Policy - INFO - Loaded policy '03-ksm'
2016-04-13 07:58:08,339 - mom.HostMonitor - INFO - HostMonitor is ready
2016-04-13 07:58:08,380 - mom.Policy - INFO - Loaded policy '04-cputune'
2016-04-13 07:58:08,381 - mom.PolicyEngine - INFO - Policy Engine starting
2016-04-13 07:58:08,383 - mom.RPCServer - INFO - Using unix socket 
/var/run/vdsm/mom-vdsm.sock

2016-04-13 07:58:08,383 - mom.RPCServer - INFO - RPC Server starting
2016-04-13 07:58:08,884 - mom.RPCServer - INFO - RPC Server ending

Can anyone give me some ideas of how to debug this?  Or even some docs 
on how to properly remove the host from the manager app and re-register 
the guest to see if that fixes things up.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problems With VMWare Import

[Charles Tassell]

Actually, I guess I can't: p2v doesn't work either (it hangs because 
when it ssh's to the oVirt host and tries to run virsh there is a 
password prompt which it apparently can't deal with.)


Has anyone successfully imported a VM from a recent version of VMWare 
(ie 5.5 or later) into oVirt?  Is there any documentation on how to do 
so?  I suppose I could just use something like CloneZilla to do it, but 
that seems like a bit of a pain, especially since we have about 50 VMs 
to move...



On 16-03-31 09:11 PM, Charles Tassell wrote:
Ah, that's a bit of a bummer but I guess I can use the p2v conversion 
for those instances.  I tried to convert another VM (this one without 
a snapshot) and got the error "virt-v2v is unable to convert this 
guest type"  I'm guessing that the problem is that our VMWare VM's are 
an the v10 format, as I know a lot of tools have issue dealing with 
the newer format.


On 16-03-31 05:51 AM, Nisim Simsolo wrote:

Hi

Can you please try do delete source VMware VM snapshot and then try 
to import it?
There is known issue for importing VMware VM with snapshot: 
https://bugzilla.redhat.com/show_bug.cgi?id=1172425 - [RFE]virt-v2v 
failed to convert VMware ESX VM with snapshot


Thanks

Nisim Simsolo
QE -Testing Engineer
IRC: nsimsolo
int phone - 8272305
mobile - 054-4779934

- Original Message -
| From: "Charles Tassell" <ctass...@gmail.com>
| To: "users" <users@ovirt.org>
| Sent: Wednesday, March 30, 2016 9:53:10 PM
| Subject: [ovirt-users] Problems With VMWare Import
|
| Hi Everyone,
|
|This doesn't seem to be a bug in oVirt so much as in the
| libvirt/virt-v2v tool it uses for importing, but I figured someone 
here

| might have run into this issue before.  I'm trying to import some VMs
| from my VMWare cluster and it's failing with a "file not found" when
| trying to download the disk images.  I have files in the datastore 
like:

|
| systest-55-01-delta.vmdk4G
| systest-55-01.vmdk  1K
| systest-55-Snapshot1.vmsn4G
| systest-55-flat.vmdk 40G
| systest-55.vmdk   1K
| ... bunch more small .vm?? files
|
| For some reason virt-v2v is trying to download a file called
| systest-55-01-flat.vmdk which doesn't exist.  I'm assuming this 
has

| something to do with the snapshots stored in the folder... Does anyone
| know a way to deal with that?  Can I just delete the snapshots or is
| that going to delete data stored on the VM since the last 
snapshot?  I'm

| using virt-v2v 1.28.1 if that makes a difference.
|
| ___
| Users mailing list
| Users@ovirt.org
| http://lists.ovirt.org/mailman/listinfo/users
|




___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Problems With VMWare Import

[Charles Tassell]

Ah, that's a bit of a bummer but I guess I can use the p2v conversion 
for those instances.  I tried to convert another VM (this one without a 
snapshot) and got the error "virt-v2v is unable to convert this guest 
type"  I'm guessing that the problem is that our VMWare VM's are an the 
v10 format, as I know a lot of tools have issue dealing with the newer 
format.


On 16-03-31 05:51 AM, Nisim Simsolo wrote:

Hi

Can you please try do delete source VMware VM snapshot and then try to import 
it?
There is known issue for importing VMware VM with snapshot: 
https://bugzilla.redhat.com/show_bug.cgi?id=1172425 - [RFE]virt-v2v failed to 
convert VMware ESX VM with snapshot

Thanks

Nisim Simsolo
QE -Testing Engineer
IRC: nsimsolo
int phone - 8272305
mobile - 054-4779934

- Original Message -
| From: "Charles Tassell" <ctass...@gmail.com>
| To: "users" <users@ovirt.org>
| Sent: Wednesday, March 30, 2016 9:53:10 PM
| Subject: [ovirt-users] Problems With VMWare Import
|
| Hi Everyone,
|
|This doesn't seem to be a bug in oVirt so much as in the
| libvirt/virt-v2v tool it uses for importing, but I figured someone here
| might have run into this issue before.  I'm trying to import some VMs
| from my VMWare cluster and it's failing with a "file not found" when
| trying to download the disk images.  I have files in the datastore like:
|
| systest-55-01-delta.vmdk4G
| systest-55-01.vmdk  1K
| systest-55-Snapshot1.vmsn4G
| systest-55-flat.vmdk 40G
| systest-55.vmdk   1K
| ... bunch more small .vm?? files
|
| For some reason virt-v2v is trying to download a file called
| systest-55-01-flat.vmdk which doesn't exist.  I'm assuming this has
| something to do with the snapshots stored in the folder...  Does anyone
| know a way to deal with that?  Can I just delete the snapshots or is
| that going to delete data stored on the VM since the last snapshot?  I'm
| using virt-v2v 1.28.1 if that makes a difference.
|
| ___
| Users mailing list
| Users@ovirt.org
| http://lists.ovirt.org/mailman/listinfo/users
|


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Problems With VMWare Import

[Charles Tassell]

Hi Everyone,

  This doesn't seem to be a bug in oVirt so much as in the 
libvirt/virt-v2v tool it uses for importing, but I figured someone here 
might have run into this issue before.  I'm trying to import some VMs 
from my VMWare cluster and it's failing with a "file not found" when 
trying to download the disk images.  I have files in the datastore like:


systest-55-01-delta.vmdk4G
systest-55-01.vmdk  1K
systest-55-Snapshot1.vmsn4G
systest-55-flat.vmdk 40G
systest-55.vmdk   1K
... bunch more small .vm?? files

For some reason virt-v2v is trying to download a file called 
systest-55-01-flat.vmdk which doesn't exist.  I'm assuming this has 
something to do with the snapshots stored in the folder...  Does anyone 
know a way to deal with that?  Can I just delete the snapshots or is 
that going to delete data stored on the VM since the last snapshot?  I'm 
using virt-v2v 1.28.1 if that makes a difference.


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Diagnosing Migration Error

[Charles Tassell]

Hi Guys,

  Yep, that was the problem, thank you very much. :)  I had the DNS 
entries setup as virt-X but the hostnames were registered as ovirt-X so 
it failed.  Migration now works perfectly. :)


On 16-03-18 04:47 PM, Nir Soffer wrote:

On Fri, Mar 18, 2016 at 9:37 PM, Nir Soffer <nsof...@redhat.com> wrote:

On Fri, Mar 18, 2016 at 9:27 PM, Nir Soffer <nsof...@redhat.com> wrote:

On Fri, Mar 18, 2016 at 7:59 PM, Charles Tassell <char...@islandadmin.ca> wrote:

Hi Martin,

   Thanks, I checked that log and found the error " AttributeError:
'SourceThread' object has no attribute '_destServer'" which from what I see
from my googling was supposedly fixed...  I'm running
vdsm-4.17.23-0.el7.centos.noarch which I believe is the latest.  The full
error is:

Thread-2789::DEBUG::2016-03-18
14:47:04,200::blockSD::422::Storage.Misc.excCmd::(getReadDelay)
/usr/bin/taskset --cpu-list 0-23 /usr/bin/dd
if=/dev/383a1e7d-8b86-4a50-8bc3-e4374c57a3e5/metadata iflag=direct
of=/dev/null bs=4096 count=1 (cwd None)
Thread-2789::DEBUG::2016-03-18
14:47:04,211::blockSD::422::Storage.Misc.excCmd::(getReadDelay) SUCCESS:
 = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied,
0.000377289 s, 10.9 MB/s\n';  = 0
mailbox.SPMMonitor::DEBUG::2016-03-18
14:47:04,999::storage_mailbox::735::Storage.Misc.excCmd::(_checkForMail)
/usr/bin/taskset --cpu-list 0-23 dd
if=/rhev/data-center/0001-0001-0001-0001-02e9/mastersd/dom_md/inbox
iflag=direct,fullblock count=1 bs=1024000 (cwd None)
mailbox.SPMMonitor::DEBUG::2016-03-18
14:47:05,015::storage_mailbox::735::Storage.Misc.excCmd::(_checkForMail)
SUCCESS:  = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB)
copied, 0.0057206 s, 179 MB/s\n';  = 0
jsonrpc.Executor/0::DEBUG::2016-03-18
14:47:06,551::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest) Calling
'VM.migrate' in bridge with {u'params': {u'tunneled': u'false',
u'autoConverge': u'false', u'src': u'ovirt-01.virt.roblib.upei.ca', u'dst':
u'ovirt-02.virt.roblib.upei.ca:54321', u'vmId':
u'2be4938e-f4a3-4322-bae3-8a9628b81835', u'abortOnError': u'true',
u'compressed': u'false', u'method': u'online'}, u'vmID':
u'2be4938e-f4a3-4322-bae3-8a9628b81835'}
jsonrpc.Executor/0::DEBUG::2016-03-18 14:47:06,552::API::546::vds::(migrate)
{u'tunneled': u'false', u'autoConverge': u'false', u'src':
u'ovirt-01.virt.roblib.upei.ca', u'dst':
u'ovirt-02.virt.roblib.upei.ca:54321', u'vmId':
u'2be4938e-f4a3-4322-bae3-8a9628b81835', u'abortOnError': u'true',
u'compressed': u'false', u'method': u'online'}
jsonrpc.Executor/0::DEBUG::2016-03-18
14:47:06,555::__init__::206::jsonrpc.Notification::(emit) Sending event
{"params": {"2be4938e-f4a3-4322-bae3-8a9628b81835": {"status": "Migration
Source"}, "notify_time": 4311153220}, "jsonrpc": "2.0", "method":
"|virt|VM_status|2be4938e-f4a3-4322-bae3-8a9628b81835"}
jsonrpc.Executor/0::DEBUG::2016-03-18
14:47:06,555::__init__::533::jsonrpc.JsonRpcServer::(_serveRequest) Return
'VM.migrate' in bridge with True
Thread-41096::ERROR::2016-03-18
14:47:06,559::migration::208::virt.vm::(_recover)
vmId=`2be4938e-f4a3-4322-bae3-8a9628b81835`::[Errno -2] Name or service not
known
Thread-41096::ERROR::2016-03-18
14:47:06,559::migration::213::virt.vm::(_recover)
vmId=`2be4938e-f4a3-4322-bae3-8a9628b81835`::Failed to destroy remote VM
Traceback (most recent call last):
   File "/usr/share/vdsm/virt/migration.py", line 211, in _recover
 self._destServer.destroy(self._vm.id)
AttributeError: 'SourceThread' object has no attribute '_destServer'
Thread-41096::DEBUG::2016-03-18
14:47:06,560::__init__::206::jsonrpc.Notification::(emit) Sending event
{"params": {"2be4938e-f4a3-4322-bae3-8a9628b81835": {"status": "Migration
Source"}, "notify_time": 4311153220}, "jsonrpc": "2.0", "method":
"|virt|VM_status|2be4938e-f4a3-4322-bae3-8a9628b81835"}
Thread-41096::ERROR::2016-03-18 14:47:06,560::migration::309::virt.vm::(run)
vmId=`2be4938e-f4a3-4322-bae3-8a9628b81835`::Failed to migrate
Traceback (most recent call last):
   File "/usr/share/vdsm/virt/migration.py", line 278, in run
 self._setupVdsConnection()
   File "/usr/share/vdsm/virt/migration.py", line 143, in _setupVdsConnection
 client = self._createClient(port)
   File "/usr/share/vdsm/virt/migration.py", line 130, in _createClient
 self.remoteHost, int(port), sslctx)
   File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1269, in
create_connected_socket
 sock.connect((host, port))
   File "/usr/lib64/python2.7/site-packages/M2Crypto/SSL/Connection.py", line
181, in connect
 self.socket.connect(addr)
   File "/usr/lib64/python2.7/socket.py", line 224, in meth
 return getattr(self._sock,name)(*args)
gaierror: [Errno -2] Name or service not known

Thi

Re: [ovirt-users] Diagnosing Migration Error

[Charles Tassell]

Hi Martin,

  Thanks, I checked that log and found the error " AttributeError: 
'SourceThread' object has no attribute '_destServer'" which from what I 
see from my googling was supposedly fixed...  I'm running 
vdsm-4.17.23-0.el7.centos.noarch which I believe is the latest.  The 
full error is:


Thread-2789::DEBUG::2016-03-18 
14:47:04,200::blockSD::422::Storage.Misc.excCmd::(getReadDelay) 
/usr/bin/taskset --cpu-list 0-23 /usr/bin/dd 
if=/dev/383a1e7d-8b86-4a50-8bc3-e4374c57a3e5/metadata iflag=direct 
of=/dev/null bs=4096 count=1 (cwd None)
Thread-2789::DEBUG::2016-03-18 
14:47:04,211::blockSD::422::Storage.Misc.excCmd::(getReadDelay) SUCCESS: 
 = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 
0.000377289 s, 10.9 MB/s\n';  = 0
mailbox.SPMMonitor::DEBUG::2016-03-18 
14:47:04,999::storage_mailbox::735::Storage.Misc.excCmd::(_checkForMail) 
/usr/bin/taskset --cpu-list 0-23 dd 
if=/rhev/data-center/0001-0001-0001-0001-02e9/mastersd/dom_md/inbox 
iflag=direct,fullblock count=1 bs=1024000 (cwd None)
mailbox.SPMMonitor::DEBUG::2016-03-18 
14:47:05,015::storage_mailbox::735::Storage.Misc.excCmd::(_checkForMail) 
SUCCESS:  = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 
MB) copied, 0.0057206 s, 179 MB/s\n';  = 0
jsonrpc.Executor/0::DEBUG::2016-03-18 
14:47:06,551::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest) 
Calling 'VM.migrate' in bridge with {u'params': {u'tunneled': u'false', 
u'autoConverge': u'false', u'src': u'ovirt-01.virt.roblib.upei.ca', 
u'dst': u'ovirt-02.virt.roblib.upei.ca:54321', u'vmId': 
u'2be4938e-f4a3-4322-bae3-8a9628b81835', u'abortOnError': u'true', 
u'compressed': u'false', u'method': u'online'}, u'vmID': 
u'2be4938e-f4a3-4322-bae3-8a9628b81835'}
jsonrpc.Executor/0::DEBUG::2016-03-18 
14:47:06,552::API::546::vds::(migrate) {u'tunneled': u'false', 
u'autoConverge': u'false', u'src': u'ovirt-01.virt.roblib.upei.ca', 
u'dst': u'ovirt-02.virt.roblib.upei.ca:54321', u'vmId': 
u'2be4938e-f4a3-4322-bae3-8a9628b81835', u'abortOnError': u'true', 
u'compressed': u'false', u'method': u'online'}
jsonrpc.Executor/0::DEBUG::2016-03-18 
14:47:06,555::__init__::206::jsonrpc.Notification::(emit) Sending event 
{"params": {"2be4938e-f4a3-4322-bae3-8a9628b81835": {"status": 
"Migration Source"}, "notify_time": 4311153220}, "jsonrpc": "2.0", 
"method": "|virt|VM_status|2be4938e-f4a3-4322-bae3-8a9628b81835"}
jsonrpc.Executor/0::DEBUG::2016-03-18 
14:47:06,555::__init__::533::jsonrpc.JsonRpcServer::(_serveRequest) 
Return 'VM.migrate' in bridge with True
Thread-41096::ERROR::2016-03-18 
14:47:06,559::migration::208::virt.vm::(_recover) 
vmId=`2be4938e-f4a3-4322-bae3-8a9628b81835`::[Errno -2] Name or service 
not known
Thread-41096::ERROR::2016-03-18 
14:47:06,559::migration::213::virt.vm::(_recover) 
vmId=`2be4938e-f4a3-4322-bae3-8a9628b81835`::Failed to destroy remote VM

Traceback (most recent call last):
  File "/usr/share/vdsm/virt/migration.py", line 211, in _recover
self._destServer.destroy(self._vm.id)
AttributeError: 'SourceThread' object has no attribute '_destServer'
Thread-41096::DEBUG::2016-03-18 
14:47:06,560::__init__::206::jsonrpc.Notification::(emit) Sending event 
{"params": {"2be4938e-f4a3-4322-bae3-8a9628b81835": {"status": 
"Migration Source"}, "notify_time": 4311153220}, "jsonrpc": "2.0", 
"method": "|virt|VM_status|2be4938e-f4a3-4322-bae3-8a9628b81835"}
Thread-41096::ERROR::2016-03-18 
14:47:06,560::migration::309::virt.vm::(run) 
vmId=`2be4938e-f4a3-4322-bae3-8a9628b81835`::Failed to migrate

Traceback (most recent call last):
  File "/usr/share/vdsm/virt/migration.py", line 278, in run
self._setupVdsConnection()
  File "/usr/share/vdsm/virt/migration.py", line 143, in 
_setupVdsConnection

client = self._createClient(port)
  File "/usr/share/vdsm/virt/migration.py", line 130, in _createClient
self.remoteHost, int(port), sslctx)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1269, in 
create_connected_socket

sock.connect((host, port))
  File "/usr/lib64/python2.7/site-packages/M2Crypto/SSL/Connection.py", 
line 181, in connect

self.socket.connect(addr)
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
gaierror: [Errno -2] Name or service not known

On 16-03-18 01:22 PM, Martin Sivak wrote:

Hi Charles,

you can probably get much more detailed explanation from the vdsm and
libvirt logs from the two hosts. Take a look at
/var/log/vdsm/vdsm.log.

Best regards

Martin Sivak
SLA / oVirt


On Fri, Mar 18, 2016 at 3:37 PM, Charles Tassell <char...@islandadmin.ca> wrote:

Hello,

   I'm trying to test migrating a VM from one host to another and it's
failing, but with no apparent error message.  I

[ovirt-users] Diagnosing Migration Error

[Charles Tassell]

Hello,

  I'm trying to test migrating a VM from one host to another and it's 
failing, but with no apparent error message.  I took a look at the 
server logs (which I've copied below) but there doesn't seem to be 
anything in here that would help track down the problem.  Should I be 
looking somewhere besides engine.log on the hosted engine? This is the 
latest version of oVirt (3.6.3.4-1.el7.centos) that I just upgraded this 
morning.  The VM has an ISO attached over NFS storage and the disk image 
is on fiber channel.


2016-03-18 14:31:18,913 INFO 
[org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-25) 
[549545f2] Lock Acquired to object 
'EngineLock:{exclusiveLocks='[2be4938e-f4a3-4322-bae3-8a9628b81835=]', 
sharedLocks='null'}'
2016-03-18 14:31:19,131 INFO 
[org.ovirt.engine.core.bll.MigrateVmToServerCommand] 
(org.ovirt.thread.pool-8-thread-45) [549545f2] Running command: 
MigrateVmToServerCommand internal: false. Entities affected :  ID: 
2be4938e-f4a3-4322-bae3-8a9628b81835 Type: VMAction group MIGRATE_VM 
with role type USER
2016-03-18 14:31:19,186 INFO 
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] 
(org.ovirt.thread.pool-8-thread-45) [549545f2] START, MigrateVDSCommand( 
MigrateVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vmId='2be4938e-f4a3-4322-bae3-8a9628b81835', 
srcHost='ovirt-01.virt.roblib.upei.ca', 
dstVdsId='1200a78f-6d05-4e5e-9ef7-6798cf741310', 
dstHost='ovirt-02.virt.roblib.upei.ca:54321', migrationMethod='ONLINE', 
tunnelMigration='false', migrationDowntime='0', autoConverge='false', 
migrateCompressed='false', consoleAddress='null'}), log id: 8f303ba
2016-03-18 14:31:19,188 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] 
(org.ovirt.thread.pool-8-thread-45) [549545f2] START, 
MigrateBrokerVDSCommand(HostName = oVirt-01, 
MigrateVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vmId='2be4938e-f4a3-4322-bae3-8a9628b81835', 
srcHost='ovirt-01.virt.roblib.upei.ca', 
dstVdsId='1200a78f-6d05-4e5e-9ef7-6798cf741310', 
dstHost='ovirt-02.virt.roblib.upei.ca:54321', migrationMethod='ONLINE', 
tunnelMigration='false', migrationDowntime='0', autoConverge='false', 
migrateCompressed='false', consoleAddress='null'}), log id: 792d7e9e
2016-03-18 14:31:19,455 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] 
(org.ovirt.thread.pool-8-thread-45) [549545f2] FINISH, 
MigrateBrokerVDSCommand, log id: 792d7e9e
2016-03-18 14:31:19,470 INFO 
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] 
(org.ovirt.thread.pool-8-thread-45) [549545f2] FINISH, 
MigrateVDSCommand, return: MigratingFrom, log id: 8f303ba
2016-03-18 14:31:19,498 INFO 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(org.ovirt.thread.pool-8-thread-45) [549545f2] Correlation ID: 549545f2, 
Job ID: 3f97f737-d32a-4ffa-b213-122dcd3ca048, Call Stack: null, Custom 
Event ID: -1, Message: Migration started (VM: cdTest02, Source: 
oVirt-01, Destination: oVirt-02, User: admin@internal).
2016-03-18 14:31:28,055 INFO 
[org.ovirt.engine.core.vdsbroker.VmAnalyzer] 
(DefaultQuartzScheduler_Worker-39) [] VM 
'2be4938e-f4a3-4322-bae3-8a9628b81835'(cdTest02) moved from 
'MigratingFrom' --> 'Up'
2016-03-18 14:31:28,055 INFO 
[org.ovirt.engine.core.vdsbroker.VmAnalyzer] 
(DefaultQuartzScheduler_Worker-39) [] Adding VM 
'2be4938e-f4a3-4322-bae3-8a9628b81835' to re-run list
2016-03-18 14:31:28,114 ERROR 
[org.ovirt.engine.core.vdsbroker.VmsMonitoring] 
(DefaultQuartzScheduler_Worker-39) [] Rerun VM 
'2be4938e-f4a3-4322-bae3-8a9628b81835'. Called from VDS 'oVirt-01'
2016-03-18 14:31:28,202 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] 
(org.ovirt.thread.pool-8-thread-46) [] START, 
MigrateStatusVDSCommand(HostName = oVirt-01, 
MigrateStatusVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vmId='2be4938e-f4a3-4322-bae3-8a9628b81835'}), log id: 618baaa6
2016-03-18 14:31:29,209 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] 
(org.ovirt.thread.pool-8-thread-46) [] FINISH, MigrateStatusVDSCommand, 
log id: 618baaa6
2016-03-18 14:31:29,250 ERROR 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(org.ovirt.thread.pool-8-thread-46) [] Correlation ID: 549545f2, Job ID: 
3f97f737-d32a-4ffa-b213-122dcd3ca048, Call Stack: null, Custom Event ID: 
-1, Message: Migration failed  (VM: cdTest02, Source: oVirt-01, 
Destination: oVirt-02).
2016-03-18 14:31:29,262 INFO 
[org.ovirt.engine.core.bll.MigrateVmToServerCommand] 
(org.ovirt.thread.pool-8-thread-46) [] Lock freed to object 
'EngineLock:{exclusiveLocks='[2be4938e-f4a3-4322-bae3-8a9628b81835=]', 
sharedLocks='null'}'

___
Users mailing list
Users@ovirt.org

[ovirt-users] 3.5 or 3.6 For Production System?

[Charles Tassell]

Hi Folks,

  I'm setting up a small virtualization system that will start with one 
or two hosts and then probably grow to 4-5 hosts.  We will eventually be 
using a shared iSCSI datastore, but right now I'll probably just use 
NFS4.  I'm wondering if I should stick with my plan of using oVirt 
3.6.2, or should I start of with 3.5 and upgrade at a later date.  Is 
3.6 generally stable enough for production use?  We're not doing 
anything very complicated, just running a few Linux webserver VMs.  No 
high availability or auto-deployment type stuff.

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Users Digest, Vol 52, Issue 20

[Charles Tassell]

Hi Budur,

  Try running the command "kvm-ok"  I'm not sure what package it's 
from, but it will tell you if your system supports the CPU 
virtualization commands required to run KVM.  If kvm-ok isn't installed, 
you can try running "egrep '(vmx|svm)' /proc/cpuinfo" vmx means you have 
Intel virtualization support, while svm is the AMD implementation.  IE, 
on my Intel i5 I get a bunch of lines like this: (**'s added for emphasis)


flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall 
nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology 
nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl 
**vmx** smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic 
popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt 
pln pts dtherm tpr_shadow vnmi flexpriority ept vpid



On 16-01-04 02:47 PM, users-requ...@ovirt.org wrote:

1. Re:  HA cluster (Budur Nagaraju)


--

Message: 1
Date: Tue, 5 Jan 2016 00:17:41 +0530
From: Budur Nagaraju 
To: Simone Tiraboschi 
Cc: users 
Subject: Re: [ovirt-users] HA cluster
Message-ID:

Re: [ovirt-users] Migration Failure With FibreChannel+NFS

[Charles Tassell]

Hi Roy,

  Hmm, doesn't look like 3.6.2 is out yet.  Is the RC stable enough 
that installing it won't cause other issues?  I tried the 3.6.1 RC last 
month and it broke everything pretty badly...  Ended up reinstalling.  
Or is there a specific package I can try upgrading from the RC with rpm 
to test with?


On 16-01-03 04:58 AM, Roy Golan wrote:

[snip]

That was a bug in handling FC in 3.6.1. Can you upgrade to 3.6.2?

On 16-01-02 06:05 PM, Roy Golan wrote:



[snip]
The running engine can't detect your hosted_storage domain and
that is why this validation error is thrown.

I think you called yours *oV**irt_FC01*. What you can do is:

  * $ psql engine engine -c "select

fn_db_add_config_value('HostedEngineStorageDomainName','oVirt_FC01','general');

  * restart the engine

I submitted https://gerrit.ovirt.org/51218 exposes that key to
engine-config


___
Users mailing list
Users@ovirt.org 
http://lists.ovirt.org/mailman/listinfo/users







___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Migration Failure With FibreChannel+NFS

[Charles Tassell]

Hi Roy,

  Thanks, I tried that (I had to run the command as postgres as when I 
tried the engine user it gave me the error: Peer authentication failed 
for user "engine"


  I restarted the engine with systemctl restart ovirt-engine but now I 
get this big error below.  Ot looks like it's trying to get a list of 
disks from one of the hosts and failing.  I'm thinking that this might 
be because the way our FC SAN works the Linux boxes can see the drives 
assigned to other machines on the bus but can't access them.  So I see 
/dev/sda through /dev/sdo but I can only read 1-2 drives.  Is there a 
way to blacklist (or only whitelist) drives?  I looked at an example 
vdsm.conf and there seems to be an irs.lvm_dev_whitelist but no docs on 
what it does.


  Or, you know, the problem might be something else entirely.

2016-01-03 08:02:31,132 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] 
(DefaultQuartzScheduler_Worker-57) [] START, FullListVDSCommand(HostName 
= , FullListVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vds='Host[,cbfd733b-8ced-487d-8754-a2217ce1210f]', 
vmIds='[2beb0a49-6f2a-460a-b253-d3fcc7b68d31]'}), log id: 1049baa8
2016-01-03 08:02:32,159 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] 
(DefaultQuartzScheduler_Worker-57) [] FINISH, FullListVDSCommand, 
return: [{status=Up, nicModel=rtl8139,pv, emulatedMachine=pc, 
guestDiskMapping={96768549-c104-4e9c-a={name=/dev/vda}, 
QEMU_DVD-ROM={name=/dev/sr0}}, 
vmId=2beb0a49-6f2a-460a-b253-d3fcc7b68d31, pid=9358, 
devices=[Ljava.lang.Object;@348c6eb9, smp=2, vmType=kvm, displayIp=0, 
display=vnc, displaySecurePort=-1, memSize=4096, displayPort=5900, 
cpuType=Westmere, 
spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, 
statusTime=4553600500, vmName=HostedEngine, clientIp=, 
pauseCode=NOERR}], log id: 1049baa8
2016-01-03 08:02:32,173 INFO 
[org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] 
(org.ovirt.thread.pool-8-thread-6) [2d59180c] START, 
GetExistingStorageDomainListQuery(GetExistingStorageDomainListParameters:{refresh='true', 
filtered='false'}), log id: 7f05a5a1
2016-01-03 08:02:32,174 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] 
(org.ovirt.thread.pool-8-thread-6) [2d59180c] START, 
HSMGetStorageDomainsListVDSCommand(HostName = oVirt-01, 
HSMGetStorageDomainsListVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
storagePoolId='----', 
storageType='null', storageDomainType='Data', path='null'}), log id: 
655dd346
2016-01-03 08:02:33,744 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] 
(org.ovirt.thread.pool-8-thread-6) [2d59180c] FINISH, 
HSMGetStorageDomainsListVDSCommand, return: 
[383a1e7d-8b86-4a50-8bc3-e4374c57a3e5, 
8dfcc83b-64d7-4ff7-9800-02ba3430ea7b], log id: 655dd346
2016-01-03 08:02:33,767 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] 
(org.ovirt.thread.pool-8-thread-6) [2d59180c] START, 
HSMGetStorageDomainInfoVDSCommand(HostName = oVirt-01, 
HSMGetStorageDomainInfoVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
storageDomainId='8dfcc83b-64d7-4ff7-9800-02ba3430ea7b'}), log id: 7100694d
2016-01-03 08:02:35,085 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] 
(org.ovirt.thread.pool-8-thread-6) [2d59180c] FINISH, 
HSMGetStorageDomainInfoVDSCommand, return: 
, log id: 7100694d
2016-01-03 08:02:35,085 INFO 
[org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] 
(org.ovirt.thread.pool-8-thread-6) [2d59180c] FINISH, 
GetExistingStorageDomainListQuery, log id: 7f05a5a1
2016-01-03 08:02:35,085 INFO 
[org.ovirt.engine.core.bll.ImportHostedEngineStorageDomainCommand] 
(org.ovirt.thread.pool-8-thread-6) [] Running command: 
ImportHostedEngineStorageDomainCommand internal: true.
2016-01-03 08:02:35,090 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] 
(org.ovirt.thread.pool-8-thread-6) [] START, 
GetDeviceListVDSCommand(HostName = oVirt-01, 
GetDeviceListVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', storageType='FCP', 
checkStatus='false', lunIds='null'}), log id: 19c10b2f
2016-01-03 08:02:35,618 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] 
(org.ovirt.thread.pool-8-thread-6) [] FINISH, GetDeviceListVDSCommand, 
return: [LUNs:{id='360060160d5732300f01ea0bed593e511', 
physicalVolumeId='TgIsrj-y7Yl-XlBr-JJK1-s30x-s7Ay-Y7CwmZ', 
volumeGroupId='vN3q6f-JEgO-1pod-Cfjf-wTTZ-vtO3-wHHR5g', 
serial='SDGC_RAID_5_APM00105002984', lunMapping='2', vendorId='DGC', 
productId='RAID 5', lunConnections='[]', deviceSize='20', pvSize='19', 
vendorName='DGC', 

[ovirt-users] Migration Failure With FibreChannel+NFS

[Charles Tassell]

Hi Everyone,

  I've been playing around with oVirt 3.6.1 to see if we can use it to 
replace VMWare, and I'm running into a problem with live migrations.  
They fail and I can't seem to find an error message that describes why 
(the error logging is VERY verbose, so maybe I'm just missing the 
important part.)
  I've setup two hosts that use a fibre channel SAN for the VM 
datastore and an NFS share for the ISO datastore.  I have a VM which is 
just booting off of a SystemRescue ISO file with a 2GB disk.  It seems 
to run fine, but when I try to migrate it to the other host I get the 
following in the engine.log of the hosted engine:


2015-12-31 09:28:20,433 INFO 
[org.ovirt.engine.core.bll.MigrateVmCommand] (default task-31) 
[61255087] Lock Acquired to object 
'EngineLock:{exclusiveLocks='[2be4938e-f4a3-4322-bae3-8a9628b81835=]', 
sharedLocks='null'}'
2015-12-31 09:28:20,526 INFO 
[org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default 
task-31) [61255087] Candidate host 'oVirt-01' 
('cbfd733b-8ced-487d-8754-a2217ce1210f') was filtered out by 
'VAR__FILTERTYPE__INTERNAL' filter 'Migration' (correlation id: null)
2015-12-31 09:28:20,646 INFO 
[org.ovirt.engine.core.bll.MigrateVmCommand] 
(org.ovirt.thread.pool-8-thread-30) [61255087] Running command: 
MigrateVmCommand internal: false. Entities affected :  ID: 
2be4938e-f4a3-4322-bae3-8a9628b81835 Type: VMAction group MIGRATE_VM 
with role type USER
2015-12-31 09:28:20,701 INFO 
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] 
(org.ovirt.thread.pool-8-thread-30) [61255087] START, MigrateVDSCommand( 
MigrateVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vmId='2be4938e-f4a3-4322-bae3-8a9628b81835', 
srcHost='ovirt-01.virt.roblib.upei.ca', 
dstVdsId='1200a78f-6d05-4e5e-9ef7-6798cf741310', 
dstHost='ovirt-02.virt.roblib.upei.ca:54321', migrationMethod='ONLINE', 
tunnelMigration='false', migrationDowntime='0', autoConverge='false', 
migrateCompressed='false', consoleAddress='null'}), log id: f2548d4
2015-12-31 09:28:20,703 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] 
(org.ovirt.thread.pool-8-thread-30) [61255087] START, 
MigrateBrokerVDSCommand(HostName = oVirt-01, 
MigrateVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vmId='2be4938e-f4a3-4322-bae3-8a9628b81835', 
srcHost='ovirt-01.virt.roblib.upei.ca', 
dstVdsId='1200a78f-6d05-4e5e-9ef7-6798cf741310', 
dstHost='ovirt-02.virt.roblib.upei.ca:54321', migrationMethod='ONLINE', 
tunnelMigration='false', migrationDowntime='0', autoConverge='false', 
migrateCompressed='false', consoleAddress='null'}), log id: 5ec26536
2015-12-31 09:28:21,435 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] 
(org.ovirt.thread.pool-8-thread-30) [61255087] FINISH, 
MigrateBrokerVDSCommand, log id: 5ec26536
2015-12-31 09:28:21,449 INFO 
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] 
(org.ovirt.thread.pool-8-thread-30) [61255087] FINISH, 
MigrateVDSCommand, return: MigratingFrom, log id: f2548d4
2015-12-31 09:28:21,504 INFO 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(org.ovirt.thread.pool-8-thread-30) [61255087] Correlation ID: 61255087, 
Job ID: ff37dfc9-f543-4e7b-983c-62cb0056959c, Call Stack: null, Custom 
Event ID: -1, Message: Migration started (VM: cdTest02, Source: 
oVirt-01, Destination: oVirt-02, User: admin@internal).
2015-12-31 09:28:22,984 WARN 
[org.ovirt.engine.core.vdsbroker.VmsMonitoring] 
(DefaultQuartzScheduler_Worker-3) [] skipping VM 
'2be4938e-f4a3-4322-bae3-8a9628b81835' from this monitoring cycle - the 
VM data has changed since fetching the data
2015-12-31 09:28:22,992 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] 
(DefaultQuartzScheduler_Worker-3) [] START, FullListVDSCommand(HostName 
= , FullListVDSCommandParameters:{runAsync='true', 
hostId='cbfd733b-8ced-487d-8754-a2217ce1210f', 
vds='Host[,cbfd733b-8ced-487d-8754-a2217ce1210f]', 
vmIds='[2beb0a49-6f2a-460a-b253-d3fcc7b68d31]'}), log id: dfa1177
2015-12-31 09:28:23,628 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] 
(DefaultQuartzScheduler_Worker-3) [] FINISH, FullListVDSCommand, return: 
[{status=Up, nicModel=rtl8139,pv, emulatedMachine=pc, 
guestDiskMapping={96768549-c104-4e9c-a={name=/dev/vda}, 
QEMU_DVD-ROM={name=/dev/sr0}}, 
vmId=2beb0a49-6f2a-460a-b253-d3fcc7b68d31, pid=9358, 
devices=[Ljava.lang.Object;@16b85d46, smp=2, vmType=kvm, displayIp=0, 
display=vnc, displaySecurePort=-1, memSize=4096, displayPort=5900, 
cpuType=Westmere, 
spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir, 
statusTime=4299531100, vmName=HostedEngine, clientIp=, 
pauseCode=NOERR}], log id: dfa1177
2015-12-31 09:28:23,636 INFO 
[org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] 
(org.ovirt.thread.pool-8-thread-32) [699a8657]