[ovirt-users] Re: Schema upgrade error while updating 4.2.4 to 4.2.5

[Jan Siml]

Hello,


So your workflow was:

- run engine-setup and got failure
was it able in your case to restore the engine db in this first run and 
start engine again?
You wrote that there was an upload failure and that you removed it: did 
you mean you removed it from the GUI, and confirming that after first 
failure the webadmin gui was still reachable?


No, WUI was down because service was stopped by engine-setup and not 
started afterwards when restore of DB backups failed. But starting 
ovirt-engine service brought engine and WUI up.


In my case after first run failure, my engine is down and trying to 
reach I get:

"
Service Unavailable

The server is temporarily unable to service your request due to 
maintenance downtime or capacity problems. Please try again later.

"

- run engine-setup again and got failure
It seems that after this second failure your engine and dwh db are down, 
so also the engine portal...

Then you manually execute the delete statement

- engine-setup again and it succeeds

Correct?


Yes, just inspected the table, deleted one row with the mentioned 
stament and sucessfully retried the upgrade.


Regards

Jan
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/GYPNS4WKUIGUV6GFKJE2W4KREWFL/

[ovirt-users] Re: Schema upgrade error while updating 4.2.4 to 4.2.5

[Jan Siml]

Hello,

On 03.08.2018 15:52, Gianluca Cecchi wrote:
On Fri, Aug 3, 2018 at 2:14 PM, Jan Siml <mailto:js...@plusline.net>> wrote:


Hello,

There are probably some more stale image transfers records,
can you please check whether 'image_transfers' table is empty?
If not, you can try removing them by executing:
"DELETE FROM image_transfers WHERE command_id NOT IN (SELECT
command_entities.command_id FROM command_entities);"
(you can simply prepend it to
'04_02_1210_add_foreign_key_to_image_transfers.sql')


that's how it worked and the update was successful. Thanks!

Jan,
are you saying that after your second update failure that failed also 
the the restore of the databases, you added the delete statement into 
the 04_02_1210_add_foreign_key_to_image_transfers.sql script?
How? because the yum rollback phase completed in my case so under 
/usr/share/ovirt-engine/dbscripts/upgrade/ I currently don't have it...

And the first update error leaves the engine down


I checked the table in psql and found only one row. Then I used the 
above statement to delete the orphaned row and retried the update. Worked.


Regards

Jan
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DRUDGDS355VKP3GP2CN2B7Z7XABP4ZOC/

[ovirt-users] Re: Schema upgrade error while updating 4.2.4 to 4.2.5

[Jan Siml]

Hello,


There are probably some more stale image transfers records,
can you please check whether 'image_transfers' table is empty?
If not, you can try removing them by executing:
"DELETE FROM image_transfers WHERE command_id NOT IN (SELECT 
command_entities.command_id FROM command_entities);"
(you can simply prepend it to 
'04_02_1210_add_foreign_key_to_image_transfers.sql')


that's how it worked and the update was successful. Thanks!

Regards

Jan
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/RQS4MHPWU7HWCZH6BIELHVGPB4BMBBGL/

[ovirt-users] Re: Schema upgrade error while updating 4.2.4 to 4.2.5

[Jan Siml]

Hello,


If there are Disk Uploads that have not completed (i.e: currently
Paused or with Error) and were not canceled the upgrade will fail.This
is due to a foreign key being added on 4.2.5, relating image_transfers
table to command_entities table.
 
When the roll-back to the previous RHV-M version finishes, open the

Administration Portal and navigate to Storage -> Disks.


there was indeed one disk upload which wasn't finshed. I have canceled 
it and removed the disk.


If it fails then share your observations with us, so we need to do 
some manual changes in database.

The seconds attempt to update failed to:

Running upgrade sql script 
'/usr/share/ovirt-engine/dbscripts/upgrade/04_02_1210_add_foreign_key_to_image_transfers.sql'...


2018-08-03 12:45:05,808+0200 DEBUG 
otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema 
plugin.execute:926 execute-output: 
['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p',
 '5432', '-u', 'engine', '-d', 'engine', '-l', 
'/var/log/ovirt-engine/setup/ovirt-engine-setup-20180803124017-k94hg1.log', 
'-c', 'apply'] stderr:
psql:/usr/share/ovirt-engine/dbscripts/upgrade/04_02_1210_add_foreign_key_to_image_transfers.sql:1: 
ERROR:  insert or update on table "image_transfers" violates foreign key 
constraint "fk_image_tran

sfers_command_enitites"
DETAIL:  Key (command_id)=(312f711f-a5c3-477d-8533-82ef88a54b77) is not 
present in table "command_entities".
FATAL: Cannot execute sql command: 
--file=/usr/share/ovirt-engine/dbscripts/upgrade/04_02_1210_add_foreign_key_to_image_transfers.sql


2018-08-03 12:45:05,809+0200 ERROR 
otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema schema._misc:434 
schema.sh: FATAL: Cannot execute sql command: 
--file=/usr/share/ovirt-engine/dbscripts/upg

rade/04_02_1210_add_foreign_key_to_image_transfers.sql
2018-08-03 12:45:05,812+0200 DEBUG otopi.context 
context._executeMethod:143 method exception

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, 
in _executeMethod

method['method']()
  File 
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/db/schema.py", 
line 436, in _misc

raise RuntimeError(_('Engine schema refresh failed'))
RuntimeError: Engine schema refresh failed
2018-08-03 12:45:05,815+0200 ERROR otopi.context 
context._executeMethod:152 Failed to execute stage 'Misc configuration': 
Engine schema refresh failed


The restore for both databases (engine and engine_hirory) fails too:

pg_restore: [archiver (db)] Error from TOC entry 7339; 0 0 COMMENT 
EXTENSION "uuid-ossp"
pg_restore: [archiver (db)] could not execute query: ERROR:  must be 
owner of extension uuid-ossp


Regards

Jan
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DSVVDV42APPUKQN6XV4ZUSI3PUAXPOWB/

[ovirt-users] Schema upgrade error while updating 4.2.4 to 4.2.5

[Jan Siml]

Hello,

I stumbled across the following bug when upgrading the ovirt engine from 
4.2.4 to 4.2.5 (snippet from upgrade log):


Running upgrade sql script 
'/usr/share/ovirt-engine/dbscripts/upgrade/04_02_1210_add_foreign_key_to_image_transfers.sql'...


2018-08-02 09:14:33,116+0200 DEBUG 
otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema 
plugin.execute:926 execute-output: 
['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', 
'5432', '-u'
, 'engine', '-d', 'engine', '-l', 
'/var/log/ovirt-engine/setup/ovirt-engine-setup-20180802091038-xclfnu.log', 
'-c', 'apply'] stderr:
psql:/usr/share/ovirt-engine/dbscripts/upgrade/04_02_1210_add_foreign_key_to_image_transfers.sql:1: 
ERROR:  insert or update on table "image_transfers" violates foreign key 
constraint "fk_image_transfers_command

_enitites"
DETAIL:  Key (command_id)=(312f711f-a5c3-477d-8533-82ef88a54b77) is not 
present in table "command_entities".
FATAL: Cannot execute sql command: 
--file=/usr/share/ovirt-engine/dbscripts/upgrade/04_02_1210_add_foreign_key_to_image_transfers.sql


2018-08-02 09:14:33,117+0200 ERROR 
otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema schema._misc:434 
schema.sh: FATAL: Cannot execute sql command: 
--file=/usr/share/ovirt-engine/dbscripts/upgrade/04_02_12

10_add_foreign_key_to_image_transfers.sql
2018-08-02 09:14:33,119+0200 DEBUG otopi.context 
context._executeMethod:143 method exception

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, 
in _executeMethod

method['method']()
  File 
"/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/db/schema.py", 
line 436, in _misc

raise RuntimeError(_('Engine schema refresh failed'))
RuntimeError: Engine schema refresh failed
2018-08-02 09:14:33,122+0200 ERROR otopi.context 
context._executeMethod:152 Failed to execute stage 'Misc configuration': 
Engine schema refresh failed


Are there already instructions on how to deal with this error and does 
anyone know why this error occurs?


Kind regards

Jan
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IHPQ2EQ52MVTVZVABGSZO2Y7RO7RCQ66/

Re: [ovirt-users] Unable to start VM after upgrade vom 4.1.9 to 4.2.1 - NPE

[Jan Siml]

Hello,


Enable network and disks on your VM than do:
Run -> ONCE Ok Ignore errors. Ok
Run
Cheeers


WTF! That worked.

Did you know, why this works and what happens in the background? Is 
there a Bugzilla bug ID for this issue?


BTW, here is the list of devices before:

engine=# select type, device, address, is_managed, is_plugged, alias 
from vm_device where vm_id in (select vm_guid from vm_static where 
vm_name='prod-hub-201');
type|device |   address 
   | is_managed | is_plugged | alias

+---+--+++
 video  | qxl   | 
   | t  | t  |
 controller | virtio-scsi   | 
   | t  | t  |
 balloon| memballoon| 
   | t  | f  | balloon0
 graphics   | spice | 
   | t  | t  |
 controller | virtio-serial | {slot=0x06, bus=0x00, domain=0x, 
type=pci, function=0x0} | t  | t  | virtio-serial0
 disk   | disk  | {slot=0x07, bus=0x00, domain=0x, 
type=pci, function=0x0} | f  | t  | virtio-disk0
 memballoon | memballoon| {slot=0x08, bus=0x00, domain=0x, 
type=pci, function=0x0} | f  | t  | balloon0
 interface  | bridge| {slot=0x03, bus=0x00, domain=0x, 
type=pci, function=0x0} | f  | t  | net0
 interface  | bridge| {slot=0x09, bus=0x00, domain=0x, 
type=pci, function=0x0} | f  | t  | net1
 controller | scsi  | {slot=0x05, bus=0x00, domain=0x, 
type=pci, function=0x0} | f  | t  | scsi0
 controller | ide   | {slot=0x01, bus=0x00, domain=0x, 
type=pci, function=0x1} | f  | t  | ide
 controller | usb   | {slot=0x01, bus=0x00, domain=0x, 
type=pci, function=0x2} | t  | t  | usb
 channel| unix  | {bus=0, controller=0, type=virtio-serial, 
port=1}| f  | t  | channel0
 channel| unix  | {bus=0, controller=0, type=virtio-serial, 
port=2}| f  | t  | channel1
 channel| spicevmc  | {bus=0, controller=0, type=virtio-serial, 
port=3}| f  | t  | channel2
 interface  | bridge| 
   | t  | t  | net1
 interface  | bridge| 
   | t  | t  | net0
 disk   | cdrom | 
   | t  | f  | ide0-1-0
 disk   | cdrom | {bus=1, controller=0, type=drive, 
target=0, unit=0}  | f  | t  | ide0-1-0
 disk   | disk  | 
   | t  | t  | virtio-disk0

(20 rows)

and afterwards:

engine=# select type, device, address, is_managed, is_plugged, alias 
from vm_device where vm_id in (select vm_guid from vm_static where 
vm_name='prod-hub-201');
type|device |   address 
   | is_managed | is_plugged | alias

+---+--+++
 channel| spicevmc  | {type=virtio-serial, bus=0, controller=0, 
port=3}| f  | t  | channel2
 channel| unix  | {type=virtio-serial, bus=0, controller=0, 
port=1}| f  | t  | channel0
 interface  | bridge| {type=pci, slot=0x04, bus=0x00, 
domain=0x, function=0x0} | t  | t  | net1
 controller | usb   | {type=pci, slot=0x01, bus=0x00, 
domain=0x, function=0x2} | t  | t  | usb
 controller | virtio-serial | {type=pci, slot=0x06, bus=0x00, 
domain=0x, function=0x0} | t  | t  | virtio-serial0
 interface  | bridge| {type=pci, slot=0x03, bus=0x00, 
domain=0x, function=0x0} | t  | t  | net0
 controller | virtio-scsi   | {type=pci, slot=0x05, bus=0x00, 
domain=0x, function=0x0} | t  | t  | scsi0
 video  | qxl   | {type=pci, slot=0x02, bus=0x00, 
domain=0x, function=0x0} | t  | t  | video0
 channel| unix  | {type=virtio-serial, bus=0, controller=0, 
port=2}| f  | t  | channel1
 balloon| memballoon| 
   | t  | t  | balloon0
 graphics   | spice | 
   | t  | t  |
 disk   | cdrom | 
   | t  | f  | ide0-1-0
 disk   | disk  | {type=pci, slot=0x07, bus=0x00, 
domain=0x, function=0x0} | t  | t  | virtio-disk0

(13 rows)

Regards

Jan
___
Users mailing list
Users@ovirt.org

Re: [ovirt-users] Unable to start VM after upgrade vom 4.1.9 to 4.2.1 - NPE

[Jan Siml]

Hello Arik,


         we have upgrade one of our oVirt engines to 4.2.1 (from
4.1.9)
         and afterwards all nodes too. The cluster compatibility
level
         has been set to 4.2.

         Now we can't start a VM after it has been powered off.
The only
         hint we found in engine.log is:

         2018-03-07 14:51:52,504+01 INFO

[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]

         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] START,
         UpdateVmDynamicDataVDSCommand(
         UpdateVmDynamicDataVDSCommandParameters:{hostId='null',
         vmId='a7bc4124-06cb-4909-9389-bcf727df1304',
         vmDynamic='org.ovirt.engine.co 

re.common.businessentities.VmDynamic@491983e9'}),


         log id: 7d49849e
         2018-03-07 14:51:52,509+01 INFO

[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]

         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] FINISH,
         UpdateVmDynamicDataVDSCommand, log id: 7d49849e
         2018-03-07 14:51:52,531+01 INFO
         [org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] START,
CreateVDSCommand(

CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b',

         vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM
         [prod-hub-201]'}), log id: 4af1f227
         2018-03-07 14:51:52,533+01 INFO

[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]

         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] START,
         CreateBrokerVDSCommand(HostName = prod-node-210,

CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b',

         vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM
         [prod-hub-201]'}), log id: 71dcc8e7
         2018-03-07 14:51:52,545+01 ERROR

[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]

         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] Failed in
         'CreateBrokerVDS' method, for vds: 'prod-node-210'; host:
         'prod-node-210': null
         2018-03-07 14:51:52,546+01 ERROR

[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]

         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] Command
         'CreateBrokerVDSCommand(HostName = prod-node-210,

CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b',

         vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM
         [prod-hub-201]'})' execution failed: null
         2018-03-07 14:51:52,546+01 INFO

[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]

         (EE-ManagedThreadFactory-engine-Thread-25)
         [f855b54a-56d9-4708-8a67-5609438ddadb] FINISH,
         CreateBrokerVDSCommand, log id: 71dcc8e7
         2018-03-07 14:51:52,546+01 ERROR
         [org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
         (EE-ManagedThreadFactory-engine-Thread-25) [f855b5
         4a-56d9-4708-8a67-5609438ddadb] Failed to create VM:
         java.lang.NullPointerException
         at

org.ovirt.engine.core.vdsbroker.builder.vminfo.LibvirtVmXmlBuilder.lambda$writeInterfaces$23(LibvirtVmXmlBuilder.java:1066)

           [vdsbroker.jar:]

         [...]

         But this doesn't lead us to the root cause. I haven't
found any
         matching bug tickets in release notes for upcoming
4.2.1. Can
         anyone help here?


     What's the mac address of that VM?
     You can find it in the UI or with:

     select mac_addr from vm_interface where vm_guid in (select
vm_guid
     from vm_static where vm_name='');


Actually, different question - does this VM has unplugged
network interface?


The VM has two NICs. Both are plugged.

The MAC addresses are 00:1a:4a:18:01:52 for nic1 and
00:1a:4a:36:01:67 for nic2.


OK, those seem like two valid mac addresses so maybe something is wrong 
with the vm devices.

Could you please 

Re: [ovirt-users] Unable to start VM after upgrade vom 4.1.9 to 4.2.1 - NPE

[Jan Siml]

Hello Arik,



we have upgrade one of our oVirt engines to 4.2.1 (from 4.1.9)
and afterwards all nodes too. The cluster compatibility level
has been set to 4.2.

Now we can't start a VM after it has been powered off. The only
hint we found in engine.log is:

2018-03-07 14:51:52,504+01 INFO
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] START,
UpdateVmDynamicDataVDSCommand(
UpdateVmDynamicDataVDSCommandParameters:{hostId='null',
vmId='a7bc4124-06cb-4909-9389-bcf727df1304',
vmDynamic='org.ovirt.engine.co

re.common.businessentities.VmDynamic@491983e9'}),
log id: 7d49849e
2018-03-07 14:51:52,509+01 INFO
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] FINISH,
UpdateVmDynamicDataVDSCommand, log id: 7d49849e
2018-03-07 14:51:52,531+01 INFO
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] START, CreateVDSCommand(

CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b',
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM
[prod-hub-201]'}), log id: 4af1f227
2018-03-07 14:51:52,533+01 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] START,
CreateBrokerVDSCommand(HostName = prod-node-210,

CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b',
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM
[prod-hub-201]'}), log id: 71dcc8e7
2018-03-07 14:51:52,545+01 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] Failed in
'CreateBrokerVDS' method, for vds: 'prod-node-210'; host:
'prod-node-210': null
2018-03-07 14:51:52,546+01 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] Command
'CreateBrokerVDSCommand(HostName = prod-node-210,

CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b',
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM
[prod-hub-201]'})' execution failed: null
2018-03-07 14:51:52,546+01 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25)
[f855b54a-56d9-4708-8a67-5609438ddadb] FINISH,
CreateBrokerVDSCommand, log id: 71dcc8e7
2018-03-07 14:51:52,546+01 ERROR
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-25) [f855b5
4a-56d9-4708-8a67-5609438ddadb] Failed to create VM:
java.lang.NullPointerException
at

org.ovirt.engine.core.vdsbroker.builder.vminfo.LibvirtVmXmlBuilder.lambda$writeInterfaces$23(LibvirtVmXmlBuilder.java:1066)
  [vdsbroker.jar:]

[...]

But this doesn't lead us to the root cause. I haven't found any
matching bug tickets in release notes for upcoming 4.2.1. Can
anyone help here?


What's the mac address of that VM?
You can find it in the UI or with:

select mac_addr from vm_interface where vm_guid in (select vm_guid
from vm_static where vm_name='');


Actually, different question - does this VM has unplugged network interface?


The VM has two NICs. Both are plugged.

The MAC addresses are 00:1a:4a:18:01:52 for nic1 and 
00:1a:4a:36:01:67 for nic2.


Regards

Jan
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Unable to start VM after upgrade vom 4.1.9 to 4.2.1 - NPE

[Jan Siml]

Hello,

we have upgrade one of our oVirt engines to 4.2.1 (from 4.1.9) and 
afterwards all nodes too. The cluster compatibility level has been set 
to 4.2.


Now we can't start a VM after it has been powered off. The only hint we 
found in engine.log is:


2018-03-07 14:51:52,504+01 INFO 
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] START, 
UpdateVmDynamicDataVDSCommand( 
UpdateVmDynamicDataVDSCommandParameters:{hostId='null', 
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', 
vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@491983e9'}), 
log id: 7d49849e
2018-03-07 14:51:52,509+01 INFO 
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] FINISH, 
UpdateVmDynamicDataVDSCommand, log id: 7d49849e
2018-03-07 14:51:52,531+01 INFO 
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] START, CreateVDSCommand( 
CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b', 
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM [prod-hub-201]'}), 
log id: 4af1f227
2018-03-07 14:51:52,533+01 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] START, 
CreateBrokerVDSCommand(HostName = prod-node-210, 
CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b', 
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM [prod-hub-201]'}), 
log id: 71dcc8e7
2018-03-07 14:51:52,545+01 ERROR 
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] Failed in 'CreateBrokerVDS' 
method, for vds: 'prod-node-210'; host: 'prod-node-210': null
2018-03-07 14:51:52,546+01 ERROR 
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] Command 
'CreateBrokerVDSCommand(HostName = prod-node-210, 
CreateVDSCommandParameters:{hostId='0add031e-c72f-473f-ab2f-4f7abd1f402b', 
vmId='a7bc4124-06cb-4909-9389-bcf727df1304', vm='VM

[prod-hub-201]'})' execution failed: null
2018-03-07 14:51:52,546+01 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) 
[f855b54a-56d9-4708-8a67-5609438ddadb] FINISH, CreateBrokerVDSCommand, 
log id: 71dcc8e7
2018-03-07 14:51:52,546+01 ERROR 
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand] 
(EE-ManagedThreadFactory-engine-Thread-25) [f855b5
4a-56d9-4708-8a67-5609438ddadb] Failed to create VM: 
java.lang.NullPointerException
at 
org.ovirt.engine.core.vdsbroker.builder.vminfo.LibvirtVmXmlBuilder.lambda$writeInterfaces$23(LibvirtVmXmlBuilder.java:1066)

 [vdsbroker.jar:]

[...]

But this doesn't lead us to the root cause. I haven't found any matching 
bug tickets in release notes for upcoming 4.2.1. Can anyone help here?


Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] oVirt Engine Default Web Page - Broken Link "Console Client Resources"

[Jan Siml]

Hello Daniel,


I just noticed that the default web page for oVirt engine
(http:///ovirt-engine/)
, at least for version 4.0.5, now
has a broken link:

Console Client Resources



as I saw on infra@ it is already addressed:

https://ovirt-jira.atlassian.net/browse/OVIRT-1085

Kind regards

Jan
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Missing Cloud-Init parameters when creating VM from template in User Portal

[Jan Siml]

Hello,

just for the record, I have reported this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1416300

Regards

Jan
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Missing Cloud-Init parameters when creating VM from template in User Portal

[Jan Siml]

Hello,

we are running oVirt Engine in version 4.0.6.3-1. While preparing 
templates I came across the following problem. The virtual machine 
created with the template should be configured via Cloud-Init on initial 
run. The checkbox 'Use Cloud-Init/Sysprep' is ticked and the necessary 
fields for provisioning via Cloud-Init are filled. When viewing/editing 
the template in Administration Portal and User Portal the checkbox is 
still ticked and fields are filled, also when creating a VM via 'VM/ New 
VM' inside Administration Portal. But not when a VM is created via 'VM/ 
New VM' inside User Portal. Checkbox is unticked and when it is 
activated all fields are unfilled.


Is this a bug or a feature? Seems to unrelated to permissions of the 
user who creates the VM, because we can reproduce it even with Super 
User permissions.


Kind regards

Jan
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Hello,

if no one has an idea how to correct the Disk/Snapshot paths in Engine 
database, I see only one possible way to solve the issue:


Stop the VM and copy image/meta files target storage to source storage 
(the one where Engine thinks the files are located). Start the VM.


Any concerns regarding this procedure? But I still hope that someone 
from oVirt team can give an advice how to correct the database entries. 
If necessary I would open a bug in Bugzilla.


Kind regards

Jan Siml


after a failed live storage migration (cause unknown) we have a
snapshot which is undeletable due to its status 'illegal' (as seen
in storage/snapshot tab). I have already found some bugs [1],[2],[3]
regarding this issue, but no way how to solve the issue within oVirt

  3.5.3.


I have attached the relevant engine.log snippet. Is there any way to
do a live merge (and therefore delete the snapshot)?

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links to [3]
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no access)


some additional informations. I have checked the images on both storages
and verified the disk paths with virsh's dumpxml.

a) The images and snapshots are on both storages.
b) The images on source storage aren't used. (modification time)
c) The images on target storage are used. (modification time)
d) virsh -r dumpxml tells me disk images are located on _target_ storage.
e) Admin interface tells me, that images and snapshot are located on
_source_ storage, which isn't true, see b), c) and d).

What can we do, to solve this issue? Is this to be corrected in database
only?

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Hello Juergen,


got exactly the same issue, with all nice side effects like performance
degradation. Until now i was not able to fix this, or to fool the engine
somehow that it whould show the image as ok again and give me a 2nd
chance to drop the snapshot.
in some cases this procedure helped (needs 2nd storage domain)
- image live migration to a different storage domain (check which
combinations are supported, iscsi - nfs domain seems unsupported. iscsi
- iscsi works)
- snapshot went into ok state, and in ~50% i was able to drop the
snapshot than. space had been reclaimed, so seems like this worked


okay, seems interesting. But I'm afraid of not knowing which image files 
Engine uses when live migration is demanded. If Engine uses the ones 
which are actually used and updates the database afterwards -- fine. But 
if the images are used that are referenced in Engine database, we will 
take a journey into the past.



other workaround is through exporting the image onto a nfs export
domain, here you can tell the engine to not export snapshots. after
re-importing everything is fine
the snapshot feature (live at least) should be avoided at all
currently simply not reliable enaugh.
your way works, too. already did that, even it was a pita to figure out
where to find what. this symlinking mess between /rhev /dev and
/var/lib/libvirt is really awesome. not.
  Jan Siml js...@plusline.net hat am 28. August 2015 um 12:56
geschrieben:
 
 
  Hello,
 
  if no one has an idea how to correct the Disk/Snapshot paths in Engine
  database, I see only one possible way to solve the issue:
 
  Stop the VM and copy image/meta files target storage to source storage
  (the one where Engine thinks the files are located). Start the VM.
 
  Any concerns regarding this procedure? But I still hope that someone
  from oVirt team can give an advice how to correct the database entries.
  If necessary I would open a bug in Bugzilla.
 
  Kind regards
 
  Jan Siml
 
   after a failed live storage migration (cause unknown) we have a
   snapshot which is undeletable due to its status 'illegal' (as seen
   in storage/snapshot tab). I have already found some bugs [1],[2],[3]
   regarding this issue, but no way how to solve the issue within oVirt
3.5.3.
  
   I have attached the relevant engine.log snippet. Is there any way to
   do a live merge (and therefore delete the snapshot)?
  
   [1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
   [2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links to [3]
   [3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no access)
  
   some additional informations. I have checked the images on both
storages
   and verified the disk paths with virsh's dumpxml.
  
   a) The images and snapshots are on both storages.
   b) The images on source storage aren't used. (modification time)
   c) The images on target storage are used. (modification time)
   d) virsh -r dumpxml tells me disk images are located on _target_
storage.
   e) Admin interface tells me, that images and snapshot are located on
   _source_ storage, which isn't true, see b), c) and d).
  
   What can we do, to solve this issue? Is this to be corrected in
database
   only?


Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Hello,


   got exactly the same issue, with all nice side effects like performance
   degradation. Until now i was not able to fix this, or to fool the
engine
   somehow that it whould show the image as ok again and give me a 2nd
   chance to drop the snapshot.
   in some cases this procedure helped (needs 2nd storage domain)
   - image live migration to a different storage domain (check which
   combinations are supported, iscsi - nfs domain seems unsupported.
iscsi
   - iscsi works)
   - snapshot went into ok state, and in ~50% i was able to drop the
   snapshot than. space had been reclaimed, so seems like this worked
 
  okay, seems interesting. But I'm afraid of not knowing which image files
  Engine uses when live migration is demanded. If Engine uses the ones
  which are actually used and updates the database afterwards -- fine. But
  if the images are used that are referenced in Engine database, we will
  take a journey into the past.
knocking on wood. so far no problems, and i used this way for sure 50
times +


This doesn't work. Engine creates the snapshots on wrong storage (old) 
and this process fails, cause the VM (qemu process) uses the images on 
other storage (new).



in cases where the live merge failed, offline merging worked in another
50%. those which fail offline, too went back to illegal snap state


I fear offline merge would cause data corruption. Because if I shut down 
the VM, the information in Engine database is still wrong. Engine thinks 
image files and snapshots are on old storage. But VM has written to the 
equal named image files on new storage. And offline merge might use the 
old files on old storage.



   other workaround is through exporting the image onto a nfs export
   domain, here you can tell the engine to not export snapshots. after
   re-importing everything is fine


Same issue as with offline merge.

Meanwhile I think, we need to shut down the VM, copy the image files 
from one storage (qemu has used before) to the other storage (the one 
Engine expects) and pray while starting the VM again.



   the snapshot feature (live at least) should be avoided at all
   currently simply not reliable enaugh.
   your way works, too. already did that, even it was a pita to figure out
   where to find what. this symlinking mess between /rhev /dev and
   /var/lib/libvirt is really awesome. not.
Jan Siml js...@plusline.net hat am 28. August 2015 um 12:56
   geschrieben:
   
   
Hello,
   
if no one has an idea how to correct the Disk/Snapshot paths in
Engine
database, I see only one possible way to solve the issue:
   
Stop the VM and copy image/meta files target storage to source
storage
(the one where Engine thinks the files are located). Start the VM.
   
Any concerns regarding this procedure? But I still hope that someone
from oVirt team can give an advice how to correct the database
entries.
If necessary I would open a bug in Bugzilla.
   
Kind regards
   
Jan Siml
   
 after a failed live storage migration (cause unknown) we have a
 snapshot which is undeletable due to its status 'illegal' (as seen
 in storage/snapshot tab). I have already found some bugs
[1],[2],[3]
 regarding this issue, but no way how to solve the issue within
oVirt
  3.5.3.

 I have attached the relevant engine.log snippet. Is there any
way to
 do a live merge (and therefore delete the snapshot)?

 [1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links
to [3]
 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no
access)

 some additional informations. I have checked the images on both
   storages
 and verified the disk paths with virsh's dumpxml.

 a) The images and snapshots are on both storages.
 b) The images on source storage aren't used. (modification time)
 c) The images on target storage are used. (modification time)
 d) virsh -r dumpxml tells me disk images are located on _target_
   storage.
 e) Admin interface tells me, that images and snapshot are
located on
 _source_ storage, which isn't true, see b), c) and d).

 What can we do, to solve this issue? Is this to be corrected in
   database
 only?


Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Hello,

got exactly the same issue, with all nice side effects like
 performance
degradation. Until now i was not able to fix this, or to fool the
  engine
somehow that it whould show the image as ok again and give me a 2nd
chance to drop the snapshot.
in some cases this procedure helped (needs 2nd storage domain)
- image live migration to a different storage domain (check which
combinations are supported, iscsi - nfs domain seems unsupported.
  iscsi
- iscsi works)
- snapshot went into ok state, and in ~50% i was able to drop the
snapshot than. space had been reclaimed, so seems like this worked
  
   okay, seems interesting. But I'm afraid of not knowing which image
 files
   Engine uses when live migration is demanded. If Engine uses the ones
   which are actually used and updates the database afterwards --
 fine. But
   if the images are used that are referenced in Engine database, we will
   take a journey into the past.
  knocking on wood. so far no problems, and i used this way for sure 50
  times +

 This doesn't work. Engine creates the snapshots on wrong storage (old)
 and this process fails, cause the VM (qemu process) uses the images on
 other storage (new).
  
 sounds like there are some other problems in your case, wrong db entries
 image - snapshot? i didnt investigate further in the vm which failed
 this process, i directly went further and exported them

Yes, engine thinks image and snapshot are on storage a, but qemu process
uses equal named images on storage b.

It seems to me, that first live storage migration was successful on qemu
level, but engine hasn't updated the database entries.

Seems to be a possible solution to correct the database entries, but I'm
not familar with the oVirt schema and won't even try it without an
advice from oVirt developers.

  in cases where the live merge failed, offline merging worked in another
  50%. those which fail offline, too went back to illegal snap state

 I fear offline merge would cause data corruption. Because if I shut down
 the VM, the information in Engine database is still wrong. Engine thinks
 image files and snapshots are on old storage. But VM has written to the
 equal named image files on new storage. And offline merge might use the
 old files on old storage.
  
 than your initial plan is an alternative. you use thin or raw on what
 kind of storage domain? but like said, manually processing is a pita due
 to the symlink mess.

We are using raw images which are thin provisioned on NFS based storage
domains. On storage b I can see an qcow formatted image file which qemu
uses and the original (raw) image which is now backing file.

other workaround is through exporting the image onto a nfs export
domain, here you can tell the engine to not export snapshots. after
re-importing everything is fine

 Same issue as with offline merge.

 Meanwhile I think, we need to shut down the VM, copy the image files
 from one storage (qemu has used before) to the other storage (the one
 Engine expects) and pray while starting the VM again.

the snapshot feature (live at least) should be avoided at all
currently simply not reliable enaugh.
your way works, too. already did that, even it was a pita to
 figure out
where to find what. this symlinking mess between /rhev /dev and
/var/lib/libvirt is really awesome. not.
 Jan Siml js...@plusline.net hat am 28. August 2015 um 12:56
geschrieben:


 Hello,

 if no one has an idea how to correct the Disk/Snapshot paths in
  Engine
 database, I see only one possible way to solve the issue:

 Stop the VM and copy image/meta files target storage to source
  storage
 (the one where Engine thinks the files are located). Start the VM.

 Any concerns regarding this procedure? But I still hope that
 someone
 from oVirt team can give an advice how to correct the database
  entries.
 If necessary I would open a bug in Bugzilla.

 Kind regards

 Jan Siml

  after a failed live storage migration (cause unknown) we have a
  snapshot which is undeletable due to its status 'illegal'
 (as seen
  in storage/snapshot tab). I have already found some bugs
  [1],[2],[3]
  regarding this issue, but no way how to solve the issue within
  oVirt
   3.5.3.
 
  I have attached the relevant engine.log snippet. Is there any
  way to
  do a live merge (and therefore delete the snapshot)?
 
  [1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
  [2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links
  to [3]
  [3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no
  access)
 
  some additional informations. I have checked the images on both
storages
  and verified the disk paths with virsh's dumpxml.
 
  a) The images and snapshots are on both storages.
  b) The images on source storage aren't used

Re: [ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Hello,


after a failed live storage migration (cause unknown) we have a
snapshot which is undeletable due to its status 'illegal' (as seen
in storage/snapshot tab). I have already found some bugs [1],[2],[3]
regarding this issue, but no way how to solve the issue within oVirt

 3.5.3.


I have attached the relevant engine.log snippet. Is there any way to
do a live merge (and therefore delete the snapshot)?

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links to [3]
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no access)


some additional informations. I have checked the images on both storages 
and verified the disk paths with virsh's dumpxml.


a) The images and snapshots are on both storages.
b) The images on source storage aren't used. (modification time)
c) The images on target storage are used. (modification time)
d) virsh -r dumpxml tells me disk images are located on _target_ storage.
e) Admin interface tells me, that images and snapshot are located on 
_source_ storage, which isn't true, see b), c) and d).


What can we do, to solve this issue? Is this to be corrected in database 
only?


Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Hello,

after a failed live storage migration (cause unknown) we have a snapshot 
which is undeletable due to its status 'illegal' (as seen in 
storage/snapshot tab). I have already found some bugs [1],[2],[3] 
regarding this issue, but no way how to solve the issue within oVirt 3.5.3.


I have attached the relevant engine.log snippet. Is there any way to do 
a live merge (and therefore delete the snapshot)?


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links to [3]
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no access)

Kind regards

Jan Siml
2015-08-26 17:31:05,203 INFO  
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] 
(DefaultQuartzScheduler_Worker-21) [598f841d] Executing Live Merge command step 
MERGE
2015-08-26 17:31:05,216 INFO  
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] 
(DefaultQuartzScheduler_Worker-21) Waiting on Live Merge child commands to 
complete
2015-08-26 17:31:05,221 INFO  [org.ovirt.engine.core.bll.MergeCommand] 
(pool-7-thread-8) [6b01a8f1] Running command: MergeCommand internal: true. 
Entities affected :  ID: 08c6d79f-28df-427e-9b0c-e9eb9ee93ecf Type: Storage
2015-08-26 17:31:05,222 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-8) 
[6b01a8f1] START, MergeVDSCommand(HostName = prod-node-09.company.net, 
MergeVDSCommandParameters{HostId = fc48853d-f70d-429a-b93b-da696739675a, 
vmId=02805d83-7ac7-4992-a332-8e342349b02b, 
storagePoolId=ab28c1c8-2f4c-4152-89cf-cd98f7e5b00d, 
storageDomainId=08c6d79f-28df-427e-9b0c-e9eb9ee93ecf, 
imageGroupId=b67eb317-e15e-45b0-bec6-099da533115c, 
imageId=a0042168-d91f-47fa-aa21-6336da0e084e, 
baseImageId=80238c98-2b50-4f06-bc4c-1db5a2947ee9, 
topImageId=a0042168-d91f-47fa-aa21-6336da0e084e, bandwidth=0}), log id: 18cfaef3
2015-08-26 17:31:05,255 ERROR 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-8) 
[6b01a8f1] Failed in MergeVDS method
2015-08-26 17:31:05,256 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-8) 
[6b01a8f1] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand 
return value 
 StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=13, mMessage=Drive 
image file could not be found]]
2015-08-26 17:31:05,256 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-8) 
[6b01a8f1] HostName = prod-node-09.company.net
2015-08-26 17:31:05,257 ERROR 
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-8) 
[6b01a8f1] Command MergeVDSCommand(HostName = prod-node-09.company.net, 
MergeVDSCommandParameters{HostId = fc48853d-f70d-429a-b93b-da696739675a, 
vmId=02805d83-7ac7-4992-a332-8e342349b02b, 
storagePoolId=ab28c1c8-2f4c-4152-89cf-cd98f7e5b00d, 
storageDomainId=08c6d79f-28df-427e-9b0c-e9eb9ee93ecf, 
imageGroupId=b67eb317-e15e-45b0-bec6-099da533115c, 
imageId=a0042168-d91f-47fa-aa21-6336da0e084e, 
baseImageId=80238c98-2b50-4f06-bc4c-1db5a2947ee9, 
topImageId=a0042168-d91f-47fa-aa21-6336da0e084e, bandwidth=0}) execution 
failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: 
Failed to MergeVDS, error = Drive image file could not be found, code = 13
2015-08-26 17:31:05,259 INFO  
[org.ovirt.engine.core.vdsbroker.vdsbroker.MergeVDSCommand] (pool-7-thread-8) 
[6b01a8f1] FINISH, MergeVDSCommand, log id: 18cfaef3
2015-08-26 17:31:05,259 ERROR [org.ovirt.engine.core.bll.MergeCommand] 
(pool-7-thread-8) [6b01a8f1] Command org.ovirt.engine.core.bll.MergeCommand 
throw Vdc Bll exception. With error message VdcBLLException: 
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: 
VDSGenericException: VDSErrorException: Failed to MergeVDS, error = Drive image 
file could not be found, code = 13 (Failed with error imageErr and code 13)
2015-08-26 17:31:05,265 ERROR [org.ovirt.engine.core.bll.MergeCommand] 
(pool-7-thread-8) [6b01a8f1] Transaction rolled-back for command: 
org.ovirt.engine.core.bll.MergeCommand.
2015-08-26 17:31:15,230 ERROR 
[org.ovirt.engine.core.bll.RemoveSnapshotSingleDiskLiveCommand] 
(DefaultQuartzScheduler_Worker-77) [598f841d] Failed child command status for 
step MERGE
2015-08-26 17:31:15,234 INFO  
[org.ovirt.engine.core.bll.RemoveSnapshotCommandCallback] 
(DefaultQuartzScheduler_Worker-77) [1ec62455] All Live Merge child commands 
have completed, status FAILED
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Delete snapshot with status illegal - live merge not possible

[Jan Siml]

Further investigations showed, that the time on the target storage went 
backwards due to a misconfigured NTP service. Someone has corrected the 
configuration while live storage migration was running and the time 
jumped backwards.


Might this be a cause for the issue?

 Original Message  
Subject: Delete snapshot with status illegal - live merge not possible
From: Jan Siml js...@plusline.net
To: users@ovirt.org
Date: 08/26/2015 06:14 PM


Hello,

after a failed live storage migration (cause unknown) we have a snapshot
which is undeletable due to its status 'illegal' (as seen in
storage/snapshot tab). I have already found some bugs [1],[2],[3]
regarding this issue, but no way how to solve the issue within oVirt 3.5.3.

I have attached the relevant engine.log snippet. Is there any way to do
a live merge (and therefore delete the snapshot)?

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1213157
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1247377 links to [3]
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1247379 (no access)

Kind regards

Jan Siml

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Engine missing snapshot which is used by VM

[Jan Siml]

Hello,

while analyzing a performance issue of a VM, I discovered the machine 
uses a snapshot which is not referenced in Engine's database.


[oVirt shell (connected)]# list disks --vm-identifier bamboo

id : 4244c296-2b81-4dff-89ab-cd7f8847576a
name   : bamboo_Disk1

[oVirt shell (connected)]# list snapshots --vm-identifier bamboo

id : 24a7de50-83ec-4e33-8f19-28b729422982
description: Active VM

Within the storage mount path I see two images:

root@prod-node-07 4244c296-2b81-4dff-89ab-cd7f8847576a]# ls
ddb30e69-73ac-454a-b9a0-b67c970958e9
ddb30e69-73ac-454a-b9a0-b67c970958e9.lease
ddb30e69-73ac-454a-b9a0-b67c970958e9.meta
e2b4e67a-616c-4be0-bb9b-6470ef9012ba
e2b4e67a-616c-4be0-bb9b-6470ef9012ba.lease
e2b4e67a-616c-4be0-bb9b-6470ef9012ba.meta

[root@prod-node-07 4244c296-2b81-4dff-89ab-cd7f8847576a]# cat *.meta
DOMAIN=fe7e2860-ab10-4010-9b80-2d40a0198594
VOLTYPE=INTERNAL
CTIME=1435308199
FORMAT=RAW
IMAGE=4244c296-2b81-4dff-89ab-cd7f8847576a
DISKTYPE=2
PUUID=----
LEGALITY=LEGAL
MTIME=0
POOL_UUID=
SIZE=1468006400
TYPE=SPARSE
DESCRIPTION=
EOF

DOMAIN=fe7e2860-ab10-4010-9b80-2d40a0198594
VOLTYPE=LEAF
CTIME=1435308200
FORMAT=COW
IMAGE=4244c296-2b81-4dff-89ab-cd7f8847576a
DISKTYPE=2
PUUID=ddb30e69-73ac-454a-b9a0-b67c970958e9
LEGALITY=LEGAL
MTIME=0
POOL_UUID=
DESCRIPTION={DiskAlias:bamboo_Disk1,DiskDescription:}
TYPE=SPARSE
SIZE=1468006400
EOF

Even if I have no clue why this has happened I would like to remove the 
snapshot and return to a consistent state. What can I do now to solve 
the problem?


--
Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Unable to reactivate host after reboot due to failed Gluster probe

[Jan Siml]

Hello,

we have a strange behavior within an oVirt cluster. Version is 3.5.1, 
engine is running on EL6 machine and hosts are using EL7 as operating 
system. The cluster uses a GlusterFS backed storage domain amongst 
others. Three of four hosts are peers in the Gluster cluster (3 bricks, 
3 replica).


When all hosts are restarted (maybe due to power outage), engine can't 
activate them again, because Gluster probe fails. The message given in 
UI is:


Gluster command [gluster peer node-03] failed on server node-03.

Checking Gluster peer and volume status on each host confirms that 
Gluster peers are known to each other and volume is up.


node-03:~ $ gluster peer status
Number of Peers: 2

Hostname: node-02
Uuid: 3fc36f55-d3a2-4efc-b2f0-31f83ed709d9
State: Peer in Cluster (Connected)

Hostname: node-01
Uuid: 18027b35-971b-4b21-bb3d-df252b4dd525
State: Peer in Cluster (Connected)

node-03:~ $ gluster volume status
Status of volume: glusterfs-1
Gluster process PortOnline  Pid
--
Brick node-01:/export/glusterfs/brick   49152   Y   12409
Brick node-02:/export/glusterfs/brick   49153   Y   9978
Brick node-03:/export/glusterfs/brick   49152   Y   10001
Self-heal Daemon on localhost   N/A Y   10003
Self-heal Daemon on node-01 N/A Y   11590
Self-heal Daemon on node-02 N/A Y   9988

Task Status of Volume glusterfs-1
--
There are no active volume tasks

Storage domain in oVirt UI is fine (active and green) and usable. But 
neither Gluster volume nor any brick is visible in UI.


If I try the command which is shown in UI it returns:

root@node-03:~ $ gluster peer probe node-03
peer probe: success. Probe on localhost not needed

root@node-03:~ $ gluster --mode=script peer probe node-03 --xml
?xml version=1.0 encoding=UTF-8 standalone=yes?
cliOutput
  opRet0/opRet
  opErrno1/opErrno
  opErrstr(null)/opErrstr
  outputProbe on localhost not needed/output
/cliOutput

Is this maybe just an engine side parsing error?

--
Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Unable to reactivate host after reboot due to failed Gluster probe

[Jan Siml]

Hello,

finally I got the nodes online. What helps was probing the not needed 
peer node-04 (no brick) from one of the other cluster nodes. When the 
node becames a Gluster peer, I am able to activate any oVirt node which 
serves bricks.


Therefore I assume, the error message which the UI returns comes from 
node-04:


root@node-04:~ $ gluster peer probe node-01
peer probe: failed: Probe returned with unknown errno 107

root@node-03:~ $ gluster peer status
Number of Peers: 2

Hostname: node-01
Uuid: 18027b35-971b-4b21-bb3d-df252b4dd525
State: Peer in Cluster (Connected)

Hostname: node-02
Uuid: 3fc36f55-d3a2-4efc-b2f0-31f83ed709d9
State: Peer in Cluster (Connected)

root@node-03:~ $ gluster peer probe node-04
peer probe: success.

root@node-03:~ $ gluster peer status
Number of Peers: 3

Hostname: node-01
Uuid: 18027b35-971b-4b21-bb3d-df252b4dd525
State: Peer in Cluster (Connected)

Hostname: node-02
Uuid: 3fc36f55-d3a2-4efc-b2f0-31f83ed709d9
State: Peer in Cluster (Connected)

Hostname: node-04
Uuid: 9cdefc68-d710-4346-93b1-76b5307e258b
State: Peer in Cluster (Connected)

This (oVirt's behavior) seems to be reproducible.

On 01/29/2015 11:10 AM, Jan Siml wrote:

Hello,

when looking into engine.log, I can see, that gluster probe returned
errno 107. But I can't figure out why:

2015-01-29 10:40:03,546 ERROR
[org.ovirt.engine.core.bll.InitVdsOnUpCommand]
(DefaultQuartzScheduler_Worker-59) [5977aac5] Could not peer probe the
gluster server node-03. Error: VdcBLLException: org.ovirt.eng
ine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException:
VDSErrorException: Failed to AddGlusterServerVDS, error = Add host failed
error: Probe returned with unknown errno 107

Just for the record: We use the /etc/hosts method because of missing
possibility to choose the network interface for Gluster. The three
Gluster peer hosts have modified /etc/hosts files with addresses binded
to a different interface than the ovirtmgmt addresses.

Example:

root@node-03:~ $ cat /etc/hosts
192.168.200.195  node-01
192.168.200.196  node-02
192.168.200.198  node-03

The /etc/hosts file on engine host isn't modified.


On 01/29/2015 10:39 AM, Jan Siml wrote:

Hello,

we have a strange behavior within an oVirt cluster. Version is 3.5.1,
engine is running on EL6 machine and hosts are using EL7 as operating
system. The cluster uses a GlusterFS backed storage domain amongst
others. Three of four hosts are peers in the Gluster cluster (3 bricks,
3 replica).

When all hosts are restarted (maybe due to power outage), engine can't
activate them again, because Gluster probe fails. The message given in
UI is:

Gluster command [gluster peer node-03] failed on server node-03.

Checking Gluster peer and volume status on each host confirms that
Gluster peers are known to each other and volume is up.

node-03:~ $ gluster peer status
Number of Peers: 2

Hostname: node-02
Uuid: 3fc36f55-d3a2-4efc-b2f0-31f83ed709d9
State: Peer in Cluster (Connected)

Hostname: node-01
Uuid: 18027b35-971b-4b21-bb3d-df252b4dd525
State: Peer in Cluster (Connected)

node-03:~ $ gluster volume status
Status of volume: glusterfs-1
Gluster processPortOnlinePid
--


Brick node-01:/export/glusterfs/brick   49152Y12409
Brick node-02:/export/glusterfs/brick49153Y9978
Brick node-03:/export/glusterfs/brick49152Y10001
Self-heal Daemon on localhostN/AY10003
Self-heal Daemon on node-01N/AY11590
Self-heal Daemon on node-02N/AY9988

Task Status of Volume glusterfs-1
--


There are no active volume tasks

Storage domain in oVirt UI is fine (active and green) and usable. But
neither Gluster volume nor any brick is visible in UI.

If I try the command which is shown in UI it returns:

root@node-03:~ $ gluster peer probe node-03
peer probe: success. Probe on localhost not needed

root@node-03:~ $ gluster --mode=script peer probe node-03 --xml
?xml version=1.0 encoding=UTF-8 standalone=yes?
cliOutput
   opRet0/opRet
   opErrno1/opErrno
   opErrstr(null)/opErrstr
   outputProbe on localhost not needed/output
/cliOutput

Is this maybe just an engine side parsing error?





--
Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Unable to reactivate host after reboot due to failed Gluster probe

[Jan Siml]

Hello,

when looking into engine.log, I can see, that gluster probe returned 
errno 107. But I can't figure out why:


2015-01-29 10:40:03,546 ERROR 
[org.ovirt.engine.core.bll.InitVdsOnUpCommand] 
(DefaultQuartzScheduler_Worker-59) [5977aac5] Could not peer probe the 
gluster server node-03. Error: VdcBLLException: org.ovirt.eng
ine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: 
VDSErrorException: Failed to AddGlusterServerVDS, error = Add host failed

error: Probe returned with unknown errno 107

Just for the record: We use the /etc/hosts method because of missing 
possibility to choose the network interface for Gluster. The three 
Gluster peer hosts have modified /etc/hosts files with addresses binded 
to a different interface than the ovirtmgmt addresses.


Example:

root@node-03:~ $ cat /etc/hosts
192.168.200.195  node-01
192.168.200.196  node-02
192.168.200.198  node-03

The /etc/hosts file on engine host isn't modified.


On 01/29/2015 10:39 AM, Jan Siml wrote:

Hello,

we have a strange behavior within an oVirt cluster. Version is 3.5.1,
engine is running on EL6 machine and hosts are using EL7 as operating
system. The cluster uses a GlusterFS backed storage domain amongst
others. Three of four hosts are peers in the Gluster cluster (3 bricks,
3 replica).

When all hosts are restarted (maybe due to power outage), engine can't
activate them again, because Gluster probe fails. The message given in
UI is:

Gluster command [gluster peer node-03] failed on server node-03.

Checking Gluster peer and volume status on each host confirms that
Gluster peers are known to each other and volume is up.

node-03:~ $ gluster peer status
Number of Peers: 2

Hostname: node-02
Uuid: 3fc36f55-d3a2-4efc-b2f0-31f83ed709d9
State: Peer in Cluster (Connected)

Hostname: node-01
Uuid: 18027b35-971b-4b21-bb3d-df252b4dd525
State: Peer in Cluster (Connected)

node-03:~ $ gluster volume status
Status of volume: glusterfs-1
Gluster processPortOnlinePid
--

Brick node-01:/export/glusterfs/brick   49152Y12409
Brick node-02:/export/glusterfs/brick49153Y9978
Brick node-03:/export/glusterfs/brick49152Y10001
Self-heal Daemon on localhostN/AY10003
Self-heal Daemon on node-01N/AY11590
Self-heal Daemon on node-02N/AY9988

Task Status of Volume glusterfs-1
--

There are no active volume tasks

Storage domain in oVirt UI is fine (active and green) and usable. But
neither Gluster volume nor any brick is visible in UI.

If I try the command which is shown in UI it returns:

root@node-03:~ $ gluster peer probe node-03
peer probe: success. Probe on localhost not needed

root@node-03:~ $ gluster --mode=script peer probe node-03 --xml
?xml version=1.0 encoding=UTF-8 standalone=yes?
cliOutput
   opRet0/opRet
   opErrno1/opErrno
   opErrstr(null)/opErrstr
   outputProbe on localhost not needed/output
/cliOutput

Is this maybe just an engine side parsing error?



--
Kind regards

Jan Siml
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users