Re: [Users] Making v2v easier?
Il 17/01/2014 17:25, users-requ...@ovirt.org ha scritto: Date: Fri, 17 Jan 2014 17:06:13 +0100 From: Sander Grendelman san...@grendelman.com To: Itamar Heim ih...@redhat.com Cc: users users@ovirt.org users@ovirt.org Subject: Re: [Users] Making v2v easier? Message-ID: CAHa6cRQzkMiZUM1615BddPsaZM=np0odnwdnyug4rpg65bw...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 On Fri, Jan 17, 2014 at 4:19 PM, Itamar Heim ih...@redhat.com wrote: I see a lot of threads about v2v pains (mostly from ESX?) I'm interested to see if we can make this simpler/easier. hear hear! if you have experience with this, please describe the steps you are using (also the source platform), Sources: - Existing KVM (virt-manager/libvirt) platform - ESX - ova/ovf templates from several sources Methods: - KVM: virt-v2v with libvirtxml option, works reasonably well, most issues are with windows guests where virt-v2v needs libguestfs-winsupport and virtio-win (RHEL only) - ESX: virt-v2v which works reasonably well_if_ the right packages (libguestfs-winsupport virtio-win) are installed. virt-v2v can be used directly from ESX/ESX host (configure .netrc first) but this is quite slow another option is to export the VM as an OVA and then import it with virt-v2v - ova/ovf templates: hit and miss with virt-v2v, especially if they contain something that is not a regular windows/linux guest. Another option is to do a direct copy of the disks on a pre-created VM, clumsy. and how you would like to see this make simpler (I'm assuming that would start from somewhere in the webadmin probably). Webadmin would be nice, but better behaviour from existing tools would be a nice start too. For example: the flow with virt-v2v is 1) Analyze source, look for disks 2) Convert/copy disks to ovirt export domain 3) Try to add virtio stuff to the copied disks on the export domain If step 3 fails ( which happens a LOT), the copied disks are removed. This is very frustrating if you just waited a couple of hours for a large VM (e.g. 200GB) to be copied:( Some kind of graceful abort/resume would be VERY welcome. Another issue with virt-v2v is that it_always_ tries to add virtio drivers. I have a virtual appliance that contains some kind of proprietary embedded OS: adding drivers will always fail, give me some option to override that and configure simple ide / e1000 hardware for the VM regarding virtio drivers, on linux, and if your kernel support them, you can recreate / regenerate initram f.s., and then you can boot your vm linux coming from vmware with virtio drivers... same process when doing p2v or v2p -- Amedeo Salvati RHC{DS,E,VA} - LPIC-3 - UCP - NCLA 11 email: ame...@oscert.net email: ame...@linux.com http://plugcomputing.it/redhatcert.php http://plugcomputing.it/lpicert.php ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Blank VM Install Console
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 18.01.2014 00:23, David Li wrote: This is my ovirt engine configuration: Configure WebSocket Proxy : True Hi, so you got the websocket proxy installed on the same machine as the engine? Did you make sure to grant access via port 6100? Also check SELinux, it might get in the way. HTH Sven -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQGcBAEBAgAGBQJS2lr+AAoJEAq0kGAWDrqlB24L/j3FNUQJ5D5CNS/gxWsf7FUF UhcPF3UkbpC8FybVTQ8LWWXUlluOi/xCoIGOAjhzeHc3+59cftYbME9K0HhzjR4p 2OUVWB4EvHuTenRJhq5cyLTSkjVinkLrN3zaIYKlz3xKrJQyBhYI8B+3iCAVFOBG ta5Qr4KZFa6YHPrS6rrT8qiuOnCMLhMaA1Iu6sAVbIgNfjFWF23Mg/W0H9bIxyR6 0zM4dtq/RFol6K/awoSPn0o54nlnrPtEXe/3p8KvpC0BU8WKguvbO1WF8igC3XGQ rkgYuk4NxFcNREOPVvjDi0G1HPbttjayXOcSQp8Y+dHePyGroZYTUhD4zxh06J55 ZyT5WyHy2pQUOp2HlcZr4hnx06jFjYCl66EF5d0QOciclpSQ00Xwj+w9dTrG4Wnz 09lG2IzpnWLUEMVnDETyhAHlS/Dww/ubzGZx2cJw2nsAcRbKRLm8vXe+DAjtPz+R vgyfhgUhY74KFWnmKssqQKhFnvbYLKAbVrpyMveT0Q== =ha4G -END PGP SIGNATURE- ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] Issues starting hosted engine VM
Hi, With the great help from sbonazzo, I managed to step past the initial bug with the hosted-engine-setup but appear to have run into another show stopper. I ran through the install process successfully up to the stage where it completed and the engine VM was to be shutdown. (The engine has already been installed on the VM and the host has been connected to the engine). The issue starts here that the host finds itself not able to start the VM up again. VDSM Logs: http://www.fpaste.org/69592/00427141/ ovirt-hosted-engine-ha agent.log http://www.fpaste.org/69595/43609139/ It seems to keep failing to start the VM.. when I restart the agent I can see the score drop to 0 after 3 boot attempts. The interesting thing seems to be in the VDSM Logs 'Virtual machine does not exist', 'code': 1}} I'm not sure where else to look. Suggestions? Cheers, Andrew ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] VM Migration failed
I looked at the logs log and only vdsm from node2 (which appears to be the src node) seems to have full info. Please attach the following complete logs: vdsm log from dst node engine log libvirt logs from both nodes. Also, can you please answer the following? libvirt and vdsm you are using? what is selinux status on both hosts (enforcing/permissive)? do you have snapshots on the vm or does it happen on a newly created disk? when you create the disk, is it from a template or is it a new image? Thanks, Dafna On 01/17/2014 10:21 PM, Itamar Heim wrote: On 01/18/2014 12:07 AM, Meital Bourvine wrote: I opened a bug about missing info in UI: https://bugzilla.redhat.com/show_bug.cgi?id=1054994 It actually failed with this error: Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::(run) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::starting migration monitor thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::libvirtconnection::108::libvirtconnection::(wrapper) Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to inquire lock: No such process Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::(cancel) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::canceling migration downtime thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::(stop) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::stopping migration monitor thread Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::(run) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::migration downtime thread exiting Thread-1415::ERROR::2014-01-17 17:01:28,410::vm::238::vm.Vm::(_recover) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to inquire lock: No such process Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::(run) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to migrate Traceback (most recent call last): File /usr/share/vdsm/vm.py, line 323, in run self._startUnderlyingMigration() File /usr/share/vdsm/vm.py, line 400, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/vm.py, line 838, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py, line 76, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-packages/libvirt.py, line 1178, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: Failed to inquire lock: No such process Thread-26::ERROR::2014-01-17 17:01:28,917::sampling::355::vm.Vm::(collect) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Stats function failed: AdvancedStatsFunction _highWrite at 0x26efb58 The problem is that it doesn't say which process... this looks like noise post the migration failing, with probably a more relevant error in libvirt log - Original Message - From: Itamar Heim ih...@redhat.com To: Edgars M. edgars.maz...@gmail.com, Meital Bourvine mbour...@redhat.com Cc: users@ovirt.org, Michal Skrivanek mskri...@redhat.com Sent: Friday, January 17, 2014 9:47:11 PM Subject: Re: [Users] VM Migration failed On 01/17/2014 06:25 PM, Edgars M. wrote: Hi Meital I tried to migrate another VM and it also failed. This is what I get in UI: Migration started (VM: nophpapp01, Source: novmnode1, Destination: novmnode2, User: edgarsm). Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nophpapp01, Source: novmnode1, Destination: novmnode2). Migration failed due to Error: Fatal error during migration (VM: nophpapp01, Source: novmnode1, Destination: novmnode2). There is nothing in /var/log/messages, neither on engine server nor nodes. See attachments for engine and vdsm logs. (I am migrating from vmnode1 to vmnode2) whatever the issue is here, can you please open a bug on trying to return more info to user on the migration error itself (for easier troubleshooting). thanks, Itamar Thanks for the help Edgars On Fri, Jan 17, 2014 at 4:46 PM, Meital Bourvine mbour...@redhat.com mailto:mbour...@redhat.com wrote: Which error are you getting in the UI? Can you please attach the full engine and vdsm logs? Also, please check if there is a relevant error in /var/log/messages *From: *Edgars M. edgars.maz...@gmail.com mailto:edgars.maz...@gmail.com *To: *users@ovirt.org mailto:users@ovirt.org *Sent: *Friday, January 17, 2014 3:42:37 PM *Subject: *[Users] VM Migration failed Hi I am experiencing issues with manual VM migration. VM fails to migrate to other node in the same Cluster. Here are some relevant engine.log entries: ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Rerun vm a31cfd62-26fc-4396-8a83-1aed68c7fd39. Called from vds novmnode1
Re: [Users] Issues starting hosted engine VM
The interesting thing - trying it with the paused option vdsm seems to create the VM hosted-engine --vm-start-paused vdsm.log http://www.fpaste.org/69604/13900482/ But I'm not sure how to then proceed to resume it. On Sat, Jan 18, 2014 at 10:23 PM, Andrew Lau and...@andrewklau.com wrote: Hi, With the great help from sbonazzo, I managed to step past the initial bug with the hosted-engine-setup but appear to have run into another show stopper. I ran through the install process successfully up to the stage where it completed and the engine VM was to be shutdown. (The engine has already been installed on the VM and the host has been connected to the engine). The issue starts here that the host finds itself not able to start the VM up again. VDSM Logs: http://www.fpaste.org/69592/00427141/ ovirt-hosted-engine-ha agent.log http://www.fpaste.org/69595/43609139/ It seems to keep failing to start the VM.. when I restart the agent I can see the score drop to 0 after 3 boot attempts. The interesting thing seems to be in the VDSM Logs 'Virtual machine does not exist', 'code': 1}} I'm not sure where else to look. Suggestions? Cheers, Andrew ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Issues starting hosted engine VM
I believe I found the issue and have reported it here https://bugzilla.redhat.com/show_bug.cgi?id=1055059 On Sat, Jan 18, 2014 at 11:33 PM, Andrew Lau and...@andrewklau.com wrote: The interesting thing - trying it with the paused option vdsm seems to create the VM hosted-engine --vm-start-paused vdsm.log http://www.fpaste.org/69604/13900482/ But I'm not sure how to then proceed to resume it. On Sat, Jan 18, 2014 at 10:23 PM, Andrew Lau and...@andrewklau.comwrote: Hi, With the great help from sbonazzo, I managed to step past the initial bug with the hosted-engine-setup but appear to have run into another show stopper. I ran through the install process successfully up to the stage where it completed and the engine VM was to be shutdown. (The engine has already been installed on the VM and the host has been connected to the engine). The issue starts here that the host finds itself not able to start the VM up again. VDSM Logs: http://www.fpaste.org/69592/00427141/ ovirt-hosted-engine-ha agent.log http://www.fpaste.org/69595/43609139/ It seems to keep failing to start the VM.. when I restart the agent I can see the score drop to 0 after 3 boot attempts. The interesting thing seems to be in the VDSM Logs 'Virtual machine does not exist', 'code': 1}} I'm not sure where else to look. Suggestions? Cheers, Andrew ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] VM Migration failed
Hi Thanks for your help. I will provide all log files a little bit later, but so far I have noticed some pattern when migration fails. Those particular VMs, which fails to migrate, were installed from ISO image, usual installation. So, I believe you can reproduce the issue by this: 1. Upload ISO image to ISO domain 2. Install new OS by booting from ISO image and check Attach CD in Boot Options 3. Delete ISO image from ISO domain 4. Try to migrate VM to another host. I have not tried this yet, but I have noticed that only those VMs fails which had been install from ISO image which is not in ISO domain anymore. I have also VMs created from templates and those VMs I can migrate just fine. I will provide more log files later. BR Edgars On Sat, Jan 18, 2014 at 1:05 PM, Dafna Ron d...@redhat.com wrote: I looked at the logs log and only vdsm from node2 (which appears to be the src node) seems to have full info. Please attach the following complete logs: vdsm log from dst node engine log libvirt logs from both nodes. Also, can you please answer the following? libvirt and vdsm you are using? what is selinux status on both hosts (enforcing/permissive)? do you have snapshots on the vm or does it happen on a newly created disk? when you create the disk, is it from a template or is it a new image? Thanks, Dafna On 01/17/2014 10:21 PM, Itamar Heim wrote: On 01/18/2014 12:07 AM, Meital Bourvine wrote: I opened a bug about missing info in UI: https://bugzilla.redhat.com/show_bug.cgi?id=1054994 It actually failed with this error: Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::(run) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::starting migration monitor thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::libvirtconnection::108::libvirtconnection::(wrapper) Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to inquire lock: No such process Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::(cancel) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::canceling migration downtime thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::(stop) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::stopping migration monitor thread Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::(run) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::migration downtime thread exiting Thread-1415::ERROR::2014-01-17 17:01:28,410::vm::238::vm.Vm::(_recover) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to inquire lock: No such process Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::(run) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Failed to migrate Traceback (most recent call last): File /usr/share/vdsm/vm.py, line 323, in run self._startUnderlyingMigration() File /usr/share/vdsm/vm.py, line 400, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/vm.py, line 838, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py, line 76, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-packages/libvirt.py, line 1178, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: Failed to inquire lock: No such process Thread-26::ERROR::2014-01-17 17:01:28,917::sampling::355::vm.Vm::(collect) vmId=`b8787906-187a-4234-a0c9-58fc4ddf2a57`::Stats function failed: AdvancedStatsFunction _highWrite at 0x26efb58 The problem is that it doesn't say which process... this looks like noise post the migration failing, with probably a more relevant error in libvirt log - Original Message - From: Itamar Heim ih...@redhat.com To: Edgars M. edgars.maz...@gmail.com, Meital Bourvine mbour...@redhat.com Cc: users@ovirt.org, Michal Skrivanek mskri...@redhat.com Sent: Friday, January 17, 2014 9:47:11 PM Subject: Re: [Users] VM Migration failed On 01/17/2014 06:25 PM, Edgars M. wrote: Hi Meital I tried to migrate another VM and it also failed. This is what I get in UI: Migration started (VM: nophpapp01, Source: novmnode1, Destination: novmnode2, User: edgarsm). Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: nophpapp01, Source: novmnode1, Destination: novmnode2). Migration failed due to Error: Fatal error during migration (VM: nophpapp01, Source: novmnode1, Destination: novmnode2). There is nothing in /var/log/messages, neither on engine server nor nodes. See attachments for engine and vdsm logs. (I am migrating from vmnode1 to vmnode2) whatever the issue is here, can you please open a bug on trying to return more info to user on the migration error itself (for easier troubleshooting). thanks, Itamar Thanks for the help Edgars On Fri, Jan 17, 2014 at 4:46 PM, Meital Bourvine mbour...@redhat.com mailto:mbour...@redhat.com wrote: Which error are you
Re: [Users] vlan on mgmt network
So I got ovirtmgmt VM ticked off. Had to remove it from all VM's then tried to add new Logical network to eth0 by drag drop but refused since ovirtmgmt was out of sync. So I synced it. and that's how I lost access to my hosted remote system. End of story. So that's it I guess, automated install doesn't work (during switch network goes dead and doesn't come back remote access is lost) so I found manual way that works. But then I need to sync it to get my ovirtmgmt not to be a VM network. And even if its not to get another network on the interface i need to sync it. I will keep trying, Thanks for the tips. P. On 17 January 2014 22:06, Itamar Heim ih...@redhat.com wrote: On 01/18/2014 12:05 AM, William Kwan wrote: I tried to define a virtual interface for vlan102, by adding ifcfg-bond0.102 manually and restarted network services. why not via ovirt-engine? On Friday, January 17, 2014 4:45 PM, Itamar Heim ih...@redhat.com wrote: On 01/17/2014 11:38 PM, William Kwan wrote: OK.. tested with bond0.102 for vlan102 I guess I can't do this with oVirt? It wouldn't pickup this interface. It still shows up eth0, eth1 and bond0 only you tried to define a vlan youself on the bond, or use a vlan via a logical network over the bond via ovirt-engine (which should work)? Will On Friday, January 17, 2014 1:15 PM, William Kwan pota...@yahoo.com mailto:pota...@yahoo.com wrote: Thanks all for the info. I'm trying the suggestions with 3.3.2-1.el6 Assaf mentioned the following which is true. You can make ovirtmgmt a non-VM, untagged (non-VLAN) network, then place as many VM, tagged networks as you'd like on the same NIC or bond. I tried a few things and I haven't been able to find a way through. Obviously or not, there are more facts/rules. Just put up a few of them here Cannot have more than one non-VLAN network on one interface Cannot have a non-VLAN VM network and VLAN-tagged networks on one interface. ( got this when I tried to drag a VLAN-tagged VM network on the same interface as the non-VLAN VM network). I'll try to test adding a bond0.vlan. I need to have bond0 - for both mgmt network and a vlan bond1 - for another subnet. Will On Thursday, January 16, 2014 1:50 PM, Juan Pablo Lorier jplor...@gmail.com mailto:jplor...@gmail.com wrote: Hi William, You can have several logical networks on the same interface. If ovirtmgmt is not a vm network you can even mix tagged and untagged networks (if not, you can't mix). You don's have to do anything besides creating the logical networks in ovirt and assign them to the interface, ovirt takes care of creating the virtual interfaces it needs. What you can't do is have more than one logical network per vlan (I think there's a feature request on this, I know I've asked it on the list:-) ). Regards, ___ Users mailing list Users@ovirt.org mailto:Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Register a disk image via oVirt REST api
On 01/17/2014 11:28 PM, Itamar Heim wrote: On 01/16/2014 07:21 PM, Satya Vempati wrote: I have a disk image (i.e. a file) that I want to register as a disk using the REST API. The REST API works with existing disks, but does the API work with disk images? can you pleas explain what do you mean by an image compared to a disk? I think that what you want is to take a file that you have that contains the image of a disk and create a disk in the system with the same content. As far as I know we don't have any direct way to do this. You will need to create an empty disk in oVirt, and then attach it to a virtual machine. Once it is attached to that virtual machine then you can write to it, but always via the virtual machine. For example, you can use the API as follows to create a new disk that isn't attached to any VM: curl \ -k \ -X POST \ -H Accept: application/xml \ -H Content-Type: application/xml \ -d disk namenewdisk/name size1073741824/size formatraw/format interfacevirtio/interface /disk \ -u admin@internal:** \ https://rhel.example.com/api/storagedomains/the_id_of_the_storage_domain/disks Then prepare a VM that you will use to copy the contents of your file to the new disk, and use the API to attach the new disk to this VM (you can preserve this VM, and use multiple times for this purpose): curl \ -k \ -X POST \ -H Accept: application/xml \ -H Content-Type: application/xml \ -d disk id='the_id_of_the_disk'/ \ -u admin@internal:** \ https://rhel.example.com/api/vms/the_id_of_the_vm/disks (Note that in order to attach the disk you have to provide the disk id returned by the API when you created it.) Then activate the disk, so that the VM can see it: curl \ -k \ -X POST \ -H Accept: application/xml \ -H Content-Type: application/xml \ -d action/ \ -u admin@internal:** \ https://rhel.example.com/api/vms/the_id_of_the_vm/disks/the_id_of_the_disk/activate Now you will have to copy the contents of the file to the disk via the VM. For example, assuming that you have SSH enabled on that VM and that the disk device inside the VM is /dev/vdb (it won't allways be this, depends on the number and order of attached disks) you can do something like this: ssh root@myvm 'cat /dev/vdb' myfile.img ssh root@myvm 'sync' Once the contents of the file have been copied you can deactivate the disk and detach it from this intermediate VM, and maybe attach it to another one. Take into account that all these operations are lengthy ones, specially the operation to create the disk, and that the RESTAPI will usually return once the operation is initiated, so you will need to wait till they are finished. For example, when creating the disk you should poll the state of the VM till it is ok. -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] VM Migration failed
On 01/18/2014 03:44 PM, Edgars M. wrote: Hi Thanks for your help. I will provide all log files a little bit later, but so far I have noticed some pattern when migration fails. Those particular VMs, which fails to migrate, were installed from ISO image, usual installation. So, I believe you can reproduce the issue by this: 1. Upload ISO image to ISO domain 2. Install new OS by booting from ISO image and check Attach CD in Boot Options 3. Delete ISO image from ISO domain 4. Try to migrate VM to another host. I have not tried this yet, but I have noticed that only those VMs fails which had been install from ISO image which is not in ISO domain anymore. I have also VMs created from templates and those VMs I can migrate just fine. did you stop the VMs post install? did you try to start them without the iso attached? otherwise, you can't start them, as there is a missing iso for the target qemu process I will provide more log files later. BR Edgars On Sat, Jan 18, 2014 at 1:05 PM, Dafna Ron d...@redhat.com mailto:d...@redhat.com wrote: I looked at the logs log and only vdsm from node2 (which appears to be the src node) seems to have full info. Please attach the following complete logs: vdsm log from dst node engine log libvirt logs from both nodes. Also, can you please answer the following? libvirt and vdsm you are using? what is selinux status on both hosts (enforcing/permissive)? do you have snapshots on the vm or does it happen on a newly created disk? when you create the disk, is it from a template or is it a new image? Thanks, Dafna On 01/17/2014 10:21 PM, Itamar Heim wrote: On 01/18/2014 12:07 AM, Meital Bourvine wrote: I opened a bug about missing info in UI: https://bugzilla.redhat.com/__show_bug.cgi?id=1054994 https://bugzilla.redhat.com/show_bug.cgi?id=1054994 It actually failed with this error: Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::starting migration monitor thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::__libvirtconnection::108::__libvirtconnection::(wrapper) Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to inquire lock: No such process Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::__(cancel) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::canceling migration downtime thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::__(stop) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::stopping migration monitor thread Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::migration downtime thread exiting Thread-1415::ERROR::2014-01-17 17:01:28,410::vm::238::vm.Vm::__(_recover) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Failed to inquire lock: No such process Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Failed to migrate Traceback (most recent call last): File /usr/share/vdsm/vm.py, line 323, in run self.___startUnderlyingMigration() File /usr/share/vdsm/vm.py, line 400, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/vm.py, line 838, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-__packages/vdsm/__libvirtconnection.py, line 76, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-__packages/libvirt.py, line 1178, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: Failed to inquire lock: No such process Thread-26::ERROR::2014-01-17 17:01:28,917::sampling::355::__vm.Vm::(collect) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Stats function failed: AdvancedStatsFunction _highWrite at 0x26efb58 The problem is that it doesn't say which process... this looks like noise post the migration failing, with probably a more relevant error in libvirt log - Original Message - From: Itamar Heim ih...@redhat.com mailto:ih...@redhat.com To: Edgars M. edgars.maz...@gmail.com mailto:edgars.maz...@gmail.com, Meital Bourvine mbour...@redhat.com
Re: [Users] vlan on mgmt network
On 01/18/2014 04:52 PM, Peter Styk wrote: So I got ovirtmgmt VM ticked off. Had to remove it from all VM's then tried to add new Logical network to eth0 by drag drop but refused since ovirtmgmt was out of sync. So I synced it. and that's how I lost access to my hosted remote system. End of story. So that's it I guess, automated install doesn't work (during switch network goes dead and doesn't come back remote access is lost) so I found manual way that works. But then I need to sync it to get my ovirtmgmt not to be a VM network. And even if its not to get another network on the interface i need to sync it. this sounds like a bug - can you provide clear reproduction steps? I will keep trying, Thanks for the tips. P. On 17 January 2014 22:06, Itamar Heim ih...@redhat.com mailto:ih...@redhat.com wrote: On 01/18/2014 12:05 AM, William Kwan wrote: I tried to define a virtual interface for vlan102, by adding ifcfg-bond0.102 manually and restarted network services. why not via ovirt-engine? On Friday, January 17, 2014 4:45 PM, Itamar Heim ih...@redhat.com mailto:ih...@redhat.com wrote: On 01/17/2014 11:38 PM, William Kwan wrote: OK.. tested with bond0.102 for vlan102 I guess I can't do this with oVirt? It wouldn't pickup this interface. It still shows up eth0, eth1 and bond0 only you tried to define a vlan youself on the bond, or use a vlan via a logical network over the bond via ovirt-engine (which should work)? Will On Friday, January 17, 2014 1:15 PM, William Kwan pota...@yahoo.com mailto:pota...@yahoo.com mailto:pota...@yahoo.com mailto:pota...@yahoo.com wrote: Thanks all for the info. I'm trying the suggestions with 3.3.2-1.el6 Assaf mentioned the following which is true. You can make ovirtmgmt a non-VM, untagged (non-VLAN) network, then place as many VM, tagged networks as you'd like on the same NIC or bond. I tried a few things and I haven't been able to find a way through. Obviously or not, there are more facts/rules. Just put up a few of them here Cannot have more than one non-VLAN network on one interface Cannot have a non-VLAN VM network and VLAN-tagged networks on one interface. ( got this when I tried to drag a VLAN-tagged VM network on the same interface as the non-VLAN VM network). I'll try to test adding a bond0.vlan. I need to have bond0 - for both mgmt network and a vlan bond1 - for another subnet. Will On Thursday, January 16, 2014 1:50 PM, Juan Pablo Lorier jplor...@gmail.com mailto:jplor...@gmail.com mailto:jplor...@gmail.com mailto:jplor...@gmail.com wrote: Hi William, You can have several logical networks on the same interface. If ovirtmgmt is not a vm network you can even mix tagged and untagged networks (if not, you can't mix). You don's have to do anything besides creating the logical networks in ovirt and assign them to the interface, ovirt takes care of creating the virtual interfaces it needs. What you can't do is have more than one logical network per vlan (I think there's a feature request on this, I know I've asked it on the list:-) ). Regards, _ Users mailing list Users@ovirt.org mailto:Users@ovirt.org mailto:Users@ovirt.org mailto:Users@ovirt.org http://lists.ovirt.org/__mailman/listinfo/users http://lists.ovirt.org/mailman/listinfo/users _ Users mailing list Users@ovirt.org mailto:Users@ovirt.org http://lists.ovirt.org/__mailman/listinfo/users http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] VM Migration failed
On 18 Jan 2014, at 18:06, Itamar Heim ih...@redhat.com wrote: On 01/18/2014 03:44 PM, Edgars M. wrote: Hi Thanks for your help. I will provide all log files a little bit later, but so far I have noticed some pattern when migration fails. Those particular VMs, which fails to migrate, were installed from ISO image, usual installation. So, I believe you can reproduce the issue by this: 1. Upload ISO image to ISO domain 2. Install new OS by booting from ISO image and check Attach CD in Boot Options 3. Delete ISO image from ISO domain 4. Try to migrate VM to another host. I have not tried this yet, but I have noticed that only those VMs fails which had been install from ISO image which is not in ISO domain anymore. I have also VMs created from templates and those VMs I can migrate just fine. did you stop the VMs post install? did you try to start them without the iso attached? otherwise, you can't start them, as there is a missing iso for the target qemu process Indeed. That is a known libvirt/qemu issue that even though the CD is defined as optional and VM can be started without it on original host, it fails upon migration when destination is being created. Thanks, michal I will provide more log files later. BR Edgars On Sat, Jan 18, 2014 at 1:05 PM, Dafna Ron d...@redhat.com mailto:d...@redhat.com wrote: I looked at the logs log and only vdsm from node2 (which appears to be the src node) seems to have full info. Please attach the following complete logs: vdsm log from dst node engine log libvirt logs from both nodes. Also, can you please answer the following? libvirt and vdsm you are using? what is selinux status on both hosts (enforcing/permissive)? do you have snapshots on the vm or does it happen on a newly created disk? when you create the disk, is it from a template or is it a new image? Thanks, Dafna On 01/17/2014 10:21 PM, Itamar Heim wrote: On 01/18/2014 12:07 AM, Meital Bourvine wrote: I opened a bug about missing info in UI: https://bugzilla.redhat.com/__show_bug.cgi?id=1054994 https://bugzilla.redhat.com/show_bug.cgi?id=1054994 It actually failed with this error: Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::starting migration monitor thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::__libvirtconnection::108::__libvirtconnection::(wrapper) Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to inquire lock: No such process Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::__(cancel) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::canceling migration downtime thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::__(stop) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::stopping migration monitor thread Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::migration downtime thread exiting Thread-1415::ERROR::2014-01-17 17:01:28,410::vm::238::vm.Vm::__(_recover) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Failed to inquire lock: No such process Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Failed to migrate Traceback (most recent call last): File /usr/share/vdsm/vm.py, line 323, in run self.___startUnderlyingMigration() File /usr/share/vdsm/vm.py, line 400, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/vm.py, line 838, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-__packages/vdsm/__libvirtconnection.py, line 76, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-__packages/libvirt.py, line 1178, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: Failed to inquire lock: No such process Thread-26::ERROR::2014-01-17 17:01:28,917::sampling::355::__vm.Vm::(collect) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Stats function failed: AdvancedStatsFunction _highWrite at 0x26efb58 The problem is that it doesn't say which process... this looks like noise post the migration failing, with probably a more
Re: [Users] VM Migration failed
sounds like this is the issue to me... Edgars can you try to confirm that? :) Michal, was there a bug opened for this? I think that perhaps we should also add a clear error message - it would help debug this more easily. On 01/18/2014 05:34 PM, Michal Skrivanek wrote: On 18 Jan 2014, at 18:06, Itamar Heim ih...@redhat.com wrote: On 01/18/2014 03:44 PM, Edgars M. wrote: Hi Thanks for your help. I will provide all log files a little bit later, but so far I have noticed some pattern when migration fails. Those particular VMs, which fails to migrate, were installed from ISO image, usual installation. So, I believe you can reproduce the issue by this: 1. Upload ISO image to ISO domain 2. Install new OS by booting from ISO image and check Attach CD in Boot Options 3. Delete ISO image from ISO domain 4. Try to migrate VM to another host. I have not tried this yet, but I have noticed that only those VMs fails which had been install from ISO image which is not in ISO domain anymore. I have also VMs created from templates and those VMs I can migrate just fine. did you stop the VMs post install? did you try to start them without the iso attached? otherwise, you can't start them, as there is a missing iso for the target qemu process Indeed. That is a known libvirt/qemu issue that even though the CD is defined as optional and VM can be started without it on original host, it fails upon migration when destination is being created. Thanks, michal I will provide more log files later. BR Edgars On Sat, Jan 18, 2014 at 1:05 PM, Dafna Ron d...@redhat.com mailto:d...@redhat.com wrote: I looked at the logs log and only vdsm from node2 (which appears to be the src node) seems to have full info. Please attach the following complete logs: vdsm log from dst node engine log libvirt logs from both nodes. Also, can you please answer the following? libvirt and vdsm you are using? what is selinux status on both hosts (enforcing/permissive)? do you have snapshots on the vm or does it happen on a newly created disk? when you create the disk, is it from a template or is it a new image? Thanks, Dafna On 01/17/2014 10:21 PM, Itamar Heim wrote: On 01/18/2014 12:07 AM, Meital Bourvine wrote: I opened a bug about missing info in UI: https://bugzilla.redhat.com/__show_bug.cgi?id=1054994 https://bugzilla.redhat.com/show_bug.cgi?id=1054994 It actually failed with this error: Thread-1417::DEBUG::2014-01-17 17:01:28,344::vm::768::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::starting migration monitor thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::__libvirtconnection::108::__libvirtconnection::(wrapper) Unknown libvirterror: ecode: 38 edom: 42 level: 2 message: Failed to inquire lock: No such process Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::745::vm.Vm::__(cancel) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::canceling migration downtime thread Thread-1415::DEBUG::2014-01-17 17:01:28,409::vm::815::vm.Vm::__(stop) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::stopping migration monitor thread Thread-1416::DEBUG::2014-01-17 17:01:28,409::vm::742::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::migration downtime thread exiting Thread-1415::ERROR::2014-01-17 17:01:28,410::vm::238::vm.Vm::__(_recover) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Failed to inquire lock: No such process Thread-1415::ERROR::2014-01-17 17:01:28,619::vm::337::vm.Vm::__(run) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Failed to migrate Traceback (most recent call last): File /usr/share/vdsm/vm.py, line 323, in run self.___startUnderlyingMigration() File /usr/share/vdsm/vm.py, line 400, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/vm.py, line 838, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-__packages/vdsm/__libvirtconnection.py, line 76, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-__packages/libvirt.py, line 1178, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: Failed to inquire lock: No such process Thread-26::ERROR::2014-01-17 17:01:28,917::sampling::355::__vm.Vm::(collect) vmId=`b8787906-187a-4234-a0c9-__58fc4ddf2a57`::Stats
[Users] engine-iso-uploader -- REST API not usable?
I ran into this problem when I tried to use engine-iso-uploader, but reading on the lists makes it sound like it may be a more general problem. There was a bug that caused this, but that was back in the ver. 3.0/3.1 days, and doesn't seem common since then. Back on Dec 24 I was able to upload an ISO file OK, so I am not sure what has changed since then. I am running a test setup, fully up to date: office2a host w/ glusterfs Centos 6 office4a host w/ glusterfs Centos 6 ov-eng01 engine on Centos 6 VM (not hosted on oVirt) office9 KVM host (not oVirt) for ov-eng01 whether I log in to ov-eng01 by ssh or execute the command from the console, I get: # engine-iso-uploader list -v Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): ERROR: Problem connecting to the REST API. Is the service available and does the CA certificate exist? checking on some things suggested on a thread about engine-iso-uploader back in March, I get: # ls -la /etc/pki/ovirt-engine/ca.pem -rw-r--r--. 1 root root 4569 Nov 10 15:13 /etc/pki/ovirt-engine/ca.pem # cat /var/log/ovirt-engine/ovirt-iso-uploader/ovirt-iso-uploader/20140117112938.log 2014-01-17 11:29:44::ERROR::engine-iso-uploader::512::root:: Problem connecting to the REST API. Is the service available and does the CA certificate exist? The thread back in March gave a work-around to upload ISO images directly, so I am not blocked from uploading images, but I would like to get things working right, as I am afraid the problem will turn around and bite me down the road. Ted Miller ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] engine-iso-uploader -- REST API not usable?
On 01/18/2014 07:49 PM, Ted Miller wrote: I ran into this problem when I tried to use engine-iso-uploader, but reading on the lists makes it sound like it may be a more general problem. There was a bug that caused this, but that was back in the ver. 3.0/3.1 days, and doesn't seem common since then. Back on Dec 24 I was able to upload an ISO file OK, so I am not sure what has changed since then. I am running a test setup, fully up to date: office2a host w/ glusterfs Centos 6 office4a host w/ glusterfs Centos 6 ov-eng01 engine on Centos 6 VM (not hosted on oVirt) office9 KVM host (not oVirt) for ov-eng01 whether I log in to ov-eng01 by ssh or execute the command from the console, I get: # engine-iso-uploader list -v Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): ERROR: Problem connecting to the REST API. Is the service available and does the CA certificate exist? checking on some things suggested on a thread about engine-iso-uploader back in March, I get: # ls -la /etc/pki/ovirt-engine/ca.pem -rw-r--r--. 1 root root 4569 Nov 10 15:13 /etc/pki/ovirt-engine/ca.pem # cat /var/log/ovirt-engine/ovirt-iso-uploader/ovirt-iso-uploader/20140117112938.log 2014-01-17 11:29:44::ERROR::engine-iso-uploader::512::root:: Problem connecting to the REST API. Is the service available and does the CA certificate exist? The thread back in March gave a work-around to upload ISO images directly, so I am not blocked from uploading images, but I would like to get things working right, as I am afraid the problem will turn around and bite me down the road. Ted Miller To discard an actual problem with the REST API I would suggest you to run the following command, from the same machine: # curl \ --cacert /etc/pki/ovirt-engine/ca.pem \ --user admin@internal:your_password \ https://your_host/api This should give you a XML document containing the entry point of the API, something like this: ?xml version=1.0 encoding=UTF-8 standalone=yes? api ... /api I would also suggest to check the /etc/ovirt-engine/isouploader.conf file, as you may have changed something there in the past. -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] Unbundle bond0 which virtmgmt is attached
Hi, FAQ type of question. If virtmgmt is assigned to bond0, which is composed of eth0 and eth1, what is the best way to unbundle eth1 from bond0? Will ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Unbundle bond0 which virtmgmt is attached
On Sat, Jan 18, 2014 at 03:17:22PM -0800, William Kwan wrote: Hi, FAQ type of question. If virtmgmt is assigned to bond0, which is composed of eth0 and eth1, what is the best way to unbundle eth1 from bond0? I'd say: - Move the host to maintanence, so it runs no VMs and does not play SPM role. - Use the Setup Network dialog to brake the bond and define ovirtmgmt directly on top of eth0. The Engine/Vdsm connection should survive this kind of change (unlike a change of management address or vlan). If the host uses a special storage domain, or is not SPM, you could even do that while VMs are running. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] vlan on mgmt network
Hi Peter, Try to check if ovirmgmt is defined in the vlan you desire (or without vlan). Maybe when you sync ovirt is trying to configure the network with options different from what you desire. I never got ovirt to finish properly to install a host, it fails to get ovirtmgmt working right so I always manually configure the vlan interfaces and the ovirtmgmt bridge in the hosts. You can get your host back if you manually correct the network, also you may take a look of what ovirt configured to try to find out if something is not set properly in the network definition. Regards El ene 18, 2014 7:56 PM, Peter Styk polf...@gmail.com escribió: On 18 January 2014 17:11, Itamar Heim ih...@redhat.com wrote: On 01/18/2014 04:52 PM, Peter Styk wrote: So I got ovirtmgmt VM ticked off. Had to remove it from all VM's then tried to add new Logical network to eth0 by drag drop but refused since ovirtmgmt was out of sync. So I synced it. and that's how I lost access to my hosted remote system. End of story. So that's it I guess, automated install doesn't work (during switch network goes dead and doesn't come back remote access is lost) so I found manual way that works. But then I need to sync it to get my ovirtmgmt not to be a VM network. And even if its not to get another network on the interface i need to sync it. this sounds like a bug - can you provide clear reproduction steps? Reproduction steps are in network configuration alone. After installing Engine, i have to set up network to the config listed here: http://styk.tv/wp-content/uploads/2014/01/oVirtHosted1_almost_working.png. Unfortunately can't rely on engine-vdsm duo to help out. Anything above the host line on the diagram is physical setup and is the only configuration that doesn't disconnect me from the net. Only thing missing on the diagram is ifcfg-eth0 has also HWADDR attribute with MAC address of physical eth0 device. Once this survives service network restart I can proceed to VDSM install and then its straight forwardthat is until I'm trying to SYNC ovirtmgmt on host inside. I should mention after ovirt engine is working I destroy default cluster and create new local one. Obviously I cannot access logs since access to host is no longer but entire setup is scripted including provisioning so I can easily rebuild entire setup within 15 or so minutes by running a script. Anyone who would like to benefit from my findings can use this script and gain access to my host and learn with me on how to overcome this. Script will provision the host (fresh os install), log in, get the files: ifcfg-eth0, ifcfg-ovirtmgmt, ifcfg-ovirtmgmt-range0, ovrit_answers and route-ovirtmgmt, then install epel 6-8, install pgp, localinstall ovirt-el6.10-1. install bridge-utils, upgrade, set hostname, then after reboot ssh alive, set local data,images,iso folders, install ovrit-engine, set ipv4 forwarding and proxy_arp=1, restart-network, and run engine-setup with ovirt_answers including cli and stop iptables as engine and vdsm rules still prevent connection if on. that's it. working system in 15 mins still to do is engine-api calls to create local cluster, join engine with vdsm local and setup private network with pfSense instance as router/nat/dhcp for 10.0.0.0/24 Peter ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users