[ovirt-users] Very old tasks hanging

[Soeren Malchow]

Dear all,

I have tasks hanging for a very long time, there are 2 different possibilities


  1.  The task has nothing to work on anymore, e.g. I have one task that is 
weeks old that was deleting a cloned VM
  2.  I have a cloned VM and that was exported and the task is still running 
after 5 days - it actually isn’t running, but the VM still has the hourglass in 
the front end and i can do nothing with it, i even rebootet all hosts in that 
cluster already and upgraded the engine and the hosts in the cluster form 3.5.3 
to 3.5.4

Any idea where to even start looking ?

Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Backup / Export Storage

[Soeren Malchow]

Dear all,

We have a large amount of Vms in the backup/export storage, basically every of 
the approx 50 machines in approx 20-30 generations, the listing (in VM Import 
Tab of the storage) was always very slow, but now i get an error – am small 
popup with a 502.

I can not even delete Vms to make the storage smaller again.

Regarding this i have a few questions

  1.  How do i reduce the amount of VMS so that i can access the VM Import 
again ? Can i just delete on storage level ?
  2.  Is there anything i can do to speed up the listing
  3.  Does anyone have an idea how to export the exported (yes :-) ) VMS to 
another storage in a way that they can be accessed and reimported if necessary, 
as far as i know the machines are already ovf formats, i would only need to get 
the name for each machine and write it somehwere else, suggestions ?

Cheers
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Backup / Export Storage

[Soeren Malchow]

Dear all,

We have a large amount of Vms in the backup/export storage, basically every of 
the approx 50 machines in approx 20-30 generations, the listing (in VM Import 
Tab of the storage) was always very slow, but now i get an error – am small 
popup with a 502.

I can not even delete Vms to make the storage smaller again.

Regarding this i have a few questions

  1.  How do i reduce the amount of VMS so that i can access the VM Import 
again ? Can i just delete on storage level ?
  2.  Is there anything i can do to speed up the listing
  3.  Does anyone have an idea how to export the exported (yes :-) ) VMS to 
another storage in a way that they can be accessed and reimported if necessary, 
as far as i know the machines are already ovf formats, i would only need to get 
the name for each machine and write it somehwere else, suggestions ?

Cheers
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Migrating hosts from RHEL6 to RHEL7

[Soeren Malchow]

Hi Ryan,

We did exactly that, we create a new cluster and then did a live migration 
between clusters, we had no problems with this (same hardware, same cpu …. )

Cheers
Soeren

From: mailto:users-boun...@ovirt.org>> on behalf of 
matthew lagoe
Date: Thursday 20 August 2015 23:25
To: "Groten, Ryan"
Cc: "users@ovirt.org"
Subject: Re: [ovirt-users] Migrating hosts from RHEL6 to RHEL7

It is possible to migrate a VM from 6 to 7 it is recommended however that you 
don't do a live migration and that you power off the VM and then turn it back 
on on the cluster running version 7 "Groten, Ryan" wrote:
Has anyone succeeded in upgrading their hosts OS version from 6 to 7?  I 
assumed it could be done without downtime and one host at a time, but when 
trying it out I found that RHEL7 hosts can’t be placed in the same Cluster as 
RHEL6 ones.
I then tried making a new Cluster and migrating VMs from the RHEL6 cluster to 
RHEL7.  Initial testing seems to show that it works from RHEL6 to RHEL7 but not 
the other way around.  Also when I select this option a warning pops up saying 
“Choosing different cluster may lead to unexpected results.  Please consult 
documentation.”

I looked in the Admin Guide and Technical Reference Guide but don’t see where 
these unexpected results are mentioned.

Thanks,
Ryan Groten
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] VM Live Backup

[Soeren Malchow]

Hi,

To support Michael here (i think he knows the script i sent around already) 
having a builtin backup would be much nicer.
The current procedure (ovirt 3.5) involves not only snapshotting and exporting, 
but also a clone before the export which puts massive unnecessary load on the 
environment.

We have developers who can potentially develop such a backup procedure, 
however, we can not do this alone.

Do you guys think we could do something like that through crowfunding or who 
would be willing to participate in something like that ?

Regards
Soeren



From: mailto:users-boun...@ovirt.org>> on behalf of 
Donny Davis
Date: Thursday 30 July 2015 23:20
To: "Prof. Dr. Michael Schefczyk"
Cc: users
Subject: Re: [ovirt-users] VM Live Backup


You could write a script to interface with the api that creates a snapshot and 
then exports or copies that to a backup nfs share And then run that on a 
cron job

On Jul 30, 2015 5:18 PM, "Prof. Dr. Michael Schefczyk" 
mailto:mich...@schefczyk.net>> wrote:
Dear All,

One feature that I am really missing in oVirt is a foolproof (i.e., in a way 
that even I will be able to implement) scheduled (cron would be good) live VM 
backup that outputs ideally qcow2 files per disk plus an xml-configuration file 
that one could upload to a regular KVM host, in case that should be required. 
As far as I understand, that is not yet on the roadmap, correct? For my plain 
SOHO KVM hosts, I am using a script to do just that. While I almost never need 
the backups - files with a human readable text name which I know how to use on 
any freshly installed KVM host in case of a crash emergency -, I feel better 
keeping them available for some time period.

Regards,

Michael
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Live VM Backups

[Soeren Malchow]

Dear Chris,

It is not true, you can snapshot a machine, then clone the snapshot and
export it for backup purposes after that you can remove the snapshot, all
on the live VM.
However, you need newer versions of libvirt to do that, right now we are
using CentOS 7.1 and the libvirt that comes with it is capable of doing
live merge which is necessary to achieve this.

But i have to warn you, we are experiencing a problem when removing the
snapshots (the part is commented in the attached script) it sometimes
kills virtual machines in a way that makes it necessary to put the
hypervisor to maintenance and then restart vdsmd and libvirtd before you
can start that VM again.

There is a bug filed already and it is in progress

https://bugzilla.redhat.com/show_bug.cgi?id=1231754

I also have to add that i newer version of libvirt (on Fedora 20 with the
libvirt preview repo) did not have that problem, so i am confident that
this will be solved soon.

Last but not least there is a plan to be able to export snapshots right
away for backup without having to clone them first, this is a huge step
forward for the backup procedure in terms of time that is needed and the
load on the storage and hypervisor systems.

I would really appreciate if you would help improving that script (we are
not python developers), i will see that i make this a github project or
something like that

Cheers
Soeren





On 08/07/15 17:13, "users-boun...@ovirt.org on behalf of Chris Jones -
BookIt.com Systems Administrator"  wrote:

> From what I can tell, you can only backup a VM to an export domain if
>the VM is shut down. Is a live VM backup not possible through oVirt? If
>not, why not? Most other virtualization tools can handle this.
>
>If it is possible, how do I do it through the backup API?
>api.vms.myvm.export requires it to be shutdown so what would the
>alternative be?
>
>Thanks.
>
>-- 
>This email was Virus checked by UTM 9. For issues please contact the
>Windows Systems Admin.
>___
>Users mailing list
>Users@ovirt.org
>http://lists.ovirt.org/mailman/listinfo/users

#!/usr/bin/python

import Queue
import threading
import time
from ovirtsdk.api import API
from ovirtsdk.xml import params
import sys
import datetime
import smtplib
from email.mime.text import MIMEText


global SNAPSHOT_NAME

VERSION = params.Version(major='3', minor='0')
ENGINE_SERVER   = ''
ENGINE_USER = ''
ENGINE_PASSWORD = ''
ENGINE_CERT = ''
NOW = datetime.datetime.now()
SNAPSHOT_NAME   = 'BACKUP_' + NOW.strftime("%Y-%m-%d-%H%M")
DAY_OF_WEEK = NOW.strftime("%w")
BACKUP  = "FULL"

exitFlag = 0

class myThread (threading.Thread):
def __init__(self, threadID, name, q):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.q = q
self.api = api
global message
def run(self):
print "Starting " + self.name
process_data(self.name, self.q)
print "Exiting " + self.name

def process_data(threadName, q):
while not exitFlag:
queueLock.acquire()
if not workQueue.empty():
data = q.get()
queueLock.release()
print "%s processing %s" % (threadName, data.name)
vm = api.vms.get(name=data.name)
vmname = data.name +"_"
newvmname = vmname + SNAPSHOT_NAME
cluster = api.clusters.get(id=vm.cluster.id)
dc = api.datacenters.get(id=cluster.data_center.id)
export = None
for sd in dc.storagedomains.list():
if sd.type_ == "export":
export = sd
if not export:
print("Export domain required, and none found, exitting...\n")
sys.exit(1)
if (data.name != 'HostedEngine' and cluster.name == 'CC-01'):
vm.snapshots.add(params.Snapshot(description=SNAPSHOT_NAME, vm=vm ))
snap = vm.snapshots.list(description=SNAPSHOT_NAME)[0]
while vm.snapshots.get(id=snap.id).snapshot_status == "locked":
print("%s Waiting for snapshot of %s to finish") % (threadName, vm.name)
time.sleep(60)
print("%s Snapshotting %s is done") % (threadName,vm.name)
try:
snapshots = params.Snapshots(snapshot=[params.Snapshot(id=snap.id)])
api.vms.add(params.VM(name=newvmname, snapshots=snapshots, cluster=cluster, template=api.templates.get(name="Blank")))
while api.vms.get(name=newvmname).status.state == "image_locked":
print("%s Waiting for clone of %s to finish") % (threadName, vm.name)
time.sleep(300)
print("%s Cloning of %s  done") % (threadName, vm.name)
api.vms.get(name=newvmname).export(params.Action(storage_domain=export))
   

Re: [ovirt-users] oVirt 3.5.3 Host Network Interfaces

[Soeren Malchow]

Dear Michael,

The network scripts look exactly as we did that, and then during setting
up the host network we also got the “already enslaved” message which did
not cause a problem since it actually was enslaved, afterwards there were
proper vdsm generated interface files.

Though we have the difference, that the ovirt already showed the bond even
when it was not managed by ovirt yet.

You can still change the one bonded interface in ovirt and add one
additional interface, just drag and drop, put the bridge in and configured
the Ips, however, if you want 802.3ad you need to change the switch ports
to do 802.3ad as well a little bit after you applied the change, to avoid
having to do the switch configuration use “custom” and “mode=5” or “mode=6”


Regards
Soeren 


On 07/07/15 01:14, "Prof. Dr. Michael Schefczyk" 
wrote:

>Dear Soeren, dear all,
>
>Thank you very much. I did try the steps on an installed system. The
>result was that after manipulating the network scripts, the server would
>communicate well with the rest of the LAN - as many other servers on
>which I use NIC bonds. However, my hosted-engine refused to start:
>"hosted-engine --vm-start" results in "Connection to localhost:54321
>refused". Needless to say, it is not possible to revert the maintenance
>mode then. ABRT points out what the problem is (but not the solution):
>"api.py:119:objectivizeNetwork:ConfigNetworkError: (24, u'nic enp0s20f1
>already enslaved to bond0')" This shows that oVirt cannot start after the
>manipulation of network scripts as it does expect enp0s20f1 no not be a
>slave of a bond. Naming of the devices becomes clear from the following
>paragraph.
>
>After multiple failed experiments to set up the server with bonded NIC
>(each implying configuring the server from scratch), I took the following
>approach this time:
>- The server has two LAN facing NICs enp0s20f0, enp0s20f1. The other two
>NICs point to the other server in the intended gluster cluster - creating
>a bond for them is not problematical.
>- Setting up Centos 7, I just used enp0s20f0 with no bond and bridge.
>- Deploying oVirt one gets asked for an unused NIC, so I selected
>enp0s20f1. Then the network was only resting on enp0s20f1 with a working
>setup of oVirt - just without bonding two NICs.
>
>I am still surprised that this HA network issue is so difficult to manage
>in a software that is largely about high availability. Can anyone please
>indicate how to proceed towards NIC bonding?
>
>Regards,
>
>Michael
>
>
>Network Scripts as set manually:
>
>DEVICE=enp0s20f0
>TYPE=Ethernet
>USERCTL=no
>SLAVE=yes
>MASTER=bond0
>BOOTPROTO=none
>HWADDR=00:25:90:F5:18:9A
>NM_CONTROLLED=no
>
>DEVICE=enp0s20f1
>TYPE=Ethernet
>USERCTL=no
>SLAVE=yes
>MASTER=bond0
>BOOTPROTO=none
>HWADDR=00:25:90:F5:18:9B
>NM_CONTROLLED=no
>
>DEVICE=bond0
>ONBOOT=yes
>BONDING_OPTS='mode=802.3ad miimon=100'
>BRIDGE=ovirtmgmt
>NM_CONTROLLED=no
>
>DEVICE=ovirtmgmt
>ONBOOT=yes
>TYPE=Bridge
>IPADDR=192.168.12.40
>NETMASK=255.255.255.0
>GATEWAY=192.168.12.1
>DNS=192.168.12.1
>NM_CONTROLLED=no
>
>
>
>
>-Ursprüngliche Nachricht-
>Von: Soeren Malchow [mailto:soeren.malc...@mcon.net]
>Gesendet: Montag, 6. Juli 2015 15:01
>An: Prof. Dr. Michael Schefczyk; users@ovirt.org
>Betreff: Re: [ovirt-users] oVirt 3.5.3 Host Network Interfaces
>
>Dear Michael
>
>We actually created the ovirtmgmt and the bond manually upfront and then
>in the ³Setup Hosts Network² we basically did this again (including
>setting the IP address), regarding the bonding in the gluster network we
>did not have a problem, you just drag one interface onto the other and
>then select the bonding mode, where you can also go for bonding mode TLB
>or ALB if you choose ³custom² or just LACP if you have switches that
>support this.
>
>
>Step by Step:
>
>- set the engine to maintenance and shut it down
>- configure the bond on the 2 nics for the ovirtmgmt bridge ( em1+em2 ->
>bond0 -> ovirtmgmt )
>- configure the IP on the bridge
>- reboot the server and see whethter it comes up correctly
>- remove maintenance and let engine start
>- Set up the ovirtmgmt in hosts networks but do not forget to set IP and
>Gateway as well.
>
>Though it should work without this hassle (if the bonding mode on the
>switch and server is compatible) but this way it is easy to get server
>and switch in the same mode and working without having to do anything in
>ovirt first.
>
>Hope that helps
>
>Regards
>Soeren
>
>
>
>On 02/07/15 00:31, "users-boun...@ovirt.org on behalf of Prof. Dr.
>Michael Schefczyk" mich...@schefczyk.net>
>wrote:
>

Re: [ovirt-users] oVirt 3.5.3 Host Network Interfaces

[Soeren Malchow]

Dear Michael

We actually created the ovirtmgmt and the bond manually upfront and then
in the ³Setup Hosts Network² we basically did this again (including
setting the IP address), regarding the bonding in the gluster network we
did not have a problem, you just drag one interface onto the other and
then select the bonding mode, where you can also go for bonding mode TLB
or ALB if you choose ³custom² or just LACP if you have switches that
support this.


Step by Step:

- set the engine to maintenance and shut it down
- configure the bond on the 2 nics for the ovirtmgmt bridge ( em1+em2 ->
bond0 -> ovirtmgmt )
- configure the IP on the bridge
- reboot the server and see whethter it comes up correctly
- remove maintenance and let engine start
- Set up the ovirtmgmt in hosts networks but do not forget to set IP and
Gateway as well.

Though it should work without this hassle (if the bonding mode on the
switch and server is compatible) but this way it is easy to get server and
switch in the same mode and working without having to do anything in ovirt
first.

Hope that helps

Regards
Soeren



On 02/07/15 00:31, "users-boun...@ovirt.org on behalf of Prof. Dr. Michael
Schefczyk" 
wrote:

>Dear All,
>
>Having set up a Centos 7 server with Gluster, oVirt 3.5.3 and hosted
>engine according to
>https://community.redhat.com/blog/2014/10/up-and-running-with-ovirt-3-5/,
>I was hoping that the NIC management and particularly NIC bond/bridge
>capabilities would have improved a bit. My server has four NICs, two
>connected to the LAN and two to an adjacent server to be used as a
>Gluster network between the two servers. My aim is to use NIC bonding for
>two NICs each.
>
>Via the engine, I would like to use Hosts -> Network Interfaces -> Setup
>Host Networks. As I use hosted engine, I cannot set the only host to
>maintenance mode. At least during normal operations, however, I am
>neither able to change the ovirt bridge from DHCP to static IP nor create
>a bond consisting of the two LAN facing NICs. In each case I get, "Error
>while executing action Setup Networks: Network is currently being used".
>Editing the network scripts manually is not an option either, as that
>does not survive a reboot. Contrary to this real view, everything should
>be easily configurable according to section 6.6 of the oVirt
>administration guide.
>
>One workaround approach could be to temporarily move one NIC connection
>from the adjacent server to the LAN or even temporarily swap both pairs
>of NICs and edit interfaces while they are not in use. Is this really the
>way forward? Should there not be a more elegant approach not requiring
>physically plugging NIC connections just to work around such issue?
>
>Regards,
>
>Michael
>
>

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Autostart VM

[Soeren Malchow]

Alternatively you can use something like Foreman/puppet to deploy new hosts, 
this usually also brings the machines right up and also takes care of IP 
address management (if configured)

And you can use neat features such as cloud-init from foreman as well

Goes a little beyond your question, but i thought i add this


From: mailto:users-boun...@ovirt.org>> on behalf of 
Koen Vanoppen mailto:vanoppen.k...@gmail.com>>
Date: Friday 3 July 2015 11:23
To: Jakub Niedermertl mailto:jnied...@redhat.com>>
Cc: "users@ovirt.org" 
mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] Autostart VM

Thanks!!!

On 1 July 2015 at 19:36, Jakub Niedermertl 
mailto:jnied...@redhat.com>> wrote:
Hi Koen,

AFAIK WebAdmin UI doesn't provide such feature. However it can be scripted 
using Java SDK, Python SDK or REST API.

http://www.ovirt.org/REST-Api
http://www.ovirt.org/Java-sdk
http://www.ovirt.org/Python-sdk

Best regards,
Jakub

- Original Message -

> From: "Koen Vanoppen" 
> mailto:vanoppen.k...@gmail.com>>
> To: users@ovirt.org
> Sent: Tuesday, June 30, 2015 1:17:56 PM
> Subject: [ovirt-users] Autostart VM

> Hi Guys,

> I couldn't find anything directly concerning my following question, so If it
> was asked before, I'm sorry...

> Is there a way to config ovirt in a way that when you created a new vm, it
> will automatically boot? Because now, when you create a VM it stays in down
> state.

> Kind regards,

> Koen

> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Can´t use a VM Network VLAN, because the Virtual Machines inside can't reach the Gateway

[Soeren Malchow]

Hi,

To make it easier to understand what the situation is, it would help to
see your bridging configuration and your interface configration.

Something like:

#> ip addr

And

#> brctl show

Regards
Soeren

On 06/07/15 13:30, "users-boun...@ovirt.org on behalf of Dan Kenigsberg"
 wrote:

>On Tue, Jun 30, 2015 at 03:48:49PM -0500, Julián Tete wrote:
>> Hi Friends of oVirt
>> 
>> I'm trying to migrate my company from VMware to oVirt.
>
>We'd like to help you do this!
>
>> In my final tests, I set up 2 more VLANs in oVirt, (VM VLANs)
>> The Virtual Machines in these VLANs, can be reached from the external
>>IPs
>> from the net range,
>> but from the Virtual machines only can ping the Host with the Bridge and
>> itself, can't reach the gateway. ¿?
>> 
>> I configured a IP Forwarding in the Host with the virtual machine, and
>> nothing changes...
>> 
>> Any idea ? This is the last duty before embrace oVirt in the company.
>> 
>> Look to the VLAN 100 configuration (My desired VM Network):
>> 
>> http://postimg.org/image/7hrog0a2n/
>> 
>> http://postimg.org/image/68b40i1vr/
>> 
>> http://postimg.org/image/lu6mlshgp/
>
>I must admit that I don't understand your problem yet. When you ping
>from inside your guest, where are your packets dropped?
>
>I'd like to point that your "Hosting" network, as most VM networks, is
>better off left without an IP address. The benefit of this is better
>security (host TCP stack is not accessible from VMs) and less chances of
>routing collisions from the host.
>
>So unless you need to use the same network for something other than VM
>communication, try to clear its address.
>
>Regards,
>Dan.
>___
>Users mailing list
>Users@ovirt.org
>http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Moving a hosted engine

[Soeren Malchow]

Dear all,

Is there somewhere a document or information on how to move a hosted engine 
form one cluster to another ?

Right now we still have one cluster running fedora 20 that is also running the 
hosted engine and we have one cluster running CentOS 7.1 where the hosted 
engine should go.

Any advice how to proceed with this ?

Regerds
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

We are still having this problem and we can not figure out what to do, i
sent the logs already as download, can i do anything else to help ?




On 04/06/15 17:08, "Soeren Malchow"  wrote:

>Hi,
>
>I would send those, but unfortunately we did not think about the journals
>getting deleted after a reboot.
>
>I just made the journals persistent on the servers, we are trying to
>trigger the error again last time we only got half way through the VM’s
>when removing the snapshots so we have a good chance that it comes up
>again.
>
>Also the libvirt logs to the journal not to libvirtd.log, i would send the
>journal directly to you and Eric via our data exchange servers
>
>
>Soeren 
>
>On 04/06/15 16:17, "Adam Litke"  wrote:
>
>>On 04/06/15 13:08 +, Soeren Malchow wrote:
>>>Hi Adam, Hi Eric,
>>>
>>>We had this issue again a few minutes ago.
>>>
>>>One machine went down exactly the same way as described, the machine had
>>>only one snapshot and it was the only snapshot that was removed, before
>>>that in the same scriptrun we deleted the snapshots of 15 other Vms,
>>>some
>>>without, some with 1 and some with several snapshots.
>>>
>>>Can i provide anything from the logs that helps ?
>>
>>Let's start with the libvirtd.log on that host.  It might be rather
>>large so we may need to find a creative place to host it.
>>
>>>
>>>Regards
>>>Soeren
>>>
>>>
>>>
>>>On 03/06/15 18:07, "Soeren Malchow"  wrote:
>>>
>>>>Hi,
>>>>
>>>>This is not happening every time, the last time i had this, it was a
>>>>script runnning, and something like th 9. Vm and the 23. Vm had a
>>>>problem,
>>>>and it is not always the same VMS, it is not about the OS (happen for
>>>>Windows and Linux alike)
>>>>
>>>>And as i said it also happened when i tried to remove the snapshots
>>>>sequentially, here is the code (i know it is probably not the elegant
>>>>way,
>>>>but i am not a developer) and the code actually has correct indentions.
>>>>
>>>><― snip ―>
>>>>
>>>>print "Snapshot deletion"
>>>>try:
>>>>time.sleep(300)
>>>>Connect()
>>>>vms = api.vms.list()
>>>>for vm in vms:
>>>>print ("Deleting snapshots for %s ") % vm.name
>>>>snapshotlist = vm.snapshots.list()
>>>>for snapshot in snapshotlist:
>>>>if snapshot.description != "Active VM":
>>>>time.sleep(30)
>>>>snapshot.delete()
>>>>try:
>>>>while
>>>>api.vms.get(name=vm.name).snapshots.get(id=snapshot.id).snapshot_status
>>>>==
>>>>"locked":
>>>>print("Waiting for snapshot %s on %s deletion
>>>>to
>>>>finish") % (snapshot.description, vm.name)
>>>>time.sleep(60)
>>>>except Exception as e:
>>>>print ("Snapshot %s does not exist anymore") %
>>>>snapshot.description
>>>>print ("Snapshot deletion for %s done") % vm.name
>>>>print ("Deletion of snapshots done")
>>>>api.disconnect()
>>>>except Exception as e:
>>>>print ("Something went wrong when deleting the snapshots\n%s") %
>>>>str(e)
>>>>
>>>>
>>>>
>>>><― snip ―>
>>>>
>>>>
>>>>Cheers
>>>>Soeren
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On 03/06/15 15:20, "Adam Litke"  wrote:
>>>>
>>>>>On 03/06/15 07:36 +, Soeren Malchow wrote:
>>>>>>Dear Adam
>>>>>>
>>>>>>First we were using a python script that was working on 4 threads and
>>>>>>therefore removing 4 snapshots at the time throughout the cluster,
>>>>>>that
>>>>>>still caused problems.
>>>>>>
>>>>>>Now i took the snapshot removing out of the threaded part an i am
>>>>>>just
>>>>>>looping through each snapshot on each VM one after another, even with
>>>>>>³sleeps² inbe

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

Hi,

I would send those, but unfortunately we did not think about the journals
getting deleted after a reboot.

I just made the journals persistent on the servers, we are trying to
trigger the error again last time we only got half way through the VM’s
when removing the snapshots so we have a good chance that it comes up
again.

Also the libvirt logs to the journal not to libvirtd.log, i would send the
journal directly to you and Eric via our data exchange servers


Soeren 

On 04/06/15 16:17, "Adam Litke"  wrote:

>On 04/06/15 13:08 +, Soeren Malchow wrote:
>>Hi Adam, Hi Eric,
>>
>>We had this issue again a few minutes ago.
>>
>>One machine went down exactly the same way as described, the machine had
>>only one snapshot and it was the only snapshot that was removed, before
>>that in the same scriptrun we deleted the snapshots of 15 other Vms, some
>>without, some with 1 and some with several snapshots.
>>
>>Can i provide anything from the logs that helps ?
>
>Let's start with the libvirtd.log on that host.  It might be rather
>large so we may need to find a creative place to host it.
>
>>
>>Regards
>>Soeren
>>
>>
>>
>>On 03/06/15 18:07, "Soeren Malchow"  wrote:
>>
>>>Hi,
>>>
>>>This is not happening every time, the last time i had this, it was a
>>>script runnning, and something like th 9. Vm and the 23. Vm had a
>>>problem,
>>>and it is not always the same VMS, it is not about the OS (happen for
>>>Windows and Linux alike)
>>>
>>>And as i said it also happened when i tried to remove the snapshots
>>>sequentially, here is the code (i know it is probably not the elegant
>>>way,
>>>but i am not a developer) and the code actually has correct indentions.
>>>
>>><― snip ―>
>>>
>>>print "Snapshot deletion"
>>>try:
>>>time.sleep(300)
>>>Connect()
>>>vms = api.vms.list()
>>>for vm in vms:
>>>print ("Deleting snapshots for %s ") % vm.name
>>>snapshotlist = vm.snapshots.list()
>>>for snapshot in snapshotlist:
>>>if snapshot.description != "Active VM":
>>>time.sleep(30)
>>>snapshot.delete()
>>>try:
>>>while
>>>api.vms.get(name=vm.name).snapshots.get(id=snapshot.id).snapshot_status
>>>==
>>>"locked":
>>>print("Waiting for snapshot %s on %s deletion to
>>>finish") % (snapshot.description, vm.name)
>>>time.sleep(60)
>>>    except Exception as e:
>>>print ("Snapshot %s does not exist anymore") %
>>>snapshot.description
>>>print ("Snapshot deletion for %s done") % vm.name
>>>print ("Deletion of snapshots done")
>>>api.disconnect()
>>>except Exception as e:
>>>print ("Something went wrong when deleting the snapshots\n%s") %
>>>str(e)
>>>
>>>
>>>
>>><― snip ―>
>>>
>>>
>>>Cheers
>>>Soeren
>>>
>>>
>>>
>>>
>>>
>>>On 03/06/15 15:20, "Adam Litke"  wrote:
>>>
>>>>On 03/06/15 07:36 +, Soeren Malchow wrote:
>>>>>Dear Adam
>>>>>
>>>>>First we were using a python script that was working on 4 threads and
>>>>>therefore removing 4 snapshots at the time throughout the cluster,
>>>>>that
>>>>>still caused problems.
>>>>>
>>>>>Now i took the snapshot removing out of the threaded part an i am just
>>>>>looping through each snapshot on each VM one after another, even with
>>>>>³sleeps² inbetween, but the problem remains.
>>>>>But i am getting the impression that it is a problem with the amount
>>>>>of
>>>>>snapshots that are deleted in a certain time, if i delete manually and
>>>>>one
>>>>>after another (meaning every 10 min or so) i do not have problems, if
>>>>>i
>>>>>delete manually and do several at once and on one VM the next one just
>>>>>after one finished, the risk seems to increase.
>>>>
>>>>Hmm.  In our lab we extensively tested removing a snapshot for a VM
>>>>with 4 disks.  This means 4 block jobs running simultaneo

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

Hi Adam, Hi Eric,

We had this issue again a few minutes ago.

One machine went down exactly the same way as described, the machine had
only one snapshot and it was the only snapshot that was removed, before
that in the same scriptrun we deleted the snapshots of 15 other Vms, some
without, some with 1 and some with several snapshots.

Can i provide anything from the logs that helps ?

Regards
Soeren 



On 03/06/15 18:07, "Soeren Malchow"  wrote:

>Hi,
>
>This is not happening every time, the last time i had this, it was a
>script runnning, and something like th 9. Vm and the 23. Vm had a problem,
>and it is not always the same VMS, it is not about the OS (happen for
>Windows and Linux alike)
>
>And as i said it also happened when i tried to remove the snapshots
>sequentially, here is the code (i know it is probably not the elegant way,
>but i am not a developer) and the code actually has correct indentions.
>
><― snip ―>
>
>print "Snapshot deletion"
>try:
>time.sleep(300)
>Connect()
>vms = api.vms.list()
>for vm in vms:
>print ("Deleting snapshots for %s ") % vm.name
>snapshotlist = vm.snapshots.list()
>for snapshot in snapshotlist:
>if snapshot.description != "Active VM":
>time.sleep(30)
>snapshot.delete()
>try:
>while
>api.vms.get(name=vm.name).snapshots.get(id=snapshot.id).snapshot_status ==
>"locked":
>print("Waiting for snapshot %s on %s deletion to
>finish") % (snapshot.description, vm.name)
>time.sleep(60)
>except Exception as e:
>print ("Snapshot %s does not exist anymore") %
>snapshot.description
>print ("Snapshot deletion for %s done") % vm.name
>print ("Deletion of snapshots done")
>api.disconnect()
>except Exception as e:
>print ("Something went wrong when deleting the snapshots\n%s") %
>str(e)
>
>
>
><― snip ―> 
>
>
>Cheers
>Soeren
>
>
>
>
>
>On 03/06/15 15:20, "Adam Litke"  wrote:
>
>>On 03/06/15 07:36 +, Soeren Malchow wrote:
>>>Dear Adam
>>>
>>>First we were using a python script that was working on 4 threads and
>>>therefore removing 4 snapshots at the time throughout the cluster, that
>>>still caused problems.
>>>
>>>Now i took the snapshot removing out of the threaded part an i am just
>>>looping through each snapshot on each VM one after another, even with
>>>³sleeps² inbetween, but the problem remains.
>>>But i am getting the impression that it is a problem with the amount of
>>>snapshots that are deleted in a certain time, if i delete manually and
>>>one
>>>after another (meaning every 10 min or so) i do not have problems, if i
>>>delete manually and do several at once and on one VM the next one just
>>>after one finished, the risk seems to increase.
>>
>>Hmm.  In our lab we extensively tested removing a snapshot for a VM
>>with 4 disks.  This means 4 block jobs running simultaneously.  Less
>>than 10 minutes later (closer to 1 minute) we would remove a second
>>snapshot for the same VM (again involving 4 block jobs).  I guess we
>>should rerun this flow on a fully updated CentOS 7.1 host to see about
>>local reproduction.  Seems your case is much simpler than this though.
>>Is this happening every time or intermittently?
>>
>>>I do not think it is the number of VMS because we had this on hosts with
>>>only 3 or 4 Vms running
>>>
>>>I will try restarting the libvirt and see what happens.
>>>
>>>We are not using RHEL 7.1 only CentOS 7.1
>>>
>>>Is there anything else we can look at when this happens again ?
>>
>>I'll defer to Eric Blake for the libvirt side of this.  Eric, would
>>enabling debug logging in libvirtd help to shine some light on the
>>problem?
>>
>>-- 
>>Adam Litke
>
>___
>Users mailing list
>Users@ovirt.org
>http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

Hi,

This is not happening every time, the last time i had this, it was a
script runnning, and something like th 9. Vm and the 23. Vm had a problem,
and it is not always the same VMS, it is not about the OS (happen for
Windows and Linux alike)

And as i said it also happened when i tried to remove the snapshots
sequentially, here is the code (i know it is probably not the elegant way,
but i am not a developer) and the code actually has correct indentions.

<― snip ―>

print "Snapshot deletion"
try:
time.sleep(300)
Connect()
vms = api.vms.list()
for vm in vms:
print ("Deleting snapshots for %s ") % vm.name
snapshotlist = vm.snapshots.list()
for snapshot in snapshotlist:
if snapshot.description != "Active VM":
time.sleep(30)
snapshot.delete()
try:
while
api.vms.get(name=vm.name).snapshots.get(id=snapshot.id).snapshot_status ==
"locked":
print("Waiting for snapshot %s on %s deletion to
finish") % (snapshot.description, vm.name)
time.sleep(60)
except Exception as e:
print ("Snapshot %s does not exist anymore") %
snapshot.description
print ("Snapshot deletion for %s done") % vm.name
print ("Deletion of snapshots done")
api.disconnect()
except Exception as e:
print ("Something went wrong when deleting the snapshots\n%s") % str(e)



<― snip ―> 


Cheers
Soeren





On 03/06/15 15:20, "Adam Litke"  wrote:

>On 03/06/15 07:36 +, Soeren Malchow wrote:
>>Dear Adam
>>
>>First we were using a python script that was working on 4 threads and
>>therefore removing 4 snapshots at the time throughout the cluster, that
>>still caused problems.
>>
>>Now i took the snapshot removing out of the threaded part an i am just
>>looping through each snapshot on each VM one after another, even with
>>³sleeps² inbetween, but the problem remains.
>>But i am getting the impression that it is a problem with the amount of
>>snapshots that are deleted in a certain time, if i delete manually and
>>one
>>after another (meaning every 10 min or so) i do not have problems, if i
>>delete manually and do several at once and on one VM the next one just
>>after one finished, the risk seems to increase.
>
>Hmm.  In our lab we extensively tested removing a snapshot for a VM
>with 4 disks.  This means 4 block jobs running simultaneously.  Less
>than 10 minutes later (closer to 1 minute) we would remove a second
>snapshot for the same VM (again involving 4 block jobs).  I guess we
>should rerun this flow on a fully updated CentOS 7.1 host to see about
>local reproduction.  Seems your case is much simpler than this though.
>Is this happening every time or intermittently?
>
>>I do not think it is the number of VMS because we had this on hosts with
>>only 3 or 4 Vms running
>>
>>I will try restarting the libvirt and see what happens.
>>
>>We are not using RHEL 7.1 only CentOS 7.1
>>
>>Is there anything else we can look at when this happens again ?
>
>I'll defer to Eric Blake for the libvirt side of this.  Eric, would
>enabling debug logging in libvirtd help to shine some light on the
>problem?
>
>-- 
>Adam Litke

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

Dear Adam

First we were using a python script that was working on 4 threads and
therefore removing 4 snapshots at the time throughout the cluster, that
still caused problems.

Now i took the snapshot removing out of the threaded part an i am just
looping through each snapshot on each VM one after another, even with
³sleeps² inbetween, but the problem remains.
But i am getting the impression that it is a problem with the amount of
snapshots that are deleted in a certain time, if i delete manually and one
after another (meaning every 10 min or so) i do not have problems, if i
delete manually and do several at once and on one VM the next one just
after one finished, the risk seems to increase.

I do not think it is the number of VMS because we had this on hosts with
only 3 or 4 Vms running

I will try restarting the libvirt and see what happens.

We are not using RHEL 7.1 only CentOS 7.1

Is there anything else we can look at when this happens again ?

Regards
Soeren 



On 02/06/15 18:53, "Adam Litke"  wrote:

>Hello Soeren.
>
>I've started to look at this issue and I'd agree that at first glance
>it looks like a libvirt issue.  The 'cannot acquire state change lock'
>messages suggest a locking bug or severe contention at least.  To help
>me better understand the problem I have a few questions about your
>setup.
>
>From your earlier report it appears that you have 15 VMs running on
>the failing host.  Are you attempting to remove snapshots from all VMs
>at the same time?  Have you tried with fewer concurrent operations?
>I'd be curious to understand if the problem is connected to the
>number of VMs running or the number of active block jobs.
>
>Have you tried RHEL-7.1 as a hypervisor host?
>
>Rather than rebooting the host, does restarting libvirtd cause the VMs
>to become responsive again?  Note that this operation may cause the
>host to move to Unresponsive state in the UI for a short period of
>time.
>
>Thanks for your report.
>
>On 31/05/15 23:39 +, Soeren Malchow wrote:
>>And sorry, another update, it does kill the VM partly, it was still
>>pingable when i wrote the last mail, but no ssh and no spice console
>>possible
>>
>>From: Soeren Malchow
>>mailto:soeren.malc...@mcon.net>>
>>Date: Monday 1 June 2015 01:35
>>To: Soeren Malchow
>>mailto:soeren.malc...@mcon.net>>,
>>"libvirt-us...@redhat.com<mailto:libvirt-us...@redhat.com>"
>>mailto:libvirt-us...@redhat.com>>, users
>>mailto:users@ovirt.org>>
>>Subject: Re: [ovirt-users] Bug in Snapshot Removing
>>
>>Small addition again:
>>
>>This error shows up in the log while removing snapshots WITHOUT
>>rendering the Vms unresponsive
>>
>>‹
>>Jun 01 01:33:45 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1657]:
>>Timed out during operation: cannot acquire state change lock
>>Jun 01 01:33:45 mc-dc3ham-compute-02-live.mc.mcon.net vdsm[6839]: vdsm
>>vm.Vm ERROR vmId=`56848f4a-cd73-4eda-bf79-7eb80ae569a9`::Error getting
>>block job info
>> 
>>Traceback (most recent call last):
>>File
>>"/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobsŠ
>>
>>‹
>>
>>
>>
>>From: Soeren Malchow
>>mailto:soeren.malc...@mcon.net>>
>>Date: Monday 1 June 2015 00:56
>>To: "libvirt-us...@redhat.com<mailto:libvirt-us...@redhat.com>"
>>mailto:libvirt-us...@redhat.com>>, users
>>mailto:users@ovirt.org>>
>>Subject: [ovirt-users] Bug in Snapshot Removing
>>
>>Dear all
>>
>>I am not sure if the mail just did not get any attention between all the
>>mails and this time it is also going to the libvirt mailing list.
>>
>>I am experiencing a problem with VM becoming unresponsive when removing
>>Snapshots (Live Merge) and i think there is a serious problem.
>>
>>Here are the previous mails,
>>
>>http://lists.ovirt.org/pipermail/users/2015-May/033083.html
>>
>>The problem is on a system with everything on the latest version, CentOS
>>7.1 and ovirt 3.5.2.1 all upgrades applied.
>>
>>This Problem did NOT exist before upgrading to CentOS 7.1 with an
>>environment running ovirt 3.5.0 and 3.5.1 and Fedora 20 with the
>>libvirt-preview repo activated.
>>
>>I think this is a bug in libvirt, not ovirt itself, but i am not sure.
>>The actual file throwing the exception is in VDSM
>>(/usr/share/vdsm/virt/vm.py, line 697).
>>
>>We are very willing to help, test and supply log files in anyway we can.
>>
>>Regards
>>Soeren
>>
>
>>___
>>Users mailing list
>>Users@ovirt.org
>>http://lists.ovirt.org/mailman/listinfo/users
>
>
>-- 
>Adam Litke

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Live Merge vanished after upgrade to 3.5.2.1

[Soeren Malchow]

Dear Allon,

We only upgraded the engine and the vdsm on the hypervisors the OS itself 
stayed the same (Fedora 20), with 3.5.0 Live merge worked, with 3.5.2.1 not.

However, we already migrated to CentOS 7.1 on the Hypervisors since we were not 
really comfortable using the virt-preview on Fedora all the time, therefore we 
can not test anymore

Regards
Soeren

From: Allon Mureinik mailto:amure...@redhat.com>>
Date: Tuesday 2 June 2015 15:11
To: Soeren Malchow mailto:soeren.malc...@mcon.net>>
Cc: "Users@ovirt.org<mailto:Users@ovirt.org>" 
mailto:Users@ovirt.org>>, Adam Litke 
mailto:ali...@redhat.com>>
Subject: Re: [ovirt-users] Live Merge vanished after upgrade to 3.5.2.1

What have you upgraded? The engine? The hypervisors?

Can you include the results of "rpm -qa | grep ovirt" from the engine and "rpm 
-qa | egrep "vdsm|libvirt|qemu"  " on the hypervisors?

From: "Soeren Malchow" mailto:soeren.malc...@mcon.net>>
To: Users@ovirt.org<mailto:Users@ovirt.org>
Sent: Thursday, May 21, 2015 12:32:53 PM
Subject: [ovirt-users] Live Merge vanished after upgrade to 3.5.2.1

Dear all,

In our environment the “Live Merge” capability is gone after the upgrade to 
ovirt 3.5.2.1

It was working before and we had our backup relying in this.

Any idea what happened ?

Environment

Hosted Engine on CentOS 6.6 with ovirt 3.5.2.1
Compute hosts on Fedora 20 with vdsm 4.16.14 and libvirt 1.2.9.1 from the 
libvirt-preview repo (for live merge)
Storage -> CentOS 7.1 with gluster 3.6.3

Cheers
Soeren

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] gluster config in 4 node cluster

[Soeren Malchow]

Hi

You can have 150MB/s for a while, but evetually the replication has to catch up 
and you will have more and more blocks out of sync.

If you really need that much speed you need to go for 10GBit links

There are some switches e.g .the ones compatible with Open Network Linux that 
are relativley cheap when it comes to 10Gbit ( http://opennetlinux.org/hcl )

Cheers
Soeren

From: Юрий Полторацкий 
mailto:y.poltorats...@gmail.com>>
Date: Monday 1 June 2015 08:56
To: Soeren Malchow mailto:soeren.malc...@mcon.net>>
Cc: "users@ovirt.org<mailto:users@ovirt.org>" 
mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] gluster config in 4 node cluster


2015-06-01 2:32 GMT+03:00 Soeren Malchow 
mailto:soeren.malc...@mcon.net>>:
Hi

For a production environment i would not build a 2 node gluster, i would build 
at least 3 nodes to have it much easier with the quorum.

Taking into account that you can use commodity hardware i would also suggest to 
split the services, but i would go for at least 3 gluster nodes – which add up 
to at least 5 nodes for a HA system.

If you want 4 in any case, then my suggestion would be to go for 4 replicas, 
each node has anything that it needs to run and you can basically use NFS to 
localhost for the storage which would make the nodes always access the local 
storage for the VMS, availability wise it makes no difference.

I have data center with one cluster based on 3 nodes with both services: virt 
and gluster, and GlusterFS as a data domain. Every node has a single HDD as a 
brick (replica 3) and 2x1G nics with bonding on top (src-dst-ip). I have 2 
vlans: 1 - management, 2 - VMs.

If in a VM I run something like 'dd if=/dev/zero of=/tmp/tmp' I have around 80+ 
MB/s write speed, at this moment on SPM node i run 'iftop -i bond0.1' and can 
see two flows with around 900 Mb/s each of them to the second and to the third 
nodes (thanks to the bonding that every flow can be done through separate nic). 
All of this is expected result and in this case we reach a limit of bandwidth 
our nics. And even if HDD of a node has 150+ MB/s linear write speed I can not 
reach such speed of write in VMs.

What result would I get with 4 nodes and gluster replica 4? I expect that write 
speed in VM will be much slower, because now SPM node will generate three 
flows, and two of them will go through one nic and we will have only 500Mb/s 
per flow and in this way the write speed of VM will be twice lower.

Am I wrong?



Cheers
Soeren

From: Юрий Полторацкий 
mailto:y.poltorats...@gmail.com>>
Date: Sunday 31 May 2015 18:32
To: "users@ovirt.org<mailto:users@ovirt.org>" 
mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] gluster config in 4 node cluster


Hi,

As for me, I would build one cluster with gluster service only based on two 
nodes (replica 2), and the other one with virt service only based on other two 
nodes. I think this variant is more scalable in future.

PS. I am a new in oVirt, so do not except that I am wrong.


28.05.2015 23:11, p...@email.cz<mailto:p...@email.cz> пишет:
Hello,
How to optimal configure  4 node cluster for any one node goes to  maintenance 
without stopping VM ??

a) replica 4  - but it takes a lot of space
b) disperse 3+1 ( raid 5 ) - but bad performance and not visible by oVirt 3.7.2
c) stripe2+replica2 = but VM paused

any other idea ?
regs.
Pa.



___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Snapshot remove bug (this time from the GUI)

[Soeren Malchow]

Dear all,

The problem i already sent a few mails about now came up while removing a 
snapshot from the GUI, the otput in the log files was exactly the same.

In vdsm.log

Thread-189739::DEBUG::2015-06-01 
09:39:10,001::libvirtconnection::143::root::(wrapper) Unknown libvirterror: 
ecode: 68 edom: 10 level: 2 message: Timed out during operation: cannot acquire 
state change lock
Thread-189739::ERROR::2015-06-01 
09:39:10,002::vm::5761::vm.Vm::(queryBlockJobs) 
vmId=`e640e9ef-1862-425a-8bdb-9cdfb0583f04`::Error getting block job info
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs
liveInfo = self._dom.blockJobInfo(drive.name, 0)
  File "/usr/share/vdsm/virt/vm.py", line 697, in f
raise toe
TimeoutError: Timed out during operation: cannot acquire state change lock



——

journalctl

This

un 01 09:33:30 mc-dc3ham-compute-03-live.mc.mcon.net libvirtd[1801]: Cannot 
start job (modify, none) for domain fab-cms-app-01-qa-fab-mcon-net; current job 
is (modify, none) owned by (1829, 0)
Jun 01 09:33:30 mc-dc3ham-compute-03-live.mc.mcon.net libvirtd[1801]: Timed out 
during operation: cannot acquire state change lock
Jun 01 09:33:30 mc-dc3ham-compute-03-live.mc.mcon.net vdsm[7003]: vdsm root 
ERROR Unhandled exception
  Traceback 
(most recent call last):
File 
"/usr/lib/python2.7/site-packages/vdsm/utils.py", line 809, in wrapper
  return 
f(*a, **kw)
File 
"/usr/share/vdsm/virt/vm.py", line 6107, in run
  
self.tryPivot()
File 
"/usr/share/vdsm/virt/vm.py", line 6092, in tryPivot
  ret = 
self.vm._dom.blockJobAbort(self.drive.name, flags)
File 
"/usr/share/vdsm/virt/vm.py", line 697, in f
  raise toe
  TimeoutError: 
Timed out during operation: cannot acquire state change lock

An this


Jun 01 09:33:45 mc-dc3ham-compute-03-live.mc.mcon.net libvirtd[1801]: Cannot 
start job (modify, none) for domain fab-cms-app-01-qa-fab-mcon-net; current job 
is (modify, none) owned by (1829, 0)
Jun 01 09:33:45 mc-dc3ham-compute-03-live.mc.mcon.net libvirtd[1801]: Timed out 
during operation: cannot acquire state change lock
Jun 01 09:33:45 mc-dc3ham-compute-03-live.mc.mcon.net vdsm[7003]: vdsm vm.Vm 
ERROR vmId=`e640e9ef-1862-425a-8bdb-9cdfb0583f04`::Error getting block job info
  Traceback 
(most recent call last):
File 
"/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs
  liveInfo 
= self._dom.blockJobInfo(drive.name, 0)
File 
"/usr/share/vdsm/virt/vm.py", line 697, in f
  raise toe
  TimeoutError: 
Timed out during operation: cannot acquire state change lock


—

And again the only solution to get the virtual machine back was to set the 
hypervisor host into maintenance, which does not completely work because the VM 
will stay on the host for ovirt and then reboot the host, after the reboot the 
virtual machine will be ok again (or restarted of configured to be highly 
acvailable)

This is a huge problem, using a GUI command that causes the need to reboot a 
complete hypervisor really worries me.

Regards
Soeren

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

And sorry, another update, it does kill the VM partly, it was still pingable 
when i wrote the last mail, but no ssh and no spice console possible

From: Soeren Malchow mailto:soeren.malc...@mcon.net>>
Date: Monday 1 June 2015 01:35
To: Soeren Malchow mailto:soeren.malc...@mcon.net>>, 
"libvirt-us...@redhat.com<mailto:libvirt-us...@redhat.com>" 
mailto:libvirt-us...@redhat.com>>, users 
mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] Bug in Snapshot Removing

Small addition again:

This error shows up in the log while removing snapshots WITHOUT rendering the 
Vms unresponsive

—
Jun 01 01:33:45 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1657]: Timed out 
during operation: cannot acquire state change lock
Jun 01 01:33:45 mc-dc3ham-compute-02-live.mc.mcon.net vdsm[6839]: vdsm vm.Vm 
ERROR vmId=`56848f4a-cd73-4eda-bf79-7eb80ae569a9`::Error getting block job info
  Traceback 
(most recent call last):
File 
"/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs…

—



From: Soeren Malchow mailto:soeren.malc...@mcon.net>>
Date: Monday 1 June 2015 00:56
To: "libvirt-us...@redhat.com<mailto:libvirt-us...@redhat.com>" 
mailto:libvirt-us...@redhat.com>>, users 
mailto:users@ovirt.org>>
Subject: [ovirt-users] Bug in Snapshot Removing

Dear all

I am not sure if the mail just did not get any attention between all the mails 
and this time it is also going to the libvirt mailing list.

I am experiencing a problem with VM becoming unresponsive when removing 
Snapshots (Live Merge) and i think there is a serious problem.

Here are the previous mails,

http://lists.ovirt.org/pipermail/users/2015-May/033083.html

The problem is on a system with everything on the latest version, CentOS 7.1 
and ovirt 3.5.2.1 all upgrades applied.

This Problem did NOT exist before upgrading to CentOS 7.1 with an environment 
running ovirt 3.5.0 and 3.5.1 and Fedora 20 with the libvirt-preview repo 
activated.

I think this is a bug in libvirt, not ovirt itself, but i am not sure. The 
actual file throwing the exception is in VDSM (/usr/share/vdsm/virt/vm.py, line 
697).

We are very willing to help, test and supply log files in anyway we can.

Regards
Soeren

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

Small addition again:

This error shows up in the log while removing snapshots WITHOUT rendering the 
Vms unresponsive

—
Jun 01 01:33:45 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1657]: Timed out 
during operation: cannot acquire state change lock
Jun 01 01:33:45 mc-dc3ham-compute-02-live.mc.mcon.net vdsm[6839]: vdsm vm.Vm 
ERROR vmId=`56848f4a-cd73-4eda-bf79-7eb80ae569a9`::Error getting block job info
  Traceback 
(most recent call last):
File 
"/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs…

—



From: Soeren Malchow mailto:soeren.malc...@mcon.net>>
Date: Monday 1 June 2015 00:56
To: "libvirt-us...@redhat.com<mailto:libvirt-us...@redhat.com>" 
mailto:libvirt-us...@redhat.com>>, users 
mailto:users@ovirt.org>>
Subject: [ovirt-users] Bug in Snapshot Removing

Dear all

I am not sure if the mail just did not get any attention between all the mails 
and this time it is also going to the libvirt mailing list.

I am experiencing a problem with VM becoming unresponsive when removing 
Snapshots (Live Merge) and i think there is a serious problem.

Here are the previous mails,

http://lists.ovirt.org/pipermail/users/2015-May/033083.html

The problem is on a system with everything on the latest version, CentOS 7.1 
and ovirt 3.5.2.1 all upgrades applied.

This Problem did NOT exist before upgrading to CentOS 7.1 with an environment 
running ovirt 3.5.0 and 3.5.1 and Fedora 20 with the libvirt-preview repo 
activated.

I think this is a bug in libvirt, not ovirt itself, but i am not sure. The 
actual file throwing the exception is in VDSM (/usr/share/vdsm/virt/vm.py, line 
697).

We are very willing to help, test and supply log files in anyway we can.

Regards
Soeren

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] gluster config in 4 node cluster

[Soeren Malchow]

Hi

For a production environment i would not build a 2 node gluster, i would build 
at least 3 nodes to have it much easier with the quorum.

Taking into account that you can use commodity hardware i would also suggest to 
split the services, but i would go for at least 3 gluster nodes – which add up 
to at least 5 nodes for a HA system.

If you want 4 in any case, then my suggestion would be to go for 4 replicas, 
each node has anything that it needs to run and you can basically use NFS to 
localhost for the storage which would make the nodes always access the local 
storage for the VMS, availability wise it makes no difference.

Cheers
Soeren

From: Юрий Полторацкий 
mailto:y.poltorats...@gmail.com>>
Date: Sunday 31 May 2015 18:32
To: "users@ovirt.org" 
mailto:users@ovirt.org>>
Subject: Re: [ovirt-users] gluster config in 4 node cluster


Hi,

As for me, I would build one cluster with gluster service only based on two 
nodes (replica 2), and the other one with virt service only based on other two 
nodes. I think this variant is more scalable in future.

PS. I am a new in oVirt, so do not except that I am wrong.


28.05.2015 23:11, p...@email.cz пишет:
Hello,
How to optimal configure  4 node cluster for any one node goes to  maintenance 
without stopping VM ??

a) replica 4  - but it takes a lot of space
b) disperse 3+1 ( raid 5 ) - but bad performance and not visible by oVirt 3.7.2
c) stripe2+replica2 = but VM paused

any other idea ?
regs.
Pa.



___
Users mailing list
Users@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Bug in Snapshot Removing

[Soeren Malchow]

Dear all

I am not sure if the mail just did not get any attention between all the mails 
and this time it is also going to the libvirt mailing list.

I am experiencing a problem with VM becoming unresponsive when removing 
Snapshots (Live Merge) and i think there is a serious problem.

Here are the previous mails,

http://lists.ovirt.org/pipermail/users/2015-May/033083.html

The problem is on a system with everything on the latest version, CentOS 7.1 
and ovirt 3.5.2.1 all upgrades applied.

This Problem did NOT exist before upgrading to CentOS 7.1 with an environment 
running ovirt 3.5.0 and 3.5.1 and Fedora 20 with the libvirt-preview repo 
activated.

I think this is a bug in libvirt, not ovirt itself, but i am not sure. The 
actual file throwing the exception is in VDSM (/usr/share/vdsm/virt/vm.py, line 
697).

We are very willing to help, test and supply log files in anyway we can.

Regards
Soeren

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Upgrade path

[Soeren Malchow]

Dear Johan

Basically we did just that last week, but we created a new cluster and put it 
new hosts with CentOS 7.1 and then migrated the machines, we also made the 
experience that Live Migration between clusters (in this Case Fedora 20 -> 
CentOS 7.1) does work if not you have to reboot the hosts.

The last thing we did not due is migrated the hosted engine as well, but from 
my understanding it should basically be that same as migrating to a hosted 
engine – i will report once we go there

Cheers
Soeren


From: Johan Kooijman mailto:m...@johankooijman.com>>
Date: Thursday 28 May 2015 09:39
To: users mailto:users@ovirt.org>>
Subject: [ovirt-users] Upgrade path

Hi all,

What would be the best upgrade path for upgrading a 3.5.2 cluster from C6 to C7?
Am I right in understanding that a cluster can have mixed hosts, but once a VM 
is on a C7 host, it cannot be migrated back to C6?

--
Met vriendelijke groeten / With kind regards,
Johan Kooijman
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] VM crashes during snapshot/clone/export and show only "?"

[Soeren Malchow]

Dear all,

We are experiencing this problem over and over again with different Vms, the 
situation is as follows

  *   we are backing up all VM’s through iterating through the VM (see attached 
python file) which basically follow the recommendations
  *   This process run well for a while but at some point we get a problem with 
a random VM (it is always different machines) the backup process tries to 
remove all snapshots and this is in the log files

Vdsm.log
<— snip —>
Thread-8246::DEBUG::2015-05-27 
16:56:00,003::libvirtconnection::143::root::(wrapper) Unknown libvirterror: 
ecode: 68 edom: 10 level: 2 message: Timed out during operation: cannot acquire 
state change lock
Thread-8246::ERROR::2015-05-27 16:56:00,016::vm::5761::vm.Vm::(queryBlockJobs) 
vmId=`84da8d5e-4a9d-4272-861a-a706ebce3160`::Error getting block job info
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs
liveInfo = self._dom.blockJobInfo(drive.name, 0)
  File "/usr/share/vdsm/virt/vm.py", line 697, in f
raise toe
TimeoutError: Timed out during operation: cannot acquire state change lock
VM Channels Listener::DEBUG::2015-05-27 
16:56:00,561::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 55.
<— snip —>

Syslog / journarctl
<— snip —>
May 27 16:55:15 mc-dc3ham-compute-04-live.mc.mcon.net libvirtd[1751]: Cannot 
start job (modify, none) for domain fab-cms-app-01-live-fab-mcon-net; current 
job is (modify, none) owned by (1780, 0)
May 27 16:55:15 mc-dc3ham-compute-04-live.mc.mcon.net libvirtd[1751]: Timed out 
during operation: cannot acquire state change lock
May 27 16:55:15 mc-dc3ham-compute-04-live.mc.mcon.net vdsm[10478]: vdsm vm.Vm 
ERROR vmId=`84da8d5e-4a9d-4272-861a-a706ebce3160`::Error getting block job info
   Traceback 
(most recent call last):
 File 
"/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs
   liveInfo 
= self._dom.blockJobInfo(drive.name, 0)
 File 
"/usr/share/vdsm/virt/vm.py", line 697, in f
   raise toe
   
TimeoutError: Timed out during operation: cannot acquire state change lock
<— snip —>

The result is, that the VM is non-operational, the qemu process is running and 
ovirt show it with a “?”, this itself would not be that bad if we could recover 
from this problem, but the only thing we found so far to resolve this, is to 
put the hypervisor host in maintenance and then with the last VM with the “?” 
reboot it, we found no other way that allowed us to reboot this VM.
Even after actually killing the qemu process, there is no way to do anything 
with this VM.

I think i understand that the problem arises when 2 threads are making requests 
against the same VM, however, in the last case the VM was not doing anything 
else as far as we can see.

I found a bug that seems to be very similar (‘a little older though) in 
launchpad, but libvirt related.

https://bugs.launchpad.net/nova/+bug/1254872

These are the libvirt versions on the server

libvirt-daemon-driver-nwfilter-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-interface-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-storage-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-1.2.8-16.el7_1.3.x86_64
libvirt-lock-sanlock-1.2.8-16.el7_1.3.x86_64
libvirt-client-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-config-nwfilter-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-secret-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.3.x86_64
libvirt-python-1.2.8-7.el7_1.1.x86_64
libvirt-daemon-driver-nodedev-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-network-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-kvm-1.2.8-16.el7_1.3.x86_64

VDSM version
vdsm-python-4.16.14-0.el7.noarch
vdsm-jsonrpc-4.16.14-0.el7.noarch
vdsm-yajsonrpc-4.16.14-0.el7.noarch
vdsm-4.16.14-0.el7.x86_64
vdsm-python-zombiereaper-4.16.14-0.el7.noarch
vdsm-cli-4.16.14-0.el7.noarch
vdsm-xmlrpc-4.16.14-0.el7.noarch

Kernel
3.10.0-229.4.2.el7.x86_64


Any idea where to go ?





Regards
Soeren


From: Soeren Malchow mailto:soeren.malc...@mcon.net>>
Date: Monday 25 May 2015 22:27
To: "users@ovirt.org<mailto:users@ovirt.org>" 
mailto:users@ovirt.org>>
Subject: [ovirt-users] VM crashes during snapshot/clone/export and show only "?"

Dear all,

In version 3.5.2 on CentOS 7.1 we now have the problem that the backup script 
seems to trigger a crash of Vms, this is the second time, the first time i 
could only solve the problem by rebooting the hypervisor host and acknowleding 
“host has been rebooted”

This problems happens while removing snapshots after 

Re: [ovirt-users] "metadata not present"

[Soeren Malchow]

Our installation should not have a mismatch, it is a clean installation of
CentOS 7.2 where we only added the ovirt, epel and gluster repos.

If I understand this last mail correctly i should be able to set QoS and
everythign is fine ?


On 27/05/15 12:34, "Martin Sivak"  wrote:

>> this is a symptom of a mismatch with libvirt version.
>> It will go away once your mom-vdsm and libvirt versions are aligned.
>
>> As long as you avoid using QoS you'll be fine.
>
>Actually this is a symptom of no QoS settings. libvirt reports an error
>every time we check if there is anything available. There is nothing we
>can do about this as it has the same error code as some other important
>messages.. It does not hurt anything though.
>
>
>Doron refers to a similar message we used to see when libvirt on CentOS 6
>did not support metadata.
>
>--
>Martin Sivák
>msi...@redhat.com
>Red Hat Czech
>RHEV-M SLA / Brno, CZ
>
>- Original Message -
>> On 26/05/15 18:36, Soeren Malchow wrote:
>> > Dear all,
>> > 
>> > Now i have to ask again, i am getting this
>> > 
>> > "Thread-7313::DEBUG::2015-05-26
>> > 17:34:08,936::libvirtconnection::143::root::(wrapper) Unknown
>> > libvirterror: ecode: 80 edom: 20 level: 2 message: metadata not found:
>> > Requested metadata element is not present²
>> > 
>> > On all cluster servers in the vdsm.log all the time, it seems as if it
>> > does not really hurt, but is this a problem or not and is there
>> > something we should do ?
>> > 
>> > Regards
>> > Soeren
>> > 
>> > 
>> 
>> Hi Soeren,
>> this is a symptom of a mismatch with libvirt version.
>> It will go away once your mom-vdsm and libvirt versions are aligned.
>> 
>> As long as you avoid using QoS you'll be fine.
>> 

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] "metadata not present"

[Soeren Malchow]

Dear all,

Now i have to ask again, i am getting this

"Thread-7313::DEBUG::2015-05-26 
17:34:08,936::libvirtconnection::143::root::(wrapper) Unknown libvirterror: 
ecode: 80 edom: 20 level: 2 message: metadata not found: Requested metadata 
element is not present”

On all cluster servers in the vdsm.log all the time, it seems as if it does not 
really hurt, but is this a problem or not and is there something we should do ?

Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] VM crashes during snapshot/clone/export and show only "?"

[Soeren Malchow]

Dear all,

In version 3.5.2 on CentOS 7.1 we now have the problem that the backup script 
seems to trigger a crash of Vms, this is the second time, the first time i 
could only solve the problem by rebooting the hypervisor host and acknowleding 
“host has been rebooted”

This problems happens while removing snapshots after snapshot –> clone -> 
export procedures.

The actual qemu process is still running after following log output, but the VM 
is not responsive anymore, i can kill the process problem.

Two questions for this:

How can I avoid this problem ?
Is there a way to tell ovirt that the qemu process is gone and that the VM can 
be started again ?

<— snip —>
May 25 22:03:47 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: metadata 
not found: Requested metadata element is not present
May 25 22:03:47 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: metadata 
not found: Requested metadata element is not present
May 25 22:03:48 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: internal 
error: End of file from monitor
May 25 22:03:48 mc-dc3ham-compute-02-live.mc.mcon.net kernel: IDMZ_MC_PUBLIC: 
port 3(vnet3) entered disabled state
May 25 22:03:48 mc-dc3ham-compute-02-live.mc.mcon.net kernel: device vnet3 left 
promiscuous mode
May 25 22:03:48 mc-dc3ham-compute-02-live.mc.mcon.net kernel: IDMZ_MC_PUBLIC: 
port 3(vnet3) entered disabled state
May 25 22:03:48 mc-dc3ham-compute-02-live.mc.mcon.net kvm[22973]: 15 guests now 
active
May 25 22:03:48 mc-dc3ham-compute-02-live.mc.mcon.net systemd-machined[14412]: 
Machine qemu-mc-glpi-app-01-live.mc.mcon.net terminated.
May 25 22:04:11 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: Cannot 
start job (modify, none) for domain mc-glpi-app-01-live.mc.mcon.net; current 
job is (modify, none) own
ed by (1534, 0)
May 25 22:04:11 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: Timed out 
during operation: cannot acquire state change lock
May 25 22:04:18 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: Cannot 
start job (modify, none) for domain mc-glpi-app-01-live.mc.mcon.net; current 
job is (modify, none) own
ed by (1534, 0)
May 25 22:04:18 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: Timed out 
during operation: cannot acquire state change lock
May 25 22:04:18 mc-dc3ham-compute-02-live.mc.mcon.net libvirtd[1386]: metadata 
not found: Requested metadata element is not present
May 25 22:04:18 mc-dc3ham-compute-02-live.mc.mcon.net vdsm[3230]: vdsm vm.Vm 
ERROR vmId=`598bdf61-2f2c-4569-9513-93043890f676`::Error getting block job info
  Traceback 
(most recent call last):
File 
"/usr/share/vdsm/virt/vm.py", line 5759, in queryBlockJobs
  liveInfo 
= self._dom.blockJobInfo(drive.name, 0)
File 
"/usr/share/vdsm/virt/vm.py", line 697, in f
  raise toe
  TimeoutError: 
Timed out during operation: cannot acquire state change lock
<— snip —>


<— snip —>
May 25 22:04:18 mc-dc3ham-compute-02-live.mc.mcon.net vdsm[3230]: vdsm vm.Vm 
ERROR vmId=`598bdf61-2f2c-4569-9513-93043890f676`::Stats function failed: 

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] glusterfs takes a long time to syncing after host has been rebooted

[Soeren Malchow]

Hi,

This is strictly speaking a gluster issue, not an ovirt issue, however, first 
of all, it does take very long to resync, that is normal, second, we have vey 
good experiences so far in tuning a few of the parameters in gluster

These are the ones (at least partly) we modified

cluster.data-self-heal-algorithmdiff
cluster.background-self-heal-count  16
performance.io-thread-count 32
performance.high-prio-threads   24
performance.normal-prio-threads 24
performance.low-prio-threads16
performance.least-prio-threads  4


Check

#> gluster colume get VOLUMENAME all

For more

And even after optimization it will still take very long

Regards
Soeren



From: Юрий Полторацкий 
mailto:y.poltorats...@gmail.com>>
Date: Monday 25 May 2015 12:46
To: "users@ovirt.org" 
mailto:users@ovirt.org>>
Subject: [ovirt-users] glusterfs takes a long time to syncing after host has 
been rebooted

Hi.

I am testing oVirt 3.5.2 with 3 hosts (Dell R210). Storage type is GlusterFS 
(replicate 3): each host has a single 3TB HDD. When I put one host into 
maintence mode, then reboot it and after system has been started, the 
glusterfsd proccess takes a long time (more then hours with gigabyte network) 
to syncing. It seems to be reload all data instead of download only changes . I 
have several VMs, but only one VM was running (postfix relay) at that moment, 
so there were no lots of changes on gluster volume.

I've googled for this issue without success.

Is it normal situation? Or what can I do to resolve the problem? I can provide 
any additional info.

Thanks.
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Switch from Fedora 20 to CentOS 7.1

[Soeren Malchow]

Dear Nir,

Thanks for the answer.

The problem is not related to ovirt, vdsm or libvirt, it was in gluster and the 
secondary ovirt cluster actually had the gluster mounted correctly and saw 
everything but it could not see the files in “dom_md”, we updated all gluster 
packages to 3.7.0 and all was good.

If someone else comes up with this - first check for this.

The switch from Fedora 20 to CentOS 7.1 works just fine if all gluster is on 
3.7.0 and the ovirt is on 3.5.2.1

Cheers
Soeren 






On 22/05/15 20:59, "Nir Soffer"  wrote:

>- Original Message -----
>> From: "Soeren Malchow" 
>> To: "Jurriën Bloemen" , users@ovirt.org
>> Sent: Thursday, May 21, 2015 7:35:02 PM
>> Subject: Re: [ovirt-users] Switch from Fedora 20 to CentOS 7.1
>> 
>> Hi,
>> 
>> We now created the new Cluster based on CentOS 7.1 which went fine, then we
>> migrated 2 machines – no problem, we have Live Migration (back), Live Merge
>> and so on, all good.
>> 
>> But some additional machine have problems starting on the new cluster and
>> this happens
>> 
>> 
>> Grep for the Thread in vdsm.log
>> <— snip —>
>> 
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:21,999::vm::2264::vm.Vm::(_startUnderlyingVm)
>> vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::Start
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,003::vm::2268::vm.Vm::(_startUnderlyingVm)
>> vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::_ongoingCreations acquired
>> vdsm/vdsm.log:Thread-5475::INFO::2015-05-21
>> 18:27:22,008::vm::3261::vm.Vm::(_run)
>> vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::VM wrapper has started
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,021::task::595::Storage.TaskManager.Task::(_updateState)
>> Task=`2bc7fe9c-204a-4ab7-a116-f7fbba32bd34`::moving from state init -> state
>> preparing
>> vdsm/vdsm.log:Thread-5475::INFO::2015-05-21
>> 18:27:22,028::logUtils::44::dispatcher::(wrapper) Run and protect:
>> getVolumeSize(sdUUID=u'276e9ba7-e19a-49c5-8ad7-26711934d5e4',
>> spUUID=u'0f954891-b1cd-4f09-99ae-75d404d95f9d',
>> imgUUID=u'eae65249-e5e8-49e7-90a0-c7385e80e6ca',
>> volUUID=u'8791f6ec-a6ef-484d-bd5a-730b22b19250', options=None)
>> vdsm/vdsm.log:Thread-5475::INFO::2015-05-21
>> 18:27:22,069::logUtils::47::dispatcher::(wrapper) Run and protect:
>> getVolumeSize, Return response: {'truesize': '2696552448', 'apparentsize':
>> '2696609792'}
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,069::task::1191::Storage.TaskManager.Task::(prepare)
>> Task=`2bc7fe9c-204a-4ab7-a116-f7fbba32bd34`::finished: {'truesize':
>> '2696552448', 'apparentsize': '2696609792'}
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,069::task::595::Storage.TaskManager.Task::(_updateState)
>> Task=`2bc7fe9c-204a-4ab7-a116-f7fbba32bd34`::moving from state preparing ->
>> state finished
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,070::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)
>> Owner.releaseAll requests {} resources {}
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,070::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)
>> Owner.cancelAll requests {}
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,070::task::993::Storage.TaskManager.Task::(_decref)
>> Task=`2bc7fe9c-204a-4ab7-a116-f7fbba32bd34`::ref 0 aborting False
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,071::task::595::Storage.TaskManager.Task::(_updateState)
>> Task=`c508cf8f-9f02-43a6-a45d-2b3f1d7e66be`::moving from state init -> state
>> preparing
>> vdsm/vdsm.log:Thread-5475::INFO::2015-05-21
>> 18:27:22,071::logUtils::44::dispatcher::(wrapper) Run and protect:
>> getVolumeSize(sdUUID=u'276e9ba7-e19a-49c5-8ad7-26711934d5e4',
>> spUUID=u'0f954891-b1cd-4f09-99ae-75d404d95f9d',
>> imgUUID=u'967d966c-3653-4ff6-9299-2fb5b4197c37',
>> volUUID=u'99b085e6-6662-43ef-8ab4-40bc00e82460', options=None)
>> vdsm/vdsm.log:Thread-5475::INFO::2015-05-21
>> 18:27:22,086::logUtils::47::dispatcher::(wrapper) Run and protect:
>> getVolumeSize, Return response: {'truesize': '1110773760', 'apparentsize':
>> '1110835200'}
>> vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21
>> 18:27:22,087::task::1191::Storage.TaskManager.Task::(prepare)
>> Task=`c508cf8f-9f02-43a6-a45d-2b3f1d7e66be`::finished: {'truesi

Re: [ovirt-users] providing hosts with foreman

[Soeren Malchow]

Dear Nathanaël 

We have all this running, i cced our “Foreman Guy” maybe he can bring you on 
the right track, but i assume that he won’t help before Tuesday

Cheers
Soeren 



On 22/05/15 18:57, "users-boun...@ovirt.org on behalf of Nathanaël Blanchet" 
 wrote:

>Hello Yaniv,
>
>Okay for the DNS proxy, everything works now as expected.
>I have a new question now about the install host workflow.
>The host is in "installing OS" state, and on the host side it has been 
>successfully installed. But now how to complete the host vdms 
>installation and registration?
>I found this on your wiki :
>"For other OS - at first step won't do the registration by themselves, 
>but foreman will do that using a plugin (plugin will send REST-API call 
>to add or approve the host) "
>After investigation, I found that this plugin was this one : 
>ruby193-rubygem-ovirt_provision_plugin (and with foreman 1.8 we can 
>activate it with foreman-installer now)
>
>But nothing happens once the OS is installed et the state is stuck on 
>"installing OS". Communication between foreman and engine is okay and 
>without firewall issue... I found nothing significative into the foreman 
>logs or somewhere else...
>What is it supposed to happen at this step?
>
>
>Le 17/05/2015 17:35, ybronhei a écrit :
>> Hey Nathenael,
>>
>> On 05/13/2015 06:28 PM, Nathanaël Blanchet wrote:
>>> Hi all,
>>>
>>> I've setup a foreman server, but when adding a new host by "discovered
>>> hosts", I can't modify the address item which is default filled with a
>>> built "mac-DNS".
>>
>> Not exactly, it set the address field to be the name you choose for 
>> the host dot (.) the domain that related to the picked host-group
>>
>>> In ovirt setup, I want to identify my future hosts by their IP and not
>>> their unknown DNS name like it is described here:
>>> http://www.ovirt.org/Features/ForemanIntegration.
>>
>> IP addresses can and should be dynamic based on your DHCP server 
>> configuration, but DNS name should stay the same. Adding the host that 
>> way to engine uses satellite to configure its DNS entry and other 
>> network configurations. That's why we lock the address field and fill 
>> it with the future FQDN.
>>
>>> How can I setup foreman to do such a thing? Is the setup of the DNS
>>> proxy related?
>>
>> Yes, the DNS setup is related to it. We depend on it. Using IP address 
>> might brake the integration between engine and satellite when the DHCP 
>> service is configured with address ttl option and can give the host 
>> different IP address in next boot. So currently we don't support 
>> address modification with Discovered\Provisioned Hosts
>>
>>>
>>
>> If that answer is not clear feel free to ping me in irc (ybronhei 
>> @freenode #ovirt) or reply here
>>
>> Regards,
>>
>
>-- 
>Nathanaël Blanchet
>
>Supervision réseau
>Pôle Infrastrutures Informatiques
>227 avenue Professeur-Jean-Louis-Viala
>34193 MONTPELLIER CEDEX 5  
>Tél. 33 (0)4 67 54 84 55
>Fax  33 (0)4 67 54 84 14
>blanc...@abes.fr
>
>___
>Users mailing list
>Users@ovirt.org
>http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Switch from Fedora 20 to CentOS 7.1

[Soeren Malchow]

ve 
users)
vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21 
18:27:24,112::resourceManager::641::Storage.ResourceManager::(releaseResource) 
Resource 'Storage.276e9ba7-e19a-49c5-8ad7-26711934d5e4' is free, finding out if 
anyone is waiting for it.
vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21 
18:27:24,113::resourceManager::649::Storage.ResourceManager::(releaseResource) 
No one is waiting for resource 'Storage.276e9ba7-e19a-49c5-8ad7-26711934d5e4', 
Clearing records.
vdsm/vdsm.log:Thread-5475::ERROR::2015-05-21 
18:27:24,113::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': 
{'message': "Volume does not exist: 
(u'8791f6ec-a6ef-484d-bd5a-730b22b19250',)", 'code': 201}}
vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21 
18:27:24,114::vm::2294::vm.Vm::(_startUnderlyingVm) 
vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::_ongoingCreations released
vdsm/vdsm.log:Thread-5475::ERROR::2015-05-21 
18:27:24,114::vm::2331::vm.Vm::(_startUnderlyingVm) 
vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::The vm start process failed
vdsm/vdsm.log:Thread-5475::DEBUG::2015-05-21 
18:27:24,117::vm::2786::vm.Vm::(setDownStatus) 
vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::Changed state to Down: Bad volume 
specification {u'index': 0, u'iface': u'virtio', u'type': u'disk', u'format': 
u'cow', u'bootOrder': u'1', u'address': {u'slot': u'0x06', u'bus': u'0x00', 
u'domain': u'0x', u'type': u'pci', u'function': u'0x0'}, u'volumeID': 
u'8791f6ec-a6ef-484d-bd5a-730b22b19250', 'apparentsize': '2696609792', 
u'imageID': u'eae65249-e5e8-49e7-90a0-c7385e80e6ca', u'specParams': {}, 
u'readonly': u'false', u'domainID': u'276e9ba7-e19a-49c5-8ad7-26711934d5e4', 
'reqsize': '0', u'deviceId': u'eae65249-e5e8-49e7-90a0-c7385e80e6ca', 
'truesize': '2696552448', u'poolID': u'0f954891-b1cd-4f09-99ae-75d404d95f9d', 
u'device': u'disk', u'shared': u'false', u'propagateErrors': u'off', 
u'optional': u'false'} (code=1)

<— snip —>


Additionally i can find this

—
Thread-5475::ERROR::2015-05-21 
18:27:24,107::task::866::Storage.TaskManager.Task::(_setError) 
Task=`7ca2f743-09b1-4499-b9e4-5f640002a2bc`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 3235, in prepareImage
raise se.VolumeDoesNotExist(leafUUID)
VolumeDoesNotExist: Volume does not exist: 
(u'8791f6ec-a6ef-484d-bd5a-730b22b19250’,)

—


—
Thread-5475::ERROR::2015-05-21 
18:27:24,114::vm::2331::vm.Vm::(_startUnderlyingVm) 
vmId=`24bd5074-64fc-4aa0-87cd-5de3dd7b50d1`::The vm start process failed
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 2271, in _startUnderlyingVm
self._run()
  File "/usr/share/vdsm/virt/vm.py", line 3266, in _run
self.preparePaths(devices[DISK_DEVICES])
  File "/usr/share/vdsm/virt/vm.py", line 2353, in preparePaths
drive['path'] = self.cif.prepareVolumePath(drive, self.id)
  File "/usr/share/vdsm/clientIF.py", line 277, in prepareVolumePath
raise vm.VolumeError(drive)
VolumeError: Bad volume specification {u'index': 0, u'iface': u'virtio', 
u'type': u'disk', u'format': u'cow', u'bootOrder': u'1', u'address': {u'slot': 
u'0x06', u'bus': u'0x00', u'domain': u'0x', u'type': u'pci', u'function': 
u'0x0'}, u'volumeID': u'8791f6ec-a6ef-484d-bd5a-730b22b19250', 'apparentsize': 
'2696609792', u'imageID': u'eae65249-e5e8-49e7-90a0-c7385e80e6ca', 
u'specParams': {}, u'readonly': u'false', u'domainID': 
u'276e9ba7-e19a-49c5-8ad7-26711934d5e4', 'reqsize': '0', u'deviceId': 
u'eae65249-e5e8-49e7-90a0-c7385e80e6ca', 'truesize': '2696552448', u'poolID': 
u'0f954891-b1cd-4f09-99ae-75d404d95f9d', u'device': u'disk', u'shared': 
u'false', u'propagateErrors': u'off', u'optional': u'false'}
Thread-5475::DEBUG::2015-05-21 18:27:24,117::v

[ovirt-users] Live Merge vanished after upgrade to 3.5.2.1

[Soeren Malchow]

Dear all,

In our environment the “Live Merge” capability is gone after the upgrade to 
ovirt 3.5.2.1

It was working before and we had our backup relying in this.

Any idea what happened ?

Environment

Hosted Engine on CentOS 6.6 with ovirt 3.5.2.1
Compute hosts on Fedora 20 with vdsm 4.16.14 and libvirt 1.2.9.1 from the 
libvirt-preview repo (for live merge)
Storage -> CentOS 7.1 with gluster 3.6.3

Cheers
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Linux VMs get stuck after Live Migration

[Soeren Malchow]

Dear all,

some additions:

  *   switching of memory balloonig does not solve the problem
  *   Once we migrated the machines we can basically not use anything anymore 
that needs some kind of timing – e.g. “ping 1.1.1.1 –i 2” or “while true; do 
echo “lala”; sleep”; done” both do the frist thing (1 ping and one “lala” 
output) but then stall

Cheers
Soeren

From: mailto:users-boun...@ovirt.org>> on behalf of 
Soeren Malchow
Date: Wednesday 20 May 2015 13:42
To: "users@ovirt.org<mailto:users@ovirt.org>"
Subject: [ovirt-users] Linux VMs get stuck after Live Migration

Dear all,

We are experiencing a problem with Linux Vms specifically the Vms we tested are 
CentOS 7.1 (so far it looks as if Windows Vms are not a problem), after the 
live migration the machines do not show the IP and hostname in the GUI, the 
same way as if the guest tools are not installed, but the machines are still 
running.

Then we have 2 different scenarios

Reboot/shutdown:
When i reboot/shutdown one of these VMS from inside the VM the OS seems to 
shutdown, however, the qemu process is still running and for ovirt the machine 
is still up, a “shutdown” in the frontend does not do anything, a “power off” 
shuts down the machine

Just wait:
After a while the machines just seem to shutdown and then basically then same 
happesn as with reboot/shutdown

The environment is:

Hosted Engine on CentOS 6.6 with ovirt 3.5.2.1
Compute hosts on Fedora 20 with vdsm 4.16.14 and libvirt 1.2.9.1 from the 
libvirt-preview repo (for live merge)
Storage -> CentOS 7.1 with gluster 3.6.3

Can anyone point us towards the right direction

Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Switch from Fedora 20 to CentOS 7.1

[Soeren Malchow]

Great, thanks, that is the plan then

From: mailto:users-boun...@ovirt.org>> on behalf of 
"Bloemen, Jurriën"
Date: Wednesday 20 May 2015 15:27
To: "users@ovirt.org<mailto:users@ovirt.org>"
Subject: Re: [ovirt-users] Switch from Fedora 20 to CentOS 7.1

Hi Soeren,

Yes! That works perfectly. Did it myself several times.

Regards,

Jurriën

On 20-05-15 14:19, Soeren Malchow wrote:

Hi Vered,

Thanks for the quick answer, ok, understood

Then i could create a new Cluster in the same datacenter with newly installed 
hosts and then migrate the machines through shutting them down in the old 
cluster and then starting them in the new cluster, only thing i loose is the 
live migration

Regards
Soeren



On 20/05/15 14:04, "Vered Volansky" <mailto:ve...@redhat.com> 
wrote:



Hi Soeren,

oVirt Clusters support one host distribution (all hosts must be of the same 
distribution).
If the cluster is empty at one point, you can add a host of a different 
distribution than the cluster occupied before.
But there can't be two type of distributions at the same time in one cluster.

Regards,
Vered

- Original Message -


From: "Soeren Malchow" <mailto:soeren.malc...@mcon.net>
To: users@ovirt.org<mailto:users@ovirt.org>
Sent: Wednesday, May 20, 2015 2:58:11 PM
Subject: [ovirt-users] Switch from Fedora 20 to CentOS 7.1

Dear all,

Would it be possible to switch from fedora 20 to centos 7.1 (as far as i
understood it has live merge support now) within one cluster, meaning


* take out one compute host
* Reinstall that compute host with Centos 7.1
* Do a hosted-engine —deploy
* Migrate VM to the CentOS 7.1 host
* Take the next fedora host and reinstall

Any experiences, recommendations or remarks on that ?

Regards
Soeren

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org>http://lists.ovirt.org/mailman/listinfo/users

This message (including any attachments) may contain information that is 
privileged or confidential. If you are not the intended recipient, please 
notify the sender and delete this email immediately from your systems and 
destroy all copies of it. You may not, directly or indirectly, use, disclose, 
distribute, print or copy this email or any part of it if you are not the 
intended recipient
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Switch from Fedora 20 to CentOS 7.1

[Soeren Malchow]

Hi Vered,

Thanks for the quick answer, ok, understood

Then i could create a new Cluster in the same datacenter with newly installed 
hosts and then migrate the machines through shutting them down in the old 
cluster and then starting them in the new cluster, only thing i loose is the 
live migration

Regards
Soeren 



On 20/05/15 14:04, "Vered Volansky"  wrote:

>Hi Soeren,
>
>oVirt Clusters support one host distribution (all hosts must be of the same 
>distribution).
>If the cluster is empty at one point, you can add a host of a different 
>distribution than the cluster occupied before.
>But there can't be two type of distributions at the same time in one cluster.
>
>Regards,
>Vered
>
>----- Original Message -
>> From: "Soeren Malchow" 
>> To: users@ovirt.org
>> Sent: Wednesday, May 20, 2015 2:58:11 PM
>> Subject: [ovirt-users] Switch from Fedora 20 to CentOS 7.1
>> 
>> Dear all,
>> 
>> Would it be possible to switch from fedora 20 to centos 7.1 (as far as i
>> understood it has live merge support now) within one cluster, meaning
>> 
>> 
>> * take out one compute host
>> * Reinstall that compute host with Centos 7.1
>> * Do a hosted-engine —deploy
>> * Migrate VM to the CentOS 7.1 host
>> * Take the next fedora host and reinstall
>> 
>> Any experiences, recommendations or remarks on that ?
>> 
>> Regards
>> Soeren
>> 
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>> 
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Switch from Fedora 20 to CentOS 7.1

[Soeren Malchow]

Dear all,

Would it be possible to switch from fedora 20 to centos 7.1 (as far as i 
understood it has live merge support now) within one cluster, meaning

  *   take out one compute host
  *   Reinstall that compute host with Centos 7.1
  *   Do a hosted-engine —deploy
  *   Migrate VM to the CentOS 7.1 host
  *   Take the next fedora host and reinstall

Any experiences, recommendations or remarks on that ?

Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Linux VMs get stuck after Live Migration

[Soeren Malchow]

Dear all,

We are experiencing a problem with Linux Vms specifically the Vms we tested are 
CentOS 7.1 (so far it looks as if Windows Vms are not a problem), after the 
live migration the machines do not show the IP and hostname in the GUI, the 
same way as if the guest tools are not installed, but the machines are still 
running.

Then we have 2 different scenarios

Reboot/shutdown:
When i reboot/shutdown one of these VMS from inside the VM the OS seems to 
shutdown, however, the qemu process is still running and for ovirt the machine 
is still up, a “shutdown” in the frontend does not do anything, a “power off” 
shuts down the machine

Just wait:
After a while the machines just seem to shutdown and then basically then same 
happesn as with reboot/shutdown

The environment is:

Hosted Engine on CentOS 6.6 with ovirt 3.5.2.1
Compute hosts on Fedora 20 with vdsm 4.16.14 and libvirt 1.2.9.1 from the 
libvirt-preview repo (for live merge)
Storage -> CentOS 7.1 with gluster 3.6.3

Can anyone point us towards the right direction

Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Backup through export and clone

[Soeren Malchow]

One small addition: if someone has an idea how to keep the load down even when 
clone within the gluster storage domain, any suggestions are welcome and we 
would test that

From: mailto:users-boun...@ovirt.org>> on behalf of 
Soeren Malchow
Date: Wednesday 20 May 2015 00:19
To: "users@ovirt.org<mailto:users@ovirt.org>"
Subject: [ovirt-users] Backup through export and clone

Dear all,

I am no python developer (basically no developer at all) but i struggled with 
the backup for a while but finally i have a script almost OK for the backup.

Due to the fact that the “snapshot -> export” feature is coming as it seems in 
3.6 we still need to clone and then export.

This script is threaded to do backups of 3 Vms concurrently, however, in our 
setup ( compute server with 3 separate gluster machines) parallel backups of 3 
VMS create very heavy load on the gluster storage servers, and i would like to 
do the “clone snapshot” to a separate intermediate storage already, can anybody 
help me with that ? (somewhere around line 70)
The cloning can hopefully go away in September with 3.6 but until then we still 
need that.

Any other improvements are more than welcome – please keep in mind i am not a 
developer and I only tried my very best to write a decent script.

Also: do we have a place (git repo or so) where we can put this script for 
everybody ?

I am pretty sure there is more than one person out there looking for something 
like that (at least i hope :-) )

Regards
Soeren

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Half Upgraded ovirt packages - to 3.5.2.1

[Soeren Malchow]

Thank you so much, that was it.

Is that documented somewhere for reference ?



On 19/05/15 17:15, "Sandro Bonazzola"  wrote:

>Il 19/05/2015 16:51, Soeren Malchow ha scritto:
>> Dear all,
>> 
>> I got a partially upgraded hosted engine, some of the packages are upgraded 
>> to 3.5.2.1 and some are not, i can not even upgrade afterwards
>> 
>> Package list:
>> 
>> ovirt-engine-sdk-python-3.5.2.1-1.el6.noarch
>> ovirt-engine-lib-3.5.2.1-1.el6.noarch
>> ovirt-engine-setup-plugin-ovirt-engine-3.5.2.1-1.el6.noarch
>> ovirt-engine-websocket-proxy-3.5.2.1-1.el6.noarch
>> ovirt-image-uploader-3.5.1-1.el6.noarch
>> ovirt-engine-jboss-as-7.1.1-1.el6.x86_64
>> ovirt-host-deploy-java-1.3.1-1.el6.noarch
>> ovirt-engine-3.5.1.1-1.el6.noarch
>> ovirt-engine-dwh-3.5.1-1.el6.noarch
>> ovirt-host-deploy-1.3.1-1.el6.noarch
>> ovirt-engine-setup-base-3.5.2.1-1.el6.noarch
>> ovirt-engine-setup-plugin-websocket-proxy-3.5.2.1-1.el6.noarch
>> ovirt-release35-004-1.noarch
>> ovirt-engine-cli-3.5.0.5-1.el6.noarch
>> ovirt-engine-backend-3.5.1.1-1.el6.noarch
>> ovirt-engine-userportal-3.5.1.1-1.el6.noarch
>> ovirt-engine-dbscripts-3.5.1.1-1.el6.noarch
>> ovirt-engine-tools-3.5.1.1-1.el6.noarch
>> ovirt-iso-uploader-3.5.2-1.el6.noarch
>> ovirt-engine-reports-setup-3.5.2-0.3.el6.noarch
>> ovirt-engine-dwh-setup-3.5.2-0.1.el6.noarch
>> ovirt-engine-setup-3.5.2.1-1.el6.noarch
>> ovirt-engine-extensions-api-impl-3.5.2.1-1.el6.noarch
>> ovirt-engine-reports-3.5.1-1.el6.noarch
>> ovirt-guest-tools-3.5.0-0.5.master.noarch
>> ovirt-engine-extension-aaa-ldap-1.0.2-1.el6.noarch
>> ovirt-engine-setup-plugin-ovirt-engine-common-3.5.2.1-1.el6.noarch
>> ovirt-engine-webadmin-portal-3.5.1.1-1.el6.noarch
>> ovirt-engine-restapi-3.5.1.1-1.el6.noarch
>> ovirt-guest-agent-1.0.10.2-1.el6.noarch
>> 
>> If i query the repo with "repoquery --show-dupes ovirt-engine” i get a 
>> complete list including the 3.5.2.1 packages but with yum i get nothing
>> 
>> ovirt-engine-0:3.5.0.1-1.el6.noarch
>> ovirt-engine-0:3.5.1-1.el6.noarch
>> ovirt-engine-0:3.5.1.1-1.el6.noarch
>> ovirt-engine-0:3.5.2-1.el6.noarch
>> ovirt-engine-0:3.5.2.1-1.el6.noarch
>> 
>> Unfortunately i have never seen such a behaviour before and i am list. 
>> 
>> Does anybody have an idea ?
>
>some of the packages are under version locking. In order to upgrade the 
>remaining rpms, you need to run engine-setup.
>It will take care of the update.
>
>
>
>> 
>> Regards
>> Soeren 
>> 
>> 
>> 
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>> 
>
>
>-- 
>Sandro Bonazzola
>Better technology. Faster innovation. Powered by community collaboration.
>See how it works at redhat.com
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Backup through export and clone

[Soeren Malchow]

Dear all,

I am no python developer (basically no developer at all) but i struggled with 
the backup for a while but finally i have a script almost OK for the backup.

Due to the fact that the “snapshot -> export” feature is coming as it seems in 
3.6 we still need to clone and then export.

This script is threaded to do backups of 3 Vms concurrently, however, in our 
setup ( compute server with 3 separate gluster machines) parallel backups of 3 
VMS create very heavy load on the gluster storage servers, and i would like to 
do the “clone snapshot” to a separate intermediate storage already, can anybody 
help me with that ? (somewhere around line 70)
The cloning can hopefully go away in September with 3.6 but until then we still 
need that.

Any other improvements are more than welcome – please keep in mind i am not a 
developer and I only tried my very best to write a decent script.

Also: do we have a place (git repo or so) where we can put this script for 
everybody ?

I am pretty sure there is more than one person out there looking for something 
like that (at least i hope :-) )

Regards
Soeren

#!/usr/bin/python

import Queue
import threading
import time
from ovirtsdk.api import API
from ovirtsdk.xml import params
import sys
import datetime
import smtplib
from email.mime.text import MIMEText


global SNAPSHOT_NAME

VERSION = params.Version(major='3', minor='0')
ENGINE_SERVER   = ''
ENGINE_USER = 'admin@internal'
ENGINE_PASSWORD = ''
ENGINE_CERT = ''
NOW = datetime.datetime.now()
SNAPSHOT_NAME   = 'BACKUP_' + NOW.strftime("%Y-%m-%d-%H%M")
DAY_OF_WEEK = NOW.strftime("%w")
BACKUP  = "FULL"

exitFlag = 0

class myThread (threading.Thread):
def __init__(self, threadID, name, q):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.q = q
self.api = api
global message
def run(self):
print "Starting " + self.name
process_data(self.name, self.q)
print "Exiting " + self.name

def process_data(threadName, q):
while not exitFlag:
queueLock.acquire()
if not workQueue.empty():
data = q.get()
queueLock.release()
print "%s processing %s" % (threadName, data.name)
vm = api.vms.get(name=data.name)
vmname = data.name +"_"
newvmname = vmname + SNAPSHOT_NAME
cluster = api.clusters.get(id=vm.cluster.id)
dc = api.datacenters.get(id=cluster.data_center.id)
export = None
for sd in dc.storagedomains.list():
if sd.type_ == "export":
export = sd
if not export:
print("Export domain required, and none found, exitting...\n")
sys.exit(1)

if vm.name != "HostedEngine":
vm.snapshots.add(params.Snapshot(description=SNAPSHOT_NAME, vm=vm ))
snap = vm.snapshots.list(description=SNAPSHOT_NAME)[0]
while vm.snapshots.get(id=snap.id).snapshot_status == "locked":
print("%s Waiting for snapshot of %s to finish") % (threadName, vm.name)
time.sleep(10)
print("%s Snapshotting %s is done") % (threadName,vm.name)
try:
snapshots = params.Snapshots(snapshot=[params.Snapshot(id=snap.id)])
api.vms.add(params.VM(name=newvmname, snapshots=snapshots, cluster=cluster, template=api.templates.get(name="Blank")))
while api.vms.get(name=newvmname).status.state == "image_locked":
print("%s Waiting for clone of %s to finish") % (threadName, vm.name)
time.sleep(60)
print("%s Cloning of %s  done") % (threadName, vm.name)
api.vms.get(name=newvmname).export(params.Action(storage_domain=export))
while api.vms.get(name=newvmname).status.state == "image_locked":
print("%s Waiting for export of %s finish") % (threadName, vm.name)
time.sleep(60)
print("%s Exporting %s done") % (threadName, vm.name)
api.vms.get(name=newvmname).delete()
except Exception as e:
print ("Something went wrong with the coling or exporting\n%s") % str(e)
snapshotlist = vm.snapshots.list()
for snapshot in snapshotlist:
if snapshot.description != "Active VM":
snapshot.delete()
time.sleep(3)
try:
while api.vms.get(name=vm.name).snapshots.get(id=snapshot.id).snapshot_status == "locked":
print("%s Waiting for snapshot %s on %s deletion to finish") % (threadName, snapshot.name, vm.name)
  

[ovirt-users] Half Upgraded ovirt packages - to 3.5.2.1

[Soeren Malchow]

Dear all,

I got a partially upgraded hosted engine, some of the packages are upgraded to 
3.5.2.1 and some are not, i can not even upgrade afterwards

Package list:

ovirt-engine-sdk-python-3.5.2.1-1.el6.noarch
ovirt-engine-lib-3.5.2.1-1.el6.noarch
ovirt-engine-setup-plugin-ovirt-engine-3.5.2.1-1.el6.noarch
ovirt-engine-websocket-proxy-3.5.2.1-1.el6.noarch
ovirt-image-uploader-3.5.1-1.el6.noarch
ovirt-engine-jboss-as-7.1.1-1.el6.x86_64
ovirt-host-deploy-java-1.3.1-1.el6.noarch
ovirt-engine-3.5.1.1-1.el6.noarch
ovirt-engine-dwh-3.5.1-1.el6.noarch
ovirt-host-deploy-1.3.1-1.el6.noarch
ovirt-engine-setup-base-3.5.2.1-1.el6.noarch
ovirt-engine-setup-plugin-websocket-proxy-3.5.2.1-1.el6.noarch
ovirt-release35-004-1.noarch
ovirt-engine-cli-3.5.0.5-1.el6.noarch
ovirt-engine-backend-3.5.1.1-1.el6.noarch
ovirt-engine-userportal-3.5.1.1-1.el6.noarch
ovirt-engine-dbscripts-3.5.1.1-1.el6.noarch
ovirt-engine-tools-3.5.1.1-1.el6.noarch
ovirt-iso-uploader-3.5.2-1.el6.noarch
ovirt-engine-reports-setup-3.5.2-0.3.el6.noarch
ovirt-engine-dwh-setup-3.5.2-0.1.el6.noarch
ovirt-engine-setup-3.5.2.1-1.el6.noarch
ovirt-engine-extensions-api-impl-3.5.2.1-1.el6.noarch
ovirt-engine-reports-3.5.1-1.el6.noarch
ovirt-guest-tools-3.5.0-0.5.master.noarch
ovirt-engine-extension-aaa-ldap-1.0.2-1.el6.noarch
ovirt-engine-setup-plugin-ovirt-engine-common-3.5.2.1-1.el6.noarch
ovirt-engine-webadmin-portal-3.5.1.1-1.el6.noarch
ovirt-engine-restapi-3.5.1.1-1.el6.noarch
ovirt-guest-agent-1.0.10.2-1.el6.noarch

If i query the repo with "repoquery --show-dupes ovirt-engine” i get a complete 
list including the 3.5.2.1 packages but with yum i get nothing

ovirt-engine-0:3.5.0.1-1.el6.noarch
ovirt-engine-0:3.5.1-1.el6.noarch
ovirt-engine-0:3.5.1.1-1.el6.noarch
ovirt-engine-0:3.5.2-1.el6.noarch
ovirt-engine-0:3.5.2.1-1.el6.noarch

Unfortunately i have never seen such a behaviour before and i am list.

Does anybody have an idea ?

Regards
Soeren

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] bonding interfaces

[Soeren Malchow]

Hi Bill,

First, i would strongly suggest that you check whether the interfaces are 
explcicitly NOT managed by NetworkManager, thats the way we do it.

We have something like this

—

DEVICE=ovirtmgmt
TYPE=Bridge
DELAY=0
STP=off
ONBOOT=yes
IPADDR=
NETMASK=
GATEWAY=
BOOTPROTO=none
DEFROUTE=yes
NM_CONTROLLED=no
HOTPLUG=no



And

—
DEVICE=bond0
BONDING_OPTS='miimon=100 updelay=0 downdelay=0 mode=802.3ad 
xmit_hash_policy=layer3+4'
BRIDGE=ovirtmgmt
ONBOOT=yes
NM_CONTROLLED=no
HOTPLUG=no
---

Physical interfaces

—

DEVICE=p3p3
HWADDR=
MASTER=bond0
SLAVE=yes
ONBOOT=yes
MTU=1500
NM_CONTROLLED=no

—

And we actually create all the interfaces BEFORE ovirt installation even though 
the files will be regenerated after installing and configuring ovirt (vdsm)

That works like a charm for us

Cheers
Soeren


From: Bill Dossett mailto:bill.doss...@pb.com>>
Date: Thursday 16 April 2015 21:49
To: "users@ovirt.org" 
mailto:users@ovirt.org>>
Subject: [ovirt-users] bonding interfaces

Hi,

I had a Gluster Cluster running that I setup with oVirt engine… “had” being the 
operative word here.

Today I connected the second network interface and then went to the host and 
tried to bond this interface to the ovirt mgmt. network… this is where the 
“had” part comes in ;-)

Now I have no working interfaces at all, can’t ping in or out and all 
interfaces say they are down, en1, 2, ovrtmgmt and bond0 when I do an ip add 
(this is centos7)

I have done this quite a few times on RHEV clusters and it always worked well, 
very impressed with it.

The differences are…

RHEV cluster I was using trunked ports and VLANS
RHEV I was using 6.4 or 6.5 RHEL

I think that’s it.

I haven’t really dug into the /etc/sysconfig/network-scripts to see what 
normally goes into the config files to create a bonded interface – only to see 
how to configure my first interface with a vlan and then I let ovirtengine do 
the rest for me as it worked so wellI

I guess I may have to rip it out and start over unless anyone has any tips on 
fixing it.  It’s just PoC anyway, so learning.

Thanks


Bill Dossett
Systems Architect

Tech Central – Global Engineering Services

T +1 303 440 3523
M +44 (0)777 590 8612
bill.doss...@pb.com
pitneybowes.com

Pitney Bowes
4750 Walnut Street | Boulder, Colorado, 80301 | USA




In Engineering?
Raise a ticket via Remedy Anywhere 
[HERE]
 takes less than a minute

CloudForms User Guide available 
[HERE]






___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Hosted Engine Migration fails

[Soeren Malchow]

,name=org.qemu.guest_agent.0
-device usb-tablet,id=input0
-vnc IP-ADDRESS:2,password
-k de
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2
-incoming tcp:[::]:49152
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7
-msg timestamp=on
<-- snip -->

Here is the content of the .meta file in the directories of the 2 images above

<--snip-->
DOMAIN=276e9ba7-e19a-49c5-8ad7-26711934d5e4
VOLTYPE=LEAF
CTIME=1424445378
FORMAT=RAW
IMAGE=c37bfa94-718c-4125-9202-bc299535eca5
DISKTYPE=2
PUUID=----
LEGALITY=LEGAL
MTIME=0
POOL_UUID=
DESCRIPTION={"DiskAlias":"al-exchange-01_Disk1","DiskDescription":"System"}
TYPE=SPARSE
SIZE=146800640
EOF
<--snip-->

<--snip-->
DOMAIN=276e9ba7-e19a-49c5-8ad7-26711934d5e4
CTIME=1424785898
FORMAT=RAW
DISKTYPE=2
LEGALITY=LEGAL
SIZE=419430400
VOLTYPE=LEAF
DESCRIPTION={"DiskAlias":"al-exchange-01_Disk2","DiskDescription":"Data"}
IMAGE=75134ccc-b74e-4955-90a5-95d4ceff403b
PUUID=----
MTIME=0
POOL_UUID=
TYPE=SPARSE
EOF
<--snip-->








From: Roy Golan [mailto:rgo...@redhat.com]
Sent: Wednesday, February 18, 2015 12:12 PM
To: Soeren Malchow; users@ovirt.org
Subject: Re: [ovirt-users] Hosted Engine Migration fails

On 02/16/2015 04:55 AM, Soeren Malchow wrote:
Dear all,

we ahve a setup with several hosts running fedora 10 with the virt-preview 
packages installed (for snapshot live merge) and a hosted engine running centos 
6.6-

We are experiencing a problem with the Live Migration of the Hosted Engine, in 
the case of setting the host for the Engine into maintenance as well as a 
manual migration.
I tried this on the "ovirtmgmt" network and when that failed I did some 
research and tried to use another network interface (separate from ovirtmgmt on 
layer 2), this also fails.

It looks as if the migration is still going through the ovirtmgmt interface, at 
least judging from the network traffic, and I think the error that I found 
(RHBZ#919201) is actually the right one.
 --
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:25,743::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (35MiB) > lowmark (15MiB). Refer to RHBZ#919201.
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:35,745::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (129MiB) > lowmark (15MiB). Refer to RHBZ#919201.
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:45,747::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (42MiB) > lowmark (15MiB). Refer to RHBZ#919201.
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:55,749::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (88MiB) > lowmark (15MiB). Refer to RHBZ#919201.
--
The ovirtmgmt interface is 2 x 1Gbit (LACP connected to Dell Switches with 
MLAG) and by far not fully utilized.
Can anyone help me where to go form here ?

still need the whole log. probably the guest (engine) is doing lots of memory 
i/o if you got a fair amount of running VMs and Hosts.  that will stalls the 
migration because
the guest pages are getting dirty faster than qemu can copy.


you have 2 options:

1. try several more times.

2. shutdown the engine vm, it should start on another host


Regards
Soeren




___

Users mailing list

Users@ovirt.org<mailto:Users@ovirt.org>

http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Backup solution using the API

[Soeren Malchow]

Dear Liron,

thanks alot, i know, i was following that, however the Live Merge Feature was 
missing in CentOS 7, that was already the discussion.

Now on Fedora 20 the Live Merge works (we have to stick to Fedora then until 
this is in CentOS 7.1 ?! )

I am experimenting with scripts to iterate through all machines and do a backup 
for all machines on a separate VM, I will let you know how that goes

Cheers
Soeren 

-Original Message-
From: Liron Aravot [mailto:lara...@redhat.com] 
Sent: Monday, February 16, 2015 10:15 AM
To: Soeren Malchow
Cc: Adam Litke; Nir Soffer; Thomas Keppler (PEBA); users@ovirt.org
Subject: Re: [ovirt-users] Backup solution using the API



- Original Message -
> From: "Soeren Malchow" 
> To: "Adam Litke" , "Nir Soffer" 
> 
> Cc: "Thomas Keppler (PEBA)" , users@ovirt.org
> Sent: Wednesday, February 11, 2015 7:30:48 PM
> Subject: Re: [ovirt-users] Backup solution using the API
> 
> Dear all,
> 
> i am a little lost, i tried quite a few things with the snapshots, so 
> far with python scripts I can iterate through the existing machines, 
> take one, make a snapshot and all this.
> 
> However, there are 2 problems I can not get around:
> 
> 1. even when on 3.5.1 I can not delete a snapshot on a running VM, if 
> I understood that correctly this relies on the Live Merge Feature 
> where the code is available in vdsm already but it needs a certain 
> libvirt version !?!?
> So question here is, can I delete a snapshot or not ? can I use only 
> the rest API not python  (excuse me I am not a developer)
> 
> 2. when I attach a snapshot to another virtual machine, how do I do 
> the backup then ? Does anybody have this already ?
Hi Soeren,
you can find detailed example for Backup/Restore flows here - 
http://www.ovirt.org/Features/Backup-Restore_API_Integration

thanks,
laravot.
> 
> The environment is running on CentOS 7 (hypervisors), Centos 6 (hosted 
> engine), the ovirt is on version 3.5.1, also we use gluster as a 
> storage backend where the gluster servers are managed within the 
> hosted engine in a separate cluster exporting the storage only.
> 
> Regards
> Soeren
> 
> 
> -Original Message-
> From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On 
> Behalf Of Adam Litke
> Sent: Tuesday, January 6, 2015 5:04 PM
> To: Nir Soffer
> Cc: Thomas Keppler (PEBA); users@ovirt.org
> Subject: Re: [ovirt-users] Backup solution using the API
> 
> On 15/12/14 04:31 -0500, Nir Soffer wrote:
> >- Original Message -
> >> From: "Blaster" 
> >> To: "Thomas Keppler (PEBA)" , 
> >> users@ovirt.org
> >> Sent: Wednesday, December 10, 2014 8:06:58 PM
> >> Subject: Re: [ovirt-users] Backup solution using the API
> >>
> >> On 11/27/2014 9:12 AM, Keppler, Thomas (PEBA) wrote:
> >>
> >>
> >>
> >> Now, before I go into any more hassle, has somebody else of you 
> >> done a live-backup solution for oVirt? Are there any recommendations?
> >> Thanks for any help provided!
> >>
> >>
> >> I've been looking for a similar scheme for the last year. It was 
> >> not
> >> (really) possible in the past as there wasn't any way to destroy a 
> >> snapshot w/o shutting down the VM. Is this still the case, or are 
> >> snap shots fully implemented now?
> >>
> >> Basically, I'd like to:
> >> Tell VM to flush it's buffers
> >> suspend VM
> >> take snap shot of boot virtual disk resume VM
> >
> >You don't need to suspend the vm, qemu can create live snapshot. When 
> >the snapshot is done, you don't care about future io, it will simply 
> >not included in the backup.
> >
> >> backup the virtual boot disk from the Hypervisor using standard 
> >> commands (tar, cp, whatever)
> >
> >You can by attaching the snapshot to another vm
> >
> >> destroy the snapshot
> >
> >You can in ovirt 3.5.1 - we do not depend any more on future libvirt 
> >features.
> >Adam, can you confirm?
> 
> Yes, on 3.5.1 you'll be able to destroy the snapshot without impacting 
> the running VM.
> 
> >>
> >> This would at least give some BMR capabilities of your VMs.
> >>
> >> Ideally, I'd also like to be able to create a snapshot from within 
> >> the VM, do
> >
> >You can do this "within" the vm by using the engine REST API. But 
> >this can be fragile - for example, if the vm pauses, your backup tool 
> >within the vm will never complete :-)
> >
> >> a yum u

[ovirt-users] Hosted Engine Migration fails

[Soeren Malchow]

Dear all,

we ahve a setup with several hosts running fedora 10 with the virt-preview 
packages installed (for snapshot live merge) and a hosted engine running centos 
6.6-

We are experiencing a problem with the Live Migration of the Hosted Engine, in 
the case of setting the host for the Engine into maintenance as well as a 
manual migration.
I tried this on the "ovirtmgmt" network and when that failed I did some 
research and tried to use another network interface (separate from ovirtmgmt on 
layer 2), this also fails.

It looks as if the migration is still going through the ovirtmgmt interface, at 
least judging from the network traffic, and I think the error that I found 
(RHBZ#919201) is actually the right one.
 --
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:25,743::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (35MiB) > lowmark (15MiB). Refer to RHBZ#919201.
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:35,745::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (129MiB) > lowmark (15MiB). Refer to RHBZ#919201.
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:45,747::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (42MiB) > lowmark (15MiB). Refer to RHBZ#919201.
vdsm.log:Thread-6745::WARNING::2015-02-16 
03:22:55,749::migration::458::vm.Vm::(monitor_migration) vmId=`XXX`::Migration 
stalling: remaining (88MiB) > lowmark (15MiB). Refer to RHBZ#919201.
--
The ovirtmgmt interface is 2 x 1Gbit (LACP connected to Dell Switches with 
MLAG) and by far not fully utilized.
Can anyone help me where to go form here ?
Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Backup solution using the API

[Soeren Malchow]

Dear all,

thank you for the help, right now we are at a state where we can do snapshots 
and remove them, since there are no ovirt packages for Fedora 21 we are using 
fedora 20 + virt-preview.

Everything is working fine from a spanshot perspective.

Regards
Soeren

From: Gianluca Cecchi [mailto:gianluca.cec...@gmail.com]
Sent: Wednesday, February 11, 2015 10:23 PM
To: Adam Litke
Cc: users; Soeren Malchow; Thomas Keppler (PEBA)
Subject: Re: [ovirt-users] Backup solution using the API


Il 11/Feb/2015 19:10 "Adam Litke" mailto:ali...@redhat.com>> 
ha scritto:
>
> On 11/02/15 17:30 +, Soeren Malchow wrote:
>>
>> Dear all,
>>
>> i am a little lost, i tried quite a few things with the snapshots, so
>> far with python scripts I can iterate through the existing machines,
>> take one, make a snapshot and all this.
>>
>> However, there are 2 problems I can not get around:
>>
>> 1. even when on 3.5.1 I can not delete a snapshot on a running VM, if
>> I understood that correctly this relies on the Live Merge Feature
>> where the code is available in vdsm already but it needs a certain
>> libvirt version !?!?  So question here is, can I delete a snapshot or
>> not ? can I use only the rest API not python  (excuse me I am not a
>> developer)
>
>
> I can answer this one at least.  For now, Centos 7 is lacking a new
> enough version of libvirt to support Live Merge (> 1.2.7).  The
> virt-preview repository for Fedora is shipping a newer libvirt.  You
> can check if vdsm will support live merge by executing the following
> on the host in question:
>
>   sudo vdsClient -s 0 getVdsCaps | grep liveMerge
>
> RHEL 7.1 will be shipping a version with the neccessary backports to
> support live merge so I imagine Centos 7.1 will pick those up
> automatically?
>
>
> --
> Adam Litke
> ___
> Users mailing list
> Users@ovirt.org<mailto:Users@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/users

I confirm that at least on f21 tc3 I was able to live delete snapshots,  also 
into the active layer.
See:
http://lists.ovirt.org/pipermail/users/2014-November/029450.html
and the whole thread.
I hope that RHEL 7.1 (and CentOS) will have the merging into the active layer 
too and not only into the chain history.

Gianluca
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Backup solution using the API

[Soeren Malchow]

Dear all,

i am a little lost, i tried quite a few things with the snapshots, so far with 
python scripts I can iterate through the existing machines, take one, make a 
snapshot and all this.

However, there are 2 problems I can not get around:

1. even when on 3.5.1 I can not delete a snapshot on a running VM, if I 
understood that correctly this relies on the Live Merge Feature where the code 
is available in vdsm already but it needs a certain libvirt version !?!?
So question here is, can I delete a snapshot or not ? can I use only the rest 
API not python  (excuse me I am not a developer)

2. when I attach a snapshot to another virtual machine, how do I do the backup 
then ? Does anybody have this already ?

The environment is running on CentOS 7 (hypervisors), Centos 6 (hosted engine), 
the ovirt is on version 3.5.1, also we use gluster as a storage backend where 
the gluster servers are managed within the hosted engine in a separate cluster 
exporting the storage only.

Regards
Soeren 


-Original Message-
From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of 
Adam Litke
Sent: Tuesday, January 6, 2015 5:04 PM
To: Nir Soffer
Cc: Thomas Keppler (PEBA); users@ovirt.org
Subject: Re: [ovirt-users] Backup solution using the API

On 15/12/14 04:31 -0500, Nir Soffer wrote:
>- Original Message -
>> From: "Blaster" 
>> To: "Thomas Keppler (PEBA)" , users@ovirt.org
>> Sent: Wednesday, December 10, 2014 8:06:58 PM
>> Subject: Re: [ovirt-users] Backup solution using the API
>>
>> On 11/27/2014 9:12 AM, Keppler, Thomas (PEBA) wrote:
>>
>>
>>
>> Now, before I go into any more hassle, has somebody else of you done 
>> a live-backup solution for oVirt? Are there any recommendations? 
>> Thanks for any help provided!
>>
>>
>> I've been looking for a similar scheme for the last year. It was not 
>> (really) possible in the past as there wasn't any way to destroy a 
>> snapshot w/o shutting down the VM. Is this still the case, or are 
>> snap shots fully implemented now?
>>
>> Basically, I'd like to:
>> Tell VM to flush it's buffers
>> suspend VM
>> take snap shot of boot virtual disk
>> resume VM
>
>You don't need to suspend the vm, qemu can create live snapshot. When 
>the snapshot is done, you don't care about future io, it will simply 
>not included in the backup.
>
>> backup the virtual boot disk from the Hypervisor using standard 
>> commands (tar, cp, whatever)
>
>You can by attaching the snapshot to another vm
>
>> destroy the snapshot
>
>You can in ovirt 3.5.1 - we do not depend any more on future libvirt features.
>Adam, can you confirm?

Yes, on 3.5.1 you'll be able to destroy the snapshot without impacting the 
running VM.

>>
>> This would at least give some BMR capabilities of your VMs.
>>
>> Ideally, I'd also like to be able to create a snapshot from within 
>> the VM, do
>
>You can do this "within" the vm by using the engine REST API. But this 
>can be fragile - for example, if the vm pauses, your backup tool within 
>the vm will never complete :-)
>
>> a yum update, see if I like it or not, if I do, then destroy the snap shot.
>
>Possible in 3.5.1 using REST API
>
>> If I don't, I want to promote the snapshot and boot from that, then 
>> destroy the "original".
>
>Same
>
>Nir

--
Adam Litke
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Backup and Restore of VMs

[Soeren Malchow]

Dear Vered,

at some point we have to start, and right now we are getting closer, even with 
the documentation it is sometime hard to find the correct place to start, 
especially without specific examples (and I have decades of experience now)

with the backup plugin that came from Lucas Vandroux we have a starting point 
right now, and we will continue form here and try to work with him on this.

Regards
Soeren


-Original Message-
From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of 
Blaster
Sent: Tuesday, December 23, 2014 5:49 PM
To: Vered Volansky
Cc: Users@ovirt.org
Subject: Re: [ovirt-users] Backup and Restore of VMs

Sounds like a Chicken/Egg problem.



On 12/23/2014 12:03 AM, Vered Volansky wrote:
> Well, real world is community...
> Maybe change the name of the thread in order to make this more clear for 
> someone from the community that might be able to could help.
> Maybe something like:
> Request for sharing real world example of VM backups.
>
> We obviously use it as part as developing, but I don't have what you're 
> asking for.
> If you try it yourself and stumble onto questions in the process, please ask 
> the list and we'll do our best to help.
>
> Best Regards,
> Vered
>
> - Original Message -
>> From: "Blaster" 
>> To: "Vered Volansky" 
>> Cc: Users@ovirt.org
>> Sent: Tuesday, December 23, 2014 5:56:13 AM
>> Subject: Re: [ovirt-users] Backup and Restore of VMs
>>
>>
>> Vered,
>>
>> It sounds like Soeren already knows about that page.  His issue seems 
>> to be, as well as the issue of others judging by comments on here, is 
>> that there aren’t any real world examples of how the API is used.
>>
>>
>>
>> On Dec 22, 2014, at 9:26 AM, Vered Volansky  wrote:
>>
>>> Please take a look at:
>>> http://www.ovirt.org/Features/Backup-Restore_API_Integration
>>>
>>> Specifically:
>>> http://www.ovirt.org/Features/Backup-Restore_API_Integration#Full_VM
>>> _Backups
>>>
>>> Regards,
>>> Vered
>>>
>>> - Original Message -
>>>> From: "Soeren Malchow" 
>>>> To: Users@ovirt.org
>>>> Sent: Friday, December 19, 2014 1:44:38 PM
>>>> Subject: [ovirt-users] Backup and Restore of VMs
>>>>
>>>>
>>>>
>>>> Dear all,
>>>>
>>>>
>>>>
>>>> ovirt: 3.5
>>>>
>>>> gluster: 3.6.1
>>>>
>>>> OS: CentOS 7 (except ovirt hosted engine = centos 6.6)
>>>>
>>>>
>>>>
>>>> i spent quite a while researching backup and restore for VMs right 
>>>> now, so far I have come up with this as a start for us
>>>>
>>>>
>>>>
>>>> - API calls to create schedule snapshots of virtual machines This 
>>>> is or short term storage and to guard against accidential deletion 
>>>> within the VM but not for storage corruption
>>>>
>>>>
>>>>
>>>> - Since we are using a gluster backend, gluster snapshots I wasn’t 
>>>> able so far to really test it since the LV needs to be thin 
>>>> provisioned and we did not do that in the setup
>>>>
>>>>
>>>>
>>>> For the API calls we have the problem that we can not find any 
>>>> existing scripts or something like that to do those snapshots (and 
>>>> i/we are not developers enough to do that).
>>>>
>>>>
>>>>
>>>> As an additional information, we have a ZFS based storage with 
>>>> deduplication that we use for other backup purposes which does a 
>>>> great job especially because of the deduplication (we can storage 
>>>> generations of backups without problems), this storage can be NFS 
>>>> exported and used as backup repository.
>>>>
>>>>
>>>>
>>>> Are there any backup and restore procedure you guys are using for 
>>>> backup and restore that works for you and can you point me into the 
>>>> right direction ?
>>>>
>>>> I am a little bit list right now and would appreciate any help.
>>>>
>>>>
>>>>
>>>> Regards
>>>>
>>>> Soeren
>>>>
>>>> ___
>>>> Users mailing list
>>>> Users@ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>> ___
>>> Users mailing list
>>> Users@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Two new plugins for oVirt

[Soeren Malchow]

Dear all,

we already translated this, however, we can not send it back until after the 
holidays, but then we are happy to share this

cheers
Soeren 

-Original Message-
From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of 
Nathanaël Blanchet
Sent: Tuesday, December 23, 2014 12:37 PM
To: Lucas Vandroux
Cc: users
Subject: Re: [ovirt-users] Two new plugins for oVirt

Hello Lucas,

I'm really interested in your vm-backup plugin, I've installed it without any 
issue but chineese is just impossible to read in the subtab menu! Is there a 
way to get it in English when compiling?
Thanks

Le 21/12/2014 14:40, Alon Bar-Lev a écrit :
> Hello Lucas,
>
> In 3.5 there is a new implementation of ldap interaction[1][2].
> The legacy implementation, including engine-manage-domain will be depreciated 
> and maintained only for regressions.
>
> Regards,
> Alon
>
> [1] 
> http://gerrit.ovirt.org/gitweb?p=ovirt-engine-extension-aaa-ldap.git;a
> =blob;f=README;hb=HEAD
> [2] http://www.ovirt.org/Features/AAA
>
> - Original Message -
>> From: "Lucas Vandroux" 
>> To: "users" 
>> Sent: Friday, December 19, 2014 8:38:05 AM
>> Subject: [ovirt-users] Two new plugins for oVirt
>>
>> Dear all,
>>
>> We developed 2 new plugins for the oVirt-Engine.
>>
>> The first one is to interact with the engine-manage-domains tool 
>> directly from WebAdmin: 
>> https://github.com/ovirt-china/manage-domains-plugin
>>
>> The second one is to schedule automatic backups of your vms:
>> https://github.com/ovirt-china/vm-backup-scheduler
>>
>> Maybe they can help you.
>>
>> Best regards,
>>
>> Lucas Vandroux for the oVirt-China Team ( http://ovirt-china.org/ )
>>
>>
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Backup and Restore of VMs

[Soeren Malchow]

Hi,

that is exactly right, altough we are getting closer now, we are just checking 
the plugins that were posted a few days ago

https://github.com/ovirt-china/vm-backup-scheduler

they seem to work pretty good so far, I will talk to a few developers after the 
holidays and see whether we can improve something, we already translated a few 
things in those plugins from mandarin to English.
We will have a look into improvements of that plugin. 

I will keep you guys posted.

Thanks
Soeren



-Original Message-
From: Blaster [mailto:blas...@556nato.com] 
Sent: Tuesday, December 23, 2014 4:56 AM
To: Vered Volansky
Cc: Soeren Malchow; Users@ovirt.org
Subject: Re: [ovirt-users] Backup and Restore of VMs


Vered,

It sounds like Soeren already knows about that page.  His issue seems to be, as 
well as the issue of others judging by comments on here, is that there aren't 
any real world examples of how the API is used.



On Dec 22, 2014, at 9:26 AM, Vered Volansky  wrote:

> Please take a look at:
> http://www.ovirt.org/Features/Backup-Restore_API_Integration
> 
> Specifically:
> http://www.ovirt.org/Features/Backup-Restore_API_Integration#Full_VM_B
> ackups
> 
> Regards,
> Vered
> 
> - Original Message -
>> From: "Soeren Malchow" 
>> To: Users@ovirt.org
>> Sent: Friday, December 19, 2014 1:44:38 PM
>> Subject: [ovirt-users] Backup and Restore of VMs
>> 
>> 
>> 
>> Dear all,
>> 
>> 
>> 
>> ovirt: 3.5
>> 
>> gluster: 3.6.1
>> 
>> OS: CentOS 7 (except ovirt hosted engine = centos 6.6)
>> 
>> 
>> 
>> i spent quite a while researching backup and restore for VMs right 
>> now, so far I have come up with this as a start for us
>> 
>> 
>> 
>> - API calls to create schedule snapshots of virtual machines This is 
>> or short term storage and to guard against accidential deletion 
>> within the VM but not for storage corruption
>> 
>> 
>> 
>> - Since we are using a gluster backend, gluster snapshots I wasn't 
>> able so far to really test it since the LV needs to be thin 
>> provisioned and we did not do that in the setup
>> 
>> 
>> 
>> For the API calls we have the problem that we can not find any 
>> existing scripts or something like that to do those snapshots (and 
>> i/we are not developers enough to do that).
>> 
>> 
>> 
>> As an additional information, we have a ZFS based storage with 
>> deduplication that we use for other backup purposes which does a 
>> great job especially because of the deduplication (we can storage 
>> generations of backups without problems), this storage can be NFS exported 
>> and used as backup repository.
>> 
>> 
>> 
>> Are there any backup and restore procedure you guys are using for 
>> backup and restore that works for you and can you point me into the right 
>> direction ?
>> 
>> I am a little bit list right now and would appreciate any help.
>> 
>> 
>> 
>> Regards
>> 
>> Soeren
>> 
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>> 
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Backup and Restore of VMs

[Soeren Malchow]

Dear all,

ovirt: 3.5
gluster: 3.6.1
OS: CentOS 7 (except ovirt hosted engine = centos 6.6)

i spent quite a while researching backup and restore for VMs right now, so far 
I have come up with this as a start for us


-  API calls to create schedule snapshots of virtual machines
This is or short term storage and to guard against accidential deletion within 
the VM but not for storage corruption


-  Since we are using a gluster backend, gluster snapshots
I wasn't able so far to really test it since the LV needs to be thin 
provisioned and we did not do that in the setup

For the API calls we have the problem that we can not find any existing scripts 
or something like that to do those snapshots (and i/we are not developers 
enough to do that).

As an additional information, we have a ZFS based storage with deduplication 
that we use for other backup purposes which does a great job especially because 
of the deduplication (we can storage generations of backups without problems), 
this storage can be NFS exported and used as backup repository.

Are there any backup and restore procedure you guys are using for backup and 
restore that works for you and can you point me into the right direction ?
I am a little bit list right now and would appreciate any help.

Regards
Soeren
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users