Re: [ovirt-users] VM pauses/hangs after migration

2016-10-13 Thread Davide Ferrari
Hello

just for the record, after I have that server replaced (only
motherboard+ram+controller, same disks), now everything works ok, so it was
definitely an hardware issue.

Thanks everyone for the troubleshoot help!


2016-10-04 18:06 GMT+02:00 Michal Skrivanek :

>
> On 3 Oct 2016, at 10:39, Davide Ferrari  wrote:
>
>
>
> 2016-09-30 15:35 GMT+02:00 Michal Skrivanek :
>
>>
>>
>> that is a very low level error really pointing at HW issues. It may or
>> may not be detected by memtest…but I would give it a try
>>
>>
> I left memtest86 running for 2 days and no error detected :(
>
>
>> The only difference that this host (vmhost01) has is that it was the
>> first host installed in my self-hosted engine installation. But I have
>> already reinstalled it from GUI and menawhile I've upgraded to 4.0.4 from
>> 4.0.3.
>>
>>
>> does it happen only for the big 96GB VM? The others which you said are
>> working, are they all small?
>> Might be worth trying other system stability tests, playing with
>> safer/slower settings in BIOS, use lower CPU cluster, etc
>>
>>
> Yep, it happens only for the 96GB VM. Other VMs with fewer RAM (16GB for
> example) can be created on or migrated to that host flawlessly. I'll try to
> play a little with BIOS settings but otherwise I'll have the HW replaced. I
> was only trying to rule out possible oVirt SW problems due to that host
> being the first I deployed (from CLI) when I installed the cluster.
>
>
> I understand. Unfortunately it really does look like some sort of
> incompatibility rather than a sw issue:/
>
>
> Thanks!
>
> --
> Davide Ferrari
> Senior Systems Engineer
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>


-- 
Davide Ferrari
Senior Systems Engineer
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] VM pauses/hangs after migration

2016-10-04 Thread Michal Skrivanek

> On 3 Oct 2016, at 10:39, Davide Ferrari  wrote:
> 
> 
> 
> 2016-09-30 15:35 GMT+02:00 Michal Skrivanek  >:
> 
> 
> that is a very low level error really pointing at HW issues. It may or may 
> not be detected by memtest…but I would give it a try
> 
> 
> I left memtest86 running for 2 days and no error detected :(
>  
>> The only difference that this host (vmhost01) has is that it was the first 
>> host installed in my self-hosted engine installation. But I have already 
>> reinstalled it from GUI and menawhile I've upgraded to 4.0.4 from 4.0.3.
> 
> does it happen only for the big 96GB VM? The others which you said are 
> working, are they all small?
> Might be worth trying other system stability tests, playing with safer/slower 
> settings in BIOS, use lower CPU cluster, etc
> 
> 
> Yep, it happens only for the 96GB VM. Other VMs with fewer RAM (16GB for 
> example) can be created on or migrated to that host flawlessly. I'll try to 
> play a little with BIOS settings but otherwise I'll have the HW replaced. I 
> was only trying to rule out possible oVirt SW problems due to that host being 
> the first I deployed (from CLI) when I installed the cluster.

I understand. Unfortunately it really does look like some sort of 
incompatibility rather than a sw issue:/

> 
> Thanks!
> 
> -- 
> Davide Ferrari
> Senior Systems Engineer
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] VM pauses/hangs after migration

2016-10-03 Thread Davide Ferrari
2016-09-30 15:35 GMT+02:00 Michal Skrivanek :

>
>
> that is a very low level error really pointing at HW issues. It may or may
> not be detected by memtest…but I would give it a try
>
>
I left memtest86 running for 2 days and no error detected :(


> The only difference that this host (vmhost01) has is that it was the first
> host installed in my self-hosted engine installation. But I have already
> reinstalled it from GUI and menawhile I've upgraded to 4.0.4 from 4.0.3.
>
>
> does it happen only for the big 96GB VM? The others which you said are
> working, are they all small?
> Might be worth trying other system stability tests, playing with
> safer/slower settings in BIOS, use lower CPU cluster, etc
>
>
Yep, it happens only for the 96GB VM. Other VMs with fewer RAM (16GB for
example) can be created on or migrated to that host flawlessly. I'll try to
play a little with BIOS settings but otherwise I'll have the HW replaced. I
was only trying to rule out possible oVirt SW problems due to that host
being the first I deployed (from CLI) when I installed the cluster.

Thanks!

-- 
Davide Ferrari
Senior Systems Engineer
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] VM pauses/hangs after migration

2016-09-30 Thread Michal Skrivanek

> On 29 Sep 2016, at 16:23, Davide Ferrari <dav...@billymob.com> wrote:
> 
> Ok, what I said is not true :( I didn't try to migrate again to the same host 
> that gave the initial problem, and the problem is still there. The 
> destination host has no HW problem (at least nothing that the system reports, 
> maybe I should try with an extensive memtest86) and the source problem now 
> has no memory issues neither. So, my question is now: how can I debug this 
> problem?

that is a very low level error really pointing at HW issues. It may or may not 
be detected by memtest…but I would give it a try

> The only difference that this host (vmhost01) has is that it was the first 
> host installed in my self-hosted engine installation. But I have already 
> reinstalled it from GUI and menawhile I've upgraded to 4.0.4 from 4.0.3.

does it happen only for the big 96GB VM? The others which you said are working, 
are they all small?
Might be worth trying other system stability tests, playing with safer/slower 
settings in BIOS, use lower CPU cluster, etc

> 
> Any idea?
> 
> 2016-09-29 13:59 GMT+02:00 Davide Ferrari <dav...@billymob.com 
> <mailto:dav...@billymob.com>>:
> Hello
> 
> Today I've the faulty DIMMs replaced, started the same VM again and did the 
> same migration and this time worked, so it was 100% due to that.
> 
> The problem that make me wonder a bit is: if it's the source host with memory 
> problem the one which blocks the correct migration, a faulty DIMM will force 
> you to stop the VMs running on that host, because you cannot simply migrate 
> them away to do the maintenence tasks...
> 
> 
> 2016-09-29 13:53 GMT+02:00 Tomas Jelinek <tjeli...@redhat.com 
> <mailto:tjeli...@redhat.com>>:
> 
> 
> - Original Message -
> > From: "Davide Ferrari" <dav...@billymob.com <mailto:dav...@billymob.com>>
> > To: "users" <users@ovirt.org <mailto:users@ovirt.org>>
> > Sent: Wednesday, September 28, 2016 2:59:59 PM
> > Subject: [ovirt-users] VM pauses/hangs after migration
> >
> > Hello
> >
> > trying to migrate a VM from one host to another, a big VM with 96GB of RAM, 
> > I
> > found that when the migration completes, the VM goes to a paused satte and
> > cannot be resumed. The libvirt/qemu log it gives is this:
> >
> > 2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section id
> > 2(ram)
> > 2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed: Input/output
> > error
> > 2016-09-28 12:18:15.872+: shutting down
> > 2016-09-28 12:22:21.467+: starting up libvirt version: 1.2.17, package:
> > 13.el7_2.5 (CentOS BuildSystem < http://bugs.centos.org 
> > <http://bugs.centos.org/> >,
> > 2016-06-23-14 :23:27, worker1.bsys.centos.org 
> > <http://worker1.bsys.centos.org/> ), qemu version: 2.3.0
> > (qemu-kvm-ev-2.3.0-31.el7.16.1)
> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name front04.billydomain.com 
> > <http://front04.billydomain.com/> -S
> > -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
> > size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
> > 32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304
> > -uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios
> > type=1,manufacturer=oVirt,product=oVirt
> > Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-004A-3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68
> > -no-user-config -nodefaults -chardev
> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/
> > domain-front04.billydomain.com/monitor.sock,server,nowait 
> > <http://domain-front04.billydomain.com/monitor.sock,server,nowait> -mon
> > chardev=charmonitor,id=monitor,mode=control -rtc
> > base=2016-09-28T14:22:21,driftfix=slew -global
> > kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
> > -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> > virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
> > virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive
> > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
> > file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/b5b49d5c-2378-4639-9469-362e37ae7473/24fd0d3c-309b-458d-9818-4321023afacf,if=none,id=drive-virtio-disk0,format=qcow2,serial=b5b49d5c-2378-4639-9469-362e37ae7473,cache=none,werror=stop,rerror=stop,aio=threads
> > -device
>

Re: [ovirt-users] VM pauses/hangs after migration

2016-09-29 Thread Davide Ferrari
Ok, what I said is not true :( I didn't try to migrate again to the same
host that gave the initial problem, and the problem is still there. The
destination host has no HW problem (at least nothing that the system
reports, maybe I should try with an extensive memtest86) and the source
problem now has no memory issues neither. So, my question is now: how can I
debug this problem? The only difference that this host (vmhost01) has is
that it was the first host installed in my self-hosted engine installation.
But I have already reinstalled it from GUI and menawhile I've upgraded to
4.0.4 from 4.0.3.

Any idea?

2016-09-29 13:59 GMT+02:00 Davide Ferrari <dav...@billymob.com>:

> Hello
>
> Today I've the faulty DIMMs replaced, started the same VM again and did
> the same migration and this time worked, so it was 100% due to that.
>
> The problem that make me wonder a bit is: if it's the source host with
> memory problem the one which blocks the correct migration, a faulty DIMM
> will force you to stop the VMs running on that host, because you cannot
> simply migrate them away to do the maintenence tasks...
>
>
> 2016-09-29 13:53 GMT+02:00 Tomas Jelinek <tjeli...@redhat.com>:
>
>>
>>
>> - Original Message -
>> > From: "Davide Ferrari" <dav...@billymob.com>
>> > To: "users" <users@ovirt.org>
>> > Sent: Wednesday, September 28, 2016 2:59:59 PM
>> > Subject: [ovirt-users] VM pauses/hangs after migration
>> >
>> > Hello
>> >
>> > trying to migrate a VM from one host to another, a big VM with 96GB of
>> RAM, I
>> > found that when the migration completes, the VM goes to a paused satte
>> and
>> > cannot be resumed. The libvirt/qemu log it gives is this:
>> >
>> > 2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section
>> id
>> > 2(ram)
>> > 2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed:
>> Input/output
>> > error
>> > 2016-09-28 12:18:15.872+: shutting down
>> > 2016-09-28 12:22:21.467+: starting up libvirt version: 1.2.17,
>> package:
>> > 13.el7_2.5 (CentOS BuildSystem < http://bugs.centos.org >,
>> > 2016-06-23-14:23:27, worker1.bsys.centos.org ), qemu version: 2.3.0
>> > (qemu-kvm-ev-2.3.0-31.el7.16.1)
>> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
>> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name
>> front04.billydomain.com -S
>> > -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
>> > size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
>> > 32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304
>> > -uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios
>> > type=1,manufacturer=oVirt,product=oVirt
>> > Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-004A-
>> 3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68
>> > -no-user-config -nodefaults -chardev
>> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/
>> > domain-front04.billydomain.com/monitor.sock,server,nowait -mon
>> > chardev=charmonitor,id=monitor,mode=control -rtc
>> > base=2016-09-28T14:22:21,driftfix=slew -global
>> > kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
>> > -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
>> > virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
>> > virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4
>> -drive
>> > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
>> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
>> > file=/rhev/data-center/0001-0001-0001-0001-03e3/
>> ba2bd397-9222-424d-aecc-eb652c0169d9/images/b5b49d5c-
>> 2378-4639-9469-362e37ae7473/24fd0d3c-309b-458d-9818-
>> 4321023afacf,if=none,id=drive-virtio-disk0,format=qcow2,
>> serial=b5b49d5c-2378-4639-9469-362e37ae7473,cache=none,
>> werror=stop,rerror=stop,aio=threads
>> > -device
>> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virti
>> o-disk0,id=virtio-disk0,bootindex=1
>> > -drive
>> > file=/rhev/data-center/0001-0001-0001-0001-03e3/
>> ba2bd397-9222-424d-aecc-eb652c0169d9/images/f02ac1ce-
>> 52cd-4b81-8b29-f8006d0469e0/ff4e49c6-3084-4234-80a1-
>> 18a67615c527,if=none,id=drive-virtio-disk1,format=raw,
>> serial=f02ac1ce-52cd-4b81-8b29-f8006d0469e0,cache=none,
>> werror=stop,rerror=stop,aio=threads
>> > -device
>> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=dr

Re: [ovirt-users] VM pauses/hangs after migration

2016-09-29 Thread Michal Skrivanek

> On 29 Sep 2016, at 13:59, Davide Ferrari <dav...@billymob.com> wrote:
> 
> Hello
> 
> Today I've the faulty DIMMs replaced, started the same VM again and did the 
> same migration and this time worked, so it was 100% due to that.
> 
> The problem that make me wonder a bit is: if it's the source host with memory 
> problem the one which blocks the correct migration, a faulty DIMM will force 
> you to stop the VMs running on that host, because you cannot simply migrate 
> them away to do the maintenence tasks…

if you have a faulty hw you should do that ASAP as you never know where it is 
going to affect you. It’s like with disk errors…you may think it’s ok when you 
rarely write to certain places, but once you try to copy it off the problematic 
storage and you read every single byte/location you’re screwed…

Thanks,
michal

> 
> 
> 2016-09-29 13:53 GMT+02:00 Tomas Jelinek <tjeli...@redhat.com 
> <mailto:tjeli...@redhat.com>>:
> 
> 
> - Original Message -
> > From: "Davide Ferrari" <dav...@billymob.com <mailto:dav...@billymob.com>>
> > To: "users" <users@ovirt.org <mailto:users@ovirt.org>>
> > Sent: Wednesday, September 28, 2016 2:59:59 PM
> > Subject: [ovirt-users] VM pauses/hangs after migration
> >
> > Hello
> >
> > trying to migrate a VM from one host to another, a big VM with 96GB of RAM, 
> > I
> > found that when the migration completes, the VM goes to a paused satte and
> > cannot be resumed. The libvirt/qemu log it gives is this:
> >
> > 2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section id
> > 2(ram)
> > 2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed: Input/output
> > error
> > 2016-09-28 12:18:15.872+: shutting down
> > 2016-09-28 12:22:21.467+: starting up libvirt version: 1.2.17, package:
> > 13.el7_2.5 (CentOS BuildSystem < http://bugs.centos.org 
> > <http://bugs.centos.org/> >,
> > 2016-06-23-14 :23:27, worker1.bsys.centos.org 
> > <http://worker1.bsys.centos.org/> ), qemu version: 2.3.0
> > (qemu-kvm-ev-2.3.0-31.el7.16.1)
> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name front04.billydomain.com 
> > <http://front04.billydomain.com/> -S
> > -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
> > size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
> > 32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304
> > -uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios
> > type=1,manufacturer=oVirt,product=oVirt
> > Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-004A-3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68
> > -no-user-config -nodefaults -chardev
> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/
> > domain-front04.billydomain.com/monitor.sock,server,nowait 
> > <http://domain-front04.billydomain.com/monitor.sock,server,nowait> -mon
> > chardev=charmonitor,id=monitor,mode=control -rtc
> > base=2016-09-28T14:22:21,driftfix=slew -global
> > kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
> > -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> > virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
> > virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive
> > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
> > file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/b5b49d5c-2378-4639-9469-362e37ae7473/24fd0d3c-309b-458d-9818-4321023afacf,if=none,id=drive-virtio-disk0,format=qcow2,serial=b5b49d5c-2378-4639-9469-362e37ae7473,cache=none,werror=stop,rerror=stop,aio=threads
> > -device
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> > -drive
> > file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/f02ac1ce-52cd-4b81-8b29-f8006d0469e0/ff4e49c6-3084-4234-80a1-18a67615c527,if=none,id=drive-virtio-disk1,format=raw,serial=f02ac1ce-52cd-4b81-8b29-f8006d0469e0,cache=none,werror=stop,rerror=stop,aio=threads
> > -device
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1
> > -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=31 -device
> > virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:56,bus=pci.0,addr=0x3
> > -chardev
> > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.com

Re: [ovirt-users] VM pauses/hangs after migration

2016-09-29 Thread Davide Ferrari
Hello

Today I've the faulty DIMMs replaced, started the same VM again and did the
same migration and this time worked, so it was 100% due to that.

The problem that make me wonder a bit is: if it's the source host with
memory problem the one which blocks the correct migration, a faulty DIMM
will force you to stop the VMs running on that host, because you cannot
simply migrate them away to do the maintenence tasks...


2016-09-29 13:53 GMT+02:00 Tomas Jelinek <tjeli...@redhat.com>:

>
>
> - Original Message -
> > From: "Davide Ferrari" <dav...@billymob.com>
> > To: "users" <users@ovirt.org>
> > Sent: Wednesday, September 28, 2016 2:59:59 PM
> > Subject: [ovirt-users] VM pauses/hangs after migration
> >
> > Hello
> >
> > trying to migrate a VM from one host to another, a big VM with 96GB of
> RAM, I
> > found that when the migration completes, the VM goes to a paused satte
> and
> > cannot be resumed. The libvirt/qemu log it gives is this:
> >
> > 2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section
> id
> > 2(ram)
> > 2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed:
> Input/output
> > error
> > 2016-09-28 12:18:15.872+: shutting down
> > 2016-09-28 12:22:21.467+: starting up libvirt version: 1.2.17,
> package:
> > 13.el7_2.5 (CentOS BuildSystem < http://bugs.centos.org >,
> > 2016-06-23-14:23:27, worker1.bsys.centos.org ), qemu version: 2.3.0
> > (qemu-kvm-ev-2.3.0-31.el7.16.1)
> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
> > QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name front04.billydomain.com
> -S
> > -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
> > size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
> > 32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304
> > -uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios
> > type=1,manufacturer=oVirt,product=oVirt
> > Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-
> 004A-3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68
> > -no-user-config -nodefaults -chardev
> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/
> > domain-front04.billydomain.com/monitor.sock,server,nowait -mon
> > chardev=charmonitor,id=monitor,mode=control -rtc
> > base=2016-09-28T14:22:21,driftfix=slew -global
> > kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
> > -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> > virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
> > virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4
> -drive
> > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
> > file=/rhev/data-center/0001-0001-0001-0001-
> 03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/
> b5b49d5c-2378-4639-9469-362e37ae7473/24fd0d3c-309b-
> 458d-9818-4321023afacf,if=none,id=drive-virtio-disk0,
> format=qcow2,serial=b5b49d5c-2378-4639-9469-362e37ae7473,
> cache=none,werror=stop,rerror=stop,aio=threads
> > -device
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-
> virtio-disk0,id=virtio-disk0,bootindex=1
> > -drive
> > file=/rhev/data-center/0001-0001-0001-0001-
> 03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/
> f02ac1ce-52cd-4b81-8b29-f8006d0469e0/ff4e49c6-3084-
> 4234-80a1-18a67615c527,if=none,id=drive-virtio-disk1,
> format=raw,serial=f02ac1ce-52cd-4b81-8b29-f8006d0469e0,
> cache=none,werror=stop,rerror=stop,aio=threads
> > -device
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-
> virtio-disk1,id=virtio-disk1
> > -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=31 -device
> > virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:
> 16:01:56,bus=pci.0,addr=0x3
> > -chardev
> > socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/
> 4511d1c0-6607-418f-ae75-34f605b2ad68.com.redhat.rhevm.vdsm,server,nowait
> > -device
> > virtserialport,bus=virtio-serial0.0,nr=1,chardev=
> charchannel0,id=channel0,name=com.redhat.rhevm.vdsm
> > -chardev
> > socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/
> 4511d1c0-6607-418f-ae75-34f605b2ad68.org.qemu.guest_agent.0,server,nowait
> > -device
> > virtserialport,bus=virtio-serial0.0,nr=2,chardev=
> charchannel1,id=channel1,name=org.qemu.guest_agent.0
> > -chardev spicevmc,id=charchannel2,name=vdagent -device
> > virtserialport,bus=virtio-serial0.0,nr=3,chardev=
> charchannel2,id=channel2,name=com.redhat.spice.0
> > -vnc 192.168.10.225:1 ,password -

Re: [ovirt-users] VM pauses/hangs after migration

2016-09-29 Thread Tomas Jelinek


- Original Message -
> From: "Davide Ferrari" <dav...@billymob.com>
> To: "users" <users@ovirt.org>
> Sent: Wednesday, September 28, 2016 2:59:59 PM
> Subject: [ovirt-users] VM pauses/hangs after migration
> 
> Hello
> 
> trying to migrate a VM from one host to another, a big VM with 96GB of RAM, I
> found that when the migration completes, the VM goes to a paused satte and
> cannot be resumed. The libvirt/qemu log it gives is this:
> 
> 2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section id
> 2(ram)
> 2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed: Input/output
> error
> 2016-09-28 12:18:15.872+: shutting down
> 2016-09-28 12:22:21.467+: starting up libvirt version: 1.2.17, package:
> 13.el7_2.5 (CentOS BuildSystem < http://bugs.centos.org >,
> 2016-06-23-14:23:27, worker1.bsys.centos.org ), qemu version: 2.3.0
> (qemu-kvm-ev-2.3.0-31.el7.16.1)
> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
> QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name front04.billydomain.com -S
> -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
> size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
> 32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304
> -uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios
> type=1,manufacturer=oVirt,product=oVirt
> Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-004A-3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68
> -no-user-config -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/
> domain-front04.billydomain.com/monitor.sock,server,nowait -mon
> chardev=charmonitor,id=monitor,mode=control -rtc
> base=2016-09-28T14:22:21,driftfix=slew -global
> kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
> -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
> virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive
> if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
> file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/b5b49d5c-2378-4639-9469-362e37ae7473/24fd0d3c-309b-458d-9818-4321023afacf,if=none,id=drive-virtio-disk0,format=qcow2,serial=b5b49d5c-2378-4639-9469-362e37ae7473,cache=none,werror=stop,rerror=stop,aio=threads
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> -drive
> file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/f02ac1ce-52cd-4b81-8b29-f8006d0469e0/ff4e49c6-3084-4234-80a1-18a67615c527,if=none,id=drive-virtio-disk1,format=raw,serial=f02ac1ce-52cd-4b81-8b29-f8006d0469e0,cache=none,werror=stop,rerror=stop,aio=threads
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1
> -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=31 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:56,bus=pci.0,addr=0x3
> -chardev
> socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.com.redhat.rhevm.vdsm,server,nowait
> -device
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm
> -chardev
> socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.org.qemu.guest_agent.0,server,nowait
> -device
> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
> -chardev spicevmc,id=charchannel2,name=vdagent -device
> virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0
> -vnc 192.168.10.225:1 ,password -k es -spice
> tls-port=5902,addr=192.168.10.225,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on
> -k es -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=8388608,vgamem_mb=16,bus=pci.0,addr=0x2
> -incoming tcp: 0.0.0.0:49156 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on
> Domain id=5 is tainted: hook-script
> red_dispatcher_loadvm_commands:
> KVM: entry failed, hardware error 0x8
> RAX=ffed RBX=8817ba00c000 RCX=0100
> RDX=
> RSI= RDI=0046 RBP=8817ba00fe98
> RSP=8817ba00fe98
> R8 = R9 = R10=
> R11=
> R12=0006

[ovirt-users] VM pauses/hangs after migration

2016-09-28 Thread Davide Ferrari
Hello

trying to migrate a VM from one host to another, a big VM with 96GB of RAM,
I found that when the migration completes, the VM goes to a paused satte
and cannot be resumed. The libvirt/qemu log it gives is this:

2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section id
2(ram)
2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed:
Input/output error
2016-09-28 12:18:15.872+: shutting down
2016-09-28 12:22:21.467+: starting up libvirt version: 1.2.17, package:
13.el7_2.5 (CentOS BuildSystem ,
2016-06-23-14:23:27, worker1.bsys.centos.org), qemu version: 2.3.0
(qemu-kvm-ev-2.3.0-31.el7.16.1)
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name front04.billydomain.com -S
-machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m
size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp
32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304
-uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios
type=1,manufacturer=oVirt,product=oVirt
Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-004A-3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68
-no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/
domain-front04.billydomain.com/monitor.sock,server,nowait -mon
chardev=charmonitor,id=monitor,mode=control -rtc
base=2016-09-28T14:22:21,driftfix=slew -global
kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device
virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive
if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive
file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/b5b49d5c-2378-4639-9469-362e37ae7473/24fd0d3c-309b-458d-9818-4321023afacf,if=none,id=drive-virtio-disk0,format=qcow2,serial=b5b49d5c-2378-4639-9469-362e37ae7473,cache=none,werror=stop,rerror=stop,aio=threads
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-drive
file=/rhev/data-center/0001-0001-0001-0001-03e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/f02ac1ce-52cd-4b81-8b29-f8006d0469e0/ff4e49c6-3084-4234-80a1-18a67615c527,if=none,id=drive-virtio-disk1,format=raw,serial=f02ac1ce-52cd-4b81-8b29-f8006d0469e0,cache=none,werror=stop,rerror=stop,aio=threads
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1
-netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=31 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:56,bus=pci.0,addr=0x3
-chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.com.redhat.rhevm.vdsm,server,nowait
-device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm
-chardev
socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.org.qemu.guest_agent.0,server,nowait
-device
virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
-chardev spicevmc,id=charchannel2,name=vdagent -device
virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0
-vnc 192.168.10.225:1,password -k es -spice
tls-port=5902,addr=192.168.10.225,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on
-k es -device
qxl-vga,id=video0,ram_size=67108864,vram_size=8388608,vgamem_mb=16,bus=pci.0,addr=0x2
-incoming tcp:0.0.0.0:49156 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on
Domain id=5 is tainted: hook-script
red_dispatcher_loadvm_commands:
KVM: entry failed, hardware error 0x8
RAX=ffed RBX=8817ba00c000 RCX=0100
RDX=
RSI= RDI=0046 RBP=8817ba00fe98
RSP=8817ba00fe98
R8 = R9 = R10=
R11=
R12=0006 R13=8817ba00c000 R14=8817ba00c000
R15=
RIP=81058e96 RFL=00010286 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =   
CS =0010   00a09b00 DPL=0 CS64 [-RA]
SS =0018   00c09300 DPL=0 DS   [-WA]
DS =   
FS =   
GS = 8817def8  
LDT=   
TR =0040 8817def93b80 2087 8b00 DPL=0 TSS64-busy
GDT= 8817def89000 007f
IDT= ff529000 0fff
CR0=80050033