Re: 11-RELEASE and live migration
> I'm able to see these glitches, as said the VM doesn't really hang, it just > gets > stuck for a long time (and I think that depends on the uptime differences > between the source and destination hosts). In the meantime, could you please > test if changing the timecounter from XENTIMER to any other (like HPET or > ACPI-fast) solves the issue? > > # sysctl -w kern.timecounter.hardware=ACPI-fast Hello Roger, I changed the timecounter from XENTIMER to ACPI-fast and then I did about 10 migrations between our 3 hosts. I always started the next migration a few seconds after the previous migration finished. The VM didn't stuck now and I didn't see the VGABios screen on the console. Regards, Christian ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: 11-RELEASE and live migration
On Mon, Dec 12, 2016 at 08:33:48AM -0600, Jay West wrote: > My setup... > > Hosts: Two Dell R310's, each one as follows: 32gb ram, L3480 cpu, 4 gigabit > nics, two 300gb disks (mirrored), where the local disk is used only to hold > xenserver 7.0 with all patches up through and including today. These > machines are 100% up to date on firmware, patches, bios, etc. > > Network: Two Dell powerconnect 6224's, stacked via dual round-robin CX4 > cables. There's an untagged management vlan, an untagged data vlan, and an > untagged ISCSI vlan - identical ports are members on each switch. On each > host, the two builtin nics (data) are connected one leg in each switch (same > vlan). The two addin ports (Intel Pro/1000) (iSCSI) on each host are also > connected one leg in each switch. The switches are 100% up to date on > firmware/OS. Client access to this is via a stack of Juniper EX2200's > trunked back to the 6224's. > > Storage: OEM version of a Tyan 2U 12 bay SAS box, similar to Tyan > S8226WGM3NR but with 7 gigabit NICS builtin. 32gb ram, FreeNAS 9.10.1-U4, > and dual AMD C32's (12 cores total). One nic is used for ILO, another for > management, another is unused , and the remaining four gigabit ports are > connected two legs in each switch (same vlan, iscsi only, 9000mtu). > Multipath is configured and active. > > The storage box is using mirrored vdev's with ZFS on top, 100% of which > present an iSCSI target so the box is doing nothing but iscsi (and an NFS > iso share for installing vm's). So in Xencenter... there is one storage > repository containing all the NAS space. Xencenter then creates the vdisks > inside that for each VM to use. > > FreeBSD 10X doesn't seem to have this problem. FreeBSD11 definitely does, > and apparently I'm not the only one who can see it. I should also point out > that Windows VM's (both Server 2012 R2 and 7 pro - both 64bit) have no > problem migrating to another host and then back. And FreeBSD can definitely > migrate to another host - just not then back to the first (at least... > immediately. I haven't tried waiting an hour or so and then trying the > migration back). > > I also cannot select the source host as the destination in Xencenter. The > host servers are completely identical in every respect. All vm's disk is via > iSCSI as above. > > I also have a completely separate architecture that is identical to the > above, except much larger, using xenserver 6.5, HP DL1000's, and cisco 3750G > stacks. I have not yet tested freebsd11 on that installation; I assumed it > wouldn't be much help as it's older versions of all the code. > > The smaller architecture above is not yet in production, so I can do testing > on it. The larger installation mentioned later above is production, and I > can't do much major testing there. Thanks for such accurate description. I've now setup a similar environment and I'm able to see these glitches, as said the VM doesn't really hang, it just gets stuck for a long time (and I think that depends on the uptime differences between the source and destination hosts). In the meantime, could you please test if changing the timecounter from XENTIMER to any other (like HPET or ACPI-fast) solves the issue? # sysctl -w kern.timecounter.hardware=ACPI-fast Thanks, Roger. ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
RE: 11-RELEASE and live migration
My setup...
Hosts: Two Dell R310's, each one as follows: 32gb ram, L3480 cpu, 4 gigabit
nics, two 300gb disks (mirrored), where the local disk is used only to hold
xenserver 7.0 with all patches up through and including today. These
machines are 100% up to date on firmware, patches, bios, etc.
Network: Two Dell powerconnect 6224's, stacked via dual round-robin CX4
cables. There's an untagged management vlan, an untagged data vlan, and an
untagged ISCSI vlan - identical ports are members on each switch. On each
host, the two builtin nics (data) are connected one leg in each switch (same
vlan). The two addin ports (Intel Pro/1000) (iSCSI) on each host are also
connected one leg in each switch. The switches are 100% up to date on
firmware/OS. Client access to this is via a stack of Juniper EX2200's
trunked back to the 6224's.
Storage: OEM version of a Tyan 2U 12 bay SAS box, similar to Tyan
S8226WGM3NR but with 7 gigabit NICS builtin. 32gb ram, FreeNAS 9.10.1-U4,
and dual AMD C32's (12 cores total). One nic is used for ILO, another for
management, another is unused , and the remaining four gigabit ports are
connected two legs in each switch (same vlan, iscsi only, 9000mtu).
Multipath is configured and active.
The storage box is using mirrored vdev's with ZFS on top, 100% of which
present an iSCSI target so the box is doing nothing but iscsi (and an NFS
iso share for installing vm's). So in Xencenter... there is one storage
repository containing all the NAS space. Xencenter then creates the vdisks
inside that for each VM to use.
FreeBSD 10X doesn't seem to have this problem. FreeBSD11 definitely does,
and apparently I'm not the only one who can see it. I should also point out
that Windows VM's (both Server 2012 R2 and 7 pro - both 64bit) have no
problem migrating to another host and then back. And FreeBSD can definitely
migrate to another host - just not then back to the first (at least...
immediately. I haven't tried waiting an hour or so and then trying the
migration back).
I also cannot select the source host as the destination in Xencenter. The
host servers are completely identical in every respect. All vm's disk is via
iSCSI as above.
I also have a completely separate architecture that is identical to the
above, except much larger, using xenserver 6.5, HP DL1000's, and cisco 3750G
stacks. I have not yet tested freebsd11 on that installation; I assumed it
wouldn't be much help as it's older versions of all the code.
The smaller architecture above is not yet in production, so I can do testing
on it. The larger installation mentioned later above is production, and I
can't do much major testing there.
Hopefully some of this info helps!
J
-Original Message-
From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org]
On Behalf Of Hoyer-Reuther, Christian
Sent: Monday, December 12, 2016 4:40 AM
To: Roger Pau Monné <roger@citrix.com>; h...@dhcp-3-221.uk.xensource.com
Cc: freebsd-xen@freebsd.org; owner-freebsd-...@freebsd.org
Subject: Re: 11-RELEASE and live migration
Hello,
I cannot select the source host as destination when I migrate from
XenCenter.
All 3 hosts use the same hardware. Disk backend for all VMs is iSCSI.
Regards,
Christian
> -Ursprüngliche Nachricht-
> Von: Roger Pau Monné [mailto:roger@citrix.com]
> Gesendet: Montag, 12. Dezember 2016 11:32
> An: Hoyer-Reuther, Christian; h...@dhcp-3-221.uk.xensource.com
> Cc: freebsd-xen@freebsd.org; owner-freebsd-...@freebsd.org
> Betreff: Re: 11-RELEASE and live migration
>
> On Thu, Dec 08, 2016 at 10:33:37AM +0100, Hoyer-Reuther, Christian wrote:
> > I did some tests and I see the problem too.
> >
> > XenServer 6.5 SP1 with almost all patches (3 hosts in pool), FreeBSD
> > 11.0-
> RELEASE-p2, xe-guest-utilities-6.2.0_2 installed via pkg.
> >
> > First migration from host 3 to host 1 is ok.
> >
> > Some seconds later I start a new migration from host 1 to host 2 and
> > when
> migration finishes (as seen in XenCenter) then the VM switches to the
> VGABios screen ("Plex86/Bochs VGABios (PCI) current-cvs 01 Sep 2016 ...
> cirrus-compatible VGA is detected"). The VM seem to hang and does not
> respond. In XenCenter I see that all the CPU's of the VM go up to 100
> percent.
> >
> > Then after 17 minutes the VGABios screen disappears and I see the
> > console,
> the CPU usage as seen in XenCenter goes down. I logged in as root
> before I started the first migration and root is still logged in. So
> it was a hang and not a reboot.
> >
> > 20 minutes later I start a new migration from host 2 to host 3 and
> > the
> problem occurs again.
> >
> > This problem does not exist with 10.3-RELEASE on the same hosts.
>
> Hello,
>
> So far I've only tested local migration (migration using the same host
> as sour
Re: 11-RELEASE and live migration
Hello,
I cannot select the source host as destination when I migrate from XenCenter.
All 3 hosts use the same hardware. Disk backend for all VMs is iSCSI.
Regards,
Christian
> -Ursprüngliche Nachricht-
> Von: Roger Pau Monné [mailto:roger@citrix.com]
> Gesendet: Montag, 12. Dezember 2016 11:32
> An: Hoyer-Reuther, Christian; h...@dhcp-3-221.uk.xensource.com
> Cc: freebsd-xen@freebsd.org; owner-freebsd-...@freebsd.org
> Betreff: Re: 11-RELEASE and live migration
>
> On Thu, Dec 08, 2016 at 10:33:37AM +0100, Hoyer-Reuther, Christian wrote:
> > I did some tests and I see the problem too.
> >
> > XenServer 6.5 SP1 with almost all patches (3 hosts in pool), FreeBSD 11.0-
> RELEASE-p2, xe-guest-utilities-6.2.0_2 installed via pkg.
> >
> > First migration from host 3 to host 1 is ok.
> >
> > Some seconds later I start a new migration from host 1 to host 2 and when
> migration finishes (as seen in XenCenter) then the VM switches to the
> VGABios screen ("Plex86/Bochs VGABios (PCI) current-cvs 01 Sep 2016 ...
> cirrus-compatible VGA is detected"). The VM seem to hang and does not
> respond. In XenCenter I see that all the CPU's of the VM go up to 100
> percent.
> >
> > Then after 17 minutes the VGABios screen disappears and I see the console,
> the CPU usage as seen in XenCenter goes down. I logged in as root before I
> started the first migration and root is still logged in. So it was a hang
> and not a reboot.
> >
> > 20 minutes later I start a new migration from host 2 to host 3 and the
> problem occurs again.
> >
> > This problem does not exist with 10.3-RELEASE on the same hosts.
>
> Hello,
>
> So far I've only tested local migration (migration using the same host as
> source
> and destination), and that seems to work fine (I've done +50
> consecutive migrations with only 10s separation between them). Is there a
> change
> that you could also try to reproduce this with local migration?
>
> Are you using the same exact hardware on the source and the destination? Are
> you
> all using iSCSI as the disk backend for this VMs?
>
> Thanks, Roger.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: 11-RELEASE and live migration
On Thu, Dec 08, 2016 at 10:33:37AM +0100, Hoyer-Reuther, Christian wrote:
> I did some tests and I see the problem too.
>
> XenServer 6.5 SP1 with almost all patches (3 hosts in pool), FreeBSD
> 11.0-RELEASE-p2, xe-guest-utilities-6.2.0_2 installed via pkg.
>
> First migration from host 3 to host 1 is ok.
>
> Some seconds later I start a new migration from host 1 to host 2 and when
> migration finishes (as seen in XenCenter) then the VM switches to the VGABios
> screen ("Plex86/Bochs VGABios (PCI) current-cvs 01 Sep 2016 ...
> cirrus-compatible VGA is detected"). The VM seem to hang and does not
> respond. In XenCenter I see that all the CPU's of the VM go up to 100 percent.
>
> Then after 17 minutes the VGABios screen disappears and I see the console,
> the CPU usage as seen in XenCenter goes down. I logged in as root before I
> started the first migration and root is still logged in. So it was a hang and
> not a reboot.
>
> 20 minutes later I start a new migration from host 2 to host 3 and the
> problem occurs again.
>
> This problem does not exist with 10.3-RELEASE on the same hosts.
Hello,
So far I've only tested local migration (migration using the same host as
source
and destination), and that seems to work fine (I've done +50
consecutive migrations with only 10s separation between them). Is there a
change
that you could also try to reproduce this with local migration?
Are you using the same exact hardware on the source and the destination? Are
you
all using iSCSI as the disk backend for this VMs?
Thanks, Roger.
___
freebsd-xen@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
RE: 11-RELEASE and live migration
Rainer wrote... -- We've also seen this (unexplainable reboots, I think they happen after migrations). But it never hangs. It seems to actually "reboot", as if somebody had pressed CTRL-ALT-DEL or typed "reboot". - I should also point out... on the same pair of hosts, there are a few windows server 2012 r2 and windows 7 pro VM's. None of them have problems migrating back and forth. It's only the F11 ones that do. First migration goes fine, and vm can be logged in to and excercised. But then migrate it back... and it goes to the fake bios post screen (I forget... cirrus bios or something).. and then just hangs without loading OS so there is no message from the OS. Strange... I have older versions of freebsd on an older version of xenserver... and the above definitely is not a problem there. Perhaps it's something amiss with my new setup, but the windows vm's migrate fine J ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: 11-RELEASE and live migration
Am 2016-12-07 17:21, schrieb Jay West: Not sure if anyone else has seen this Fresh install of xenserver 7 on two dell PE's, local storage only for xenserver 7 installation on each. All VM storage via iscsi to a nas (multipath, split between two stacked dell powerconnect switches that are properly configured for iscsi traffic). Freebsd11-Release iso used to spin up a few freebsd VM's, and xe-guest-utilities-6.2.0_2 added via 'pkg install'. Live migration from one host to the other works fine (doesn't matter which of the two hosts is the source and which is the destination). But after it is done migrating if you then do a live migration back to the server where it was a few minutes ago on the recipient host the VM reboots upon migration. It gets to the bios screen and then just hangs. So... 1) Why does it reboot, 2) Why does it then get stuck just after bios post but before OS load, and 3) Has anyone else seen this? We've also seen this (unexplainable reboots, I think they happen after migrations). But it never hangs. It seems to actually "reboot", as if somebody had pressed CTRL-ALT-DEL or typed "reboot". XenServer 6.5SP2 ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
