> There's something ugly going on with the network. . . .
And to close the loop on this issue for anyone else trying P2V migrations.
Turns out, there were two independent problems that interacted to make a big
mess.
Problem #1 - I promise never again to set up a 100mb NFS server with gb NF
There's something ugly going on with the network. Take a look at this
experiment. I have a bunch of unused local disk space on my thing2 host and I
wondered what would happen if I tried copying that large VM disk image from
that RHEV Export NFS server to a local disk by hand? So I tried it - s
> Greg, is this with Matt's latest F16 packages?
No. Not yet. I only wanted to change one thing at a time so right this second
I'm still using Matt's special 0.8.4.1 from a few days ago. But now I think I
need to try again with 0.8.5 (I think that's the version.)
Trying to import that VM int
On Tue, Dec 06, 2011 at 01:02:49AM -0600, Greg Scott wrote:
> > Trying a P2V again, this time using a physical Fedora system for
> > the NFS RHEV Export domain instead of the Storagetek NFS server.
> > Everything else is the same. Same source server, same conversion
> > server, same v2v and p2
> Trying a P2V again, this time using a physical Fedora system for
> the NFS RHEV Export domain instead of the Storagetek NFS server.
> Everything else is the same. Same source server, same conversion
> server, same v2v and p2v versions for now. The only variable
> different is the RHEV Expo
Trying a P2V again, this time using a physical Fedora system for the NFS RHEV
Export domain instead of the Storagetek NFS server. Everything else is the
same. Same source server, same conversion server, same v2v and p2v versions
for now. The only variable different is the RHEV Export NFS serv
Very strange behavior. Fredy killed the P2V at the source server console about
3 hours ago. After roughly 67 hours, the bar graph on the console was at 60
percent. I just checked and noticed virt-p2v-server was still running on the
conversion server. So I did an strace in a different window.
This time after roughly 7 1/2 hours, the exact amount of disk space used in the
NFS Export domain. And then as soon as I run df a couple of times, now the
disk space usage starts to climb again. One other symptom to note - that 3rd
df took nearly 2 minutes to respond. See the date commands be
Our P2V conversion server is a RHEV VM. I wonder if there are BIOS settings on
the hosts about going into standby?
> Well now this is very strange. That P2V we started up Friday afternoon
> crossed into hour 52 a few minutes ago. But in the past 3 1/2 hours, it
> hasn't used one more byte of
Well now this is very strange. That P2V we started up Friday afternoon crossed
into hour 52 a few minutes ago. But in the past 3 1/2 hours, it hasn't used
one more byte of disk space in the NFS Export service. But then after firing
up an strace just now, it started using space again. What's
used so far.
- Greg
-Original Message-
From: libguestfs-boun...@redhat.com [mailto:libguestfs-boun...@redhat.com] On
Behalf Of Greg Scott
Sent: Sunday, December 04, 2011 6:57 AM
To: Matthew Booth; Richard W.M. Jones; libguestfs@redhat.com
Cc: Joey Bertalan
Subject: Re: [Libguestfs] P2Vs seem to r
40+ hour after starting, that P2V is **still** running and the logfile is 0
length. But curiously, the RHEV Export storage it wants has not increased in
about the past 12+ hours. The top display is now showing 99%+ idle. Could
virt-p2v-server be stuck in a loop of some kind?
- Greg
Watching strace -p 23339 -tt -T in another window, where 23339 is the PID of
our virt-v2v-server process - reads just machine gun past. But now it's doing
a bunch of writes and they blip past kind of like individual shotgun blasts.
OK, lousy metaphor but you get the idea. And some of the writ
Trying strace against virt-p2v-server again, this time to stdout -
A bunch of writes, then another hang. I annotated this one a little bit. That
hang went for more than 3 **minutes** from one write to the next. Combined
with a top display showing the overall system is spending about 1/2 its t
Trying strace again didn't work very well - I wonder if it was stuck
waiting for a network I/O to finish?
[root@Fedora16-64P2V log]# ps ax | grep p2v
23339 ?Ds 2:44 /usr/bin/perl /usr/bin/virt-p2v-server
24048 pts/0S+ 0:00 grep --color=auto p2v
[root@Fedora16-64P2V log]#
[root@
I found a presentation from IBM that shows how to do a bunch of
drilldown stuff:
http://linuxvm.org/present/SHARE103/S9303ob.pdf
It mentions a utility named strace that shows system calls. I ran one
for a couple seconds against the running virt-p2v-server - here are the
results:
[root@Fedora16-6
And some more clues. Apparently, our virt-p2v-server is dropping lots
of received packets. I wonder why?
[root@Fedora16-64P2V log]# netstat -i
Kernel Interface table
Iface MTU MetRX-OK RX-ERR RX-DRP RX-OVRTX-OK TX-ERR TX-DRP
TX-OVR Flg
eth0 1500 0 347387803 0 29625
14 hours after starting it up - about 41 GB done. The top utility still
shows about 50-50 between id and wa. Assuming virt-p2v-server is
stalling, I wonder how to find if it's stalling waiting on network I/O
from the source server or waiting on network I/O to the NFS server?
- Greg
___
; libguestfs@redhat.com
Cc: Fredy Hernández; Joey Bertalan
Subject: RE: [Libguestfs] P2Vs seem to require a very robust Ethernet
After 5 or 6 hours, we aborted today's attempt, disconnected the 2nd HDD
(sdb) from the original host, and started up a new one.
Since all we have to go on are guesses -
Here's another nugget of info. Watching top on my conversion server, I
consistently see roughly 50% id and 50% wa. It varies second by second
by a percent or two but it's generally around 50-50. The "wa" state
means waiting for an I/O to complete so it seems like something is
taking a **very** l
ehalf Of Greg Scott
Sent: Friday, December 02, 2011 3:18 PM
To: Matthew Booth; Richard W.M. Jones; libguestfs@redhat.com
Cc: Fredy Hernandez; Joey Bertalan
Subject: Re: [Libguestfs] P2Vs seem to require a very robust Ethernet
Our track record for P2Vs isn't very good so far; we've
Our track record for P2Vs isn't very good so far; we've had one run to
completion in about 8 or 9 tries so far. We've tried source systems
with different versions of Windows, all but one failed. The one success
was a test Windows XP machine with one hard drive.
We boot a Windows system from the
I did one with ntbackup and restore a few months ago and ran into problems
with the virtual cpu not matching the virtual bios. That one was in a
libvirt environment and that VM will retire soon. But with the current
project, these VMs will be around a while.
Sent from my Droid Charge on Verizon 4
On 12/02/2011 03:12 AM, Greg Scott wrote:
Hey Matt -
With a new release of virt-v2v a while away, what if we were to try a
P2V by hand? I'm thinking of maybe provisioning a Windows RHEV VM,
doing an ntbackup of the source physical machine and doing a bare metal
restore on the VM. I've done it
Hey Matt -
With a new release of virt-v2v a while away, what if we were to try a
P2V by hand? I'm thinking of maybe provisioning a Windows RHEV VM,
doing an ntbackup of the source physical machine and doing a bare metal
restore on the VM. I've done it that way before with mixed results.
We ha
OK, thanks.
For what it's worth, we just had another P2V attempt die. This source
system is an old HP Pavillion with Windows Server 2003 installed. The
P2V hung on the source server at 13 percent. Looking at the log file on
the Fedora conversion server:
[root@Fedora16-64P2V log]# more virt-p2
On 11/28/2011 04:51 PM, Greg Scott wrote:
.
.
.
We've been chasing network issues - but maybe the network and
infrastruture is fine and something is going on with the source
server.
How do we check this?
I'm currently working on getting a new release out. I can't say it
will
fix this prob
.
.
.
>> We've been chasing network issues - but maybe the network and
>> infrastruture is fine and something is going on with the source
server.
>> How do we check this?
> I'm currently working on getting a new release out. I can't say it
will
> fix this problem, but the new P2V client might hel
On 11/24/2011 11:45 AM, Greg Scott wrote:
We've had one successful P2V with a test XP workstation and several
failures with a Windows 2003 Server. Both P2Vs use an old Storagetek
NAS for NFS storage for the RHEV Export domain. Both use my bleeding
edge 64 bit Fedora 16 RHEV VM as a migration se
To: libguestfs@redhat.com
Cc: Fredy Hernandez; Joey Bertalan
Subject: Re: [Libguestfs] P2Vs seem to require a very robust Ethernet
Trying that same P2V again, this time with everything connected to a
brand new HP Procurve 2910al-48G. All NICs on my hosts now show up at
1000 Gb now and this P2V sti
Trying that same P2V again, this time with everything connected to a
brand new HP Procurve 2910al-48G. All NICs on my hosts now show up at
1000 Gb now and this P2V still failed, this time after running 5 hours.
The source server is an aging HP Proliant ML110 running Windows 2003 (I
think). The co
Now that we can gather diagnostic info, I think I know why our P2Vs kept
failing last week. Another one just died right in front of my eyes. I
think either the Ethernet or NFS server at this site occasionally
"blips" offline when it gets busy and that messes up P2V migrations.
The RHEV export dom
32 matches
Mail list logo
