Re: Timeout with live migration

2015-10-13 Thread Rafael Weingärtner
t; for helping nail this down! > > > From: Rafael Weingärtner [rafaelweingart...@gmail.com] > Sent: Tuesday, October 13, 2015 6:12 PM > To: users@cloudstack.apache.org > Subject: [Questionable] Re: Timeout with live migration > > Nice,

Re: Timeout with live migration

2015-10-13 Thread Ryan Farrington
] Re: Timeout with live migration Nice, thanks. Did that solve your problem? Did you migrate the volume? On Tue, Oct 13, 2015 at 7:00 PM, Ryan Farrington wrote: > Issue #1) Terrible parameter names > https://issues.apache.org/jira/browse/CLOUDSTACK-8946 > > Issue #2) Wait value for M

Re: Timeout with live migration

2015-10-13 Thread Rafael Weingärtner
apache.org/jira/browse/CLOUDSTACK-8949 > > > > > From: Rafael Weingärtner [rafaelweingart...@gmail.com] > Sent: Tuesday, October 13, 2015 3:43 PM > To: users@cloudstack.apache.org > Subject: [Questionable] Re: Timeout with live migra

Re: Timeout with live migration

2015-10-13 Thread Ryan Farrington
, October 13, 2015 3:43 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration That is good. Now I can report to you what is in the code. Let’s start: First: when I looked at the problem at first time, I went straight to the class that sends commands to Xen, and

Re: Timeout with live migration

2015-10-13 Thread Rafael Weingärtner
ccd-4648-8928-35ca90924d7c","result":true,"wait":0}}] > } > 2015-10-13 15:27:13,397 DEBUG [c.c.a.m.AgentAttache] > (DirectAgent-430:ctx-ac6d7aeb) Seq 38-1788936343: No more commands found > 2015-10-13 15:27:13,397 DEBUG [c.c.a.t.Request] > (Job-Executor-1:c

Re: Timeout with live migration

2015-10-13 Thread Ryan Farrington
936343: No more commands found 2015-10-13 15:27:13,397 DEBUG [c.c.a.t.Request] (Job-Executor-1:ctx-8e0ebced ctx-f49e7503) Seq 38-1788936343: Received: { Ans: , MgmtId: 42756806312036, via: 38, Ver: v1, Flags: 110, { MigrateVolumeAnswer } } From: Rafael Weingärtner [

Re: Timeout with live migration

2015-10-13 Thread Rafael Weingärtner
> From: Rafael Weingärtner [rafaelweingart...@gmail.com] > Sent: Tuesday, October 13, 2015 7:52 AM > To: users@cloudstack.apache.org > Subject: [Questionable] Re: [Questionable] Re: Timeout with live migration > > I guess so, for some reason that I do not understand,

Re: Timeout with live migration

2015-10-13 Thread Ryan Farrington
ils once I have it created? From: Rafael Weingärtner [rafaelweingart...@gmail.com] Sent: Tuesday, October 13, 2015 7:52 AM To: users@cloudstack.apache.org Subject: [Questionable] Re: [Questionable] Re: Timeout with live migration I guess so, for some r

Re: [Questionable] Re: Timeout with live migration

2015-10-13 Thread Rafael Weingärtner
M > To: users@cloudstack.apache.org > Subject: [Questionable] Re: Timeout with live migration > > There is your problem, there are currently two distinct values conrolling > those async jobs. > Change that value and everything will work for u. > Can you open a jira ticket? > > On Mon,

Re: [Questionable] Re: Timeout with live migration

2015-10-13 Thread Jakub Kublik
[rafaelweingart...@gmail.com] Sent: Monday, October 12, 2015 10:12 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration There is your problem, there are currently two distinct values conrolling those async jobs. Change that value and everything will work for u. Can

RE: [Questionable] Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
day, October 12, 2015 10:12 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration There is your problem, there are currently two distinct values conrolling those async jobs. Change that value and everything will work for u. Can you open a jira ticket? On Mon, Oct

Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
600 > > > > > From: Rafael Weingärtner [rafaelweingart...@gmail.com] > Sent: Monday, October 12, 2015 9:46 PM > To: users@cloudstack.apache.org > Subject: [Questionable] Re: Timeout with live migration > > I found something odd, > can you check the pa

Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
wait is currently configured to be 3600 From: Rafael Weingärtner [rafaelweingart...@gmail.com] Sent: Monday, October 12, 2015 9:46 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration I found something odd, can you

Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
tore.PrimaryDataStoreImpl@4fa7a45f), > > no need to delete from object in store ref table > > 2015-10-12 18:41:20,479 DEBUG [c.c.s.VolumeApiServiceImpl] > > (Job-Executor-63:ctx-f7b6817d ctx-c6b92515) migrate volume > > failed:com.cloud.utils.exception.CloudRuntimeException: Failed to send > > command, due to Agent:38, com.cloud.exception.OperationTimedo

Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
elweingart...@gmail.com] Sent: Monday, October 12, 2015 8:46 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration I thought you using the command “migrateVirtualMachineWithVolume” but it seems that you are using “migrateVolume” command from ACS's API. For the

Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
uot;errorcode":530,"errortext":"Failed > to migrate volume"} > 2015-10-12 18:41:20,486 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Job-Executor-63:ctx-f7b6817d) Done executing > org.apache.cloudstack.api.command.user.volume.MigrateVolumeCmd for job-5257 > 2015-10-12

Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
.f.j.i.AsyncJobMonitor] (Job-Executor-63:ctx-f7b6817d) Remove job-5257 from job monitoring ________ From: Rafael Weingärtner [rafaelweingart...@gmail.com] Sent: Monday, October 12, 2015 8:24 PM To: users@cloudstack.apache.org Subject: [Questionable] Re:

Re: [Questionable] Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
familiar with ESX this is a "Storage VMotion" where as in > XenServer it is called "Storage XenMotion". > > > From: Rafael Weingärtner [rafaelweingart...@gmail.com] > Sent: Monday, October 12, 2015 7:53 PM > To: users@cloudstack.apa

RE: [Questionable] Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
il.com] Sent: Monday, October 12, 2015 7:53 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration what do you mean with livre migrating data volume ?! I understand a live migration of a VM, but volumes... do you mean live migrating a VM that has a volume attached? ar

Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
October 12, 2015 7:37 PM > To: users@cloudstack.apache.org > Subject: [Questionable] Re: Timeout with live migration > > Are you live migrating a VM, or migrating a volume of a stopped VM to a > different primary storage? > > If it is a running VM, is the VM allocated in a sh

Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
: Timeout with live migration Are you live migrating a VM, or migrating a volume of a stopped VM to a different primary storage? If it is a running VM, is the VM allocated in a shared storage or local storage? On Mon, Oct 12, 2015 at 9:17 PM, Ryan Farrington wrote: > The slow transfer is related

Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
day, October 12, 2015 6:40 PM > To: users@cloudstack.apache.org > Subject: [Questionable] Re: Timeout with live migration > > I would first check your NICs' speed and load, the amount of RAM allocated > for the migrating VM and than check the hypervisor log files. > > On Mon, Oct

Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
onday, October 12, 2015 6:40 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration I would first check your NICs' speed and load, the amount of RAM allocated for the migrating VM and than check the hypervisor log files. On Mon, Oct 12, 2015 at 8:19 PM

Re: Timeout with live migration

2015-10-12 Thread Ryan Farrington
) job.cancel.threshold.minutes = 60 (1 hour) job.expire.minutes = 1440 (24 hours) From: Jan-Arve Nygård [jan.arve.nyg...@gmail.com] Sent: Monday, October 12, 2015 6:19 PM To: users@cloudstack.apache.org Subject: [Questionable] Re: Timeout with live migration

Re: Timeout with live migration

2015-10-12 Thread Rafael Weingärtner
I would first check your NICs' speed and load, the amount of RAM allocated for the migrating VM and than check the hypervisor log files. On Mon, Oct 12, 2015 at 8:19 PM, Jan-Arve Nygård wrote: > What version are you running? Check if the copy.volume.wait setting is set > to 7200 and increase it.

Re: Timeout with live migration

2015-10-12 Thread Jan-Arve Nygård
What version are you running? Check if the copy.volume.wait setting is set to 7200 and increase it. If not you could also check job.cancel.threshold.minutes and job.expire.minutes. -Jan-Arve 2015-10-13 0:46 GMT+02:00 Ryan Farrington : > We are experiencing a failure in cloudstack waiting for an

Timeout with live migration

2015-10-12 Thread Ryan Farrington
We are experiencing a failure in cloudstack waiting for an async job performing a live migration of a volume to finish. I've copied the relevant log entries below.We acknowledge that the migration will take a few hours based on the volume of the data and we are looking for a way to increase the