Hi Anshul, Thanks! Did you ever try live migrating from CloudStack with storage between clusters? I can't get that to work. I'll collect some details in the next days but was wondering if it was supposed to work.
Regards, Remi Sent from my iPhone > On 24 Jun 2016, at 06:30, Anshul Gangwar <anshul.gang...@accelerite.com> > wrote: > > Created the PR https://github.com/apache/cloudstack/pull/1596. > > Regards, > Anshul > > On 24/06/16, 2:33 AM, "Dave Garbus" <davegar...@spiceworks.com> wrote: > > Is there anybody that wants to take this on? > > Thanks, > > - Dave > >> On May 24, 2016, at 10:57 AM, Dave Garbus <davegar...@spiceworks.com> wrote: >> >> I updated the issue referenced in this email >> (https://issues.apache.org/jira/browse/CLOUDSTACK-9353) to include >> CloudStack 4.8 as an affected version. If we could at least get someone >> assigned to it, that’d be a good start. >> >> Thanks, >> >> - Dave >> >> On May 24, 2016, at 9:42 AM, Ivan Derbenev >> <ivan.derbe...@tech-corps.com<mailto:ivan.derbe...@tech-corps.com>> wrote: >> >> Is there a way I can fix this on my installation, without recompiling jar >> files? >> >> Regards, >> IT engineer >> Farheap, Russia >> Ivan Derbenev >> >> -----Original Message----- >> From: Koushik Das [mailto:koushik....@accelerite.com] >> Sent: Tuesday, May 24, 2016 2:06 PM >> To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >> Subject: Re: Xenserver 6.5 live migration with local volumes >> >> This is a product bug and needs to be fixed. There is some serialization >> issue that needs fixing. >> >> In the below code volumeToFiler is null as Map is not a serializable type. >> It needs to be replaced with getVolumeToFilerAsList() and then the data >> needs to be read appropriately. >> >> final Map<VolumeTO, StorageFilerTO> volumeToFiler = >> command.getVolumeToFiler(); >> >> -Koushik >> >> On 24/05/16, 3:23 PM, "Ivan Derbenev" >> <ivan.derbe...@tech-corps.com<mailto:ivan.derbe...@tech-corps.com>> wrote: >> >> Hello! >> After the migration onto XS 6.5 (and installing all the updates) and CS >> 4.8 I can't migrate VMs with Local Storage >> >> The issue is similar to this: >> https://issues.apache.org/jira/browse/CLOUDSTACK-9353, but it doesn't have >> any solutions Any ideas? >> >> Errors: >> >> 2016-05-24 12:10:16,440 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] >> (DirectAgentCronJob-26:ctx-ab4250ff) (logid:4fa92c13) Done with process >> of VM state report. host: 136592 >> 2016-05-24 12:10:16,450 DEBUG [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Looking for network >> named cloud-main >> 2016-05-24 12:10:16,466 DEBUG [c.c.h.x.r.XsLocalNetwork] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Found a network called >> cloud-main on host=10.7.130.103; >> Network=9632020b-4d98-1bed-9ae8-477e0993cab0; >> pif=3c7d1121-2bde-26cf-9478-18e51ffca75f >> 2016-05-24 12:10:16,477 DEBUG [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Creating VLAN 1007 on >> host 10.7.130.103 on device eth0 >> 2016-05-24 12:10:17,163 DEBUG [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) VLAN is created for >> 1007. The uuid is ec053fee-0426-9665-150a-77399562196b >> 2016-05-24 12:10:17,170 WARN >> [c.c.h.x.r.w.x.XenServer610MigrateWithStorageCommandWrapper] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Catch Exception >> java.lang.NullPointerException. Storage motion failed due to >> java.lang.NullPointerException java.lang.NullPointerException >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:86) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:54) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122) >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677) >> at >> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> 2016-05-24 12:10:17,171 DEBUG [c.c.a.m.DirectAgentAttache] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Seq >> 136589-479633360314957887: Response Received: >> 2016-05-24 12:10:17,171 DEBUG [c.c.a.t.Request] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Seq >> 136589-479633360314957887: Processing: { Ans: , MgmtId: >> 128437433941265, via: >> 136589(cloudstack-computing03-sar01.tech-corps.com<http://cloudstack-computing03-sar01.tech-corps.com>), >> Ver: v1, Flags: >> 110, >> [{"com.cloud.agent.api.MigrateWithStorageAnswer":{"result":false,"detai >> ls":"Exception: java.lang.NullPointerException\nMessage: null\nStack: >> java.lang.NullPointerException\n\tat >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610Migr >> ateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageComm >> andWrapper.java:86)\n\tat >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610Migr >> ateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageComm >> andWrapper.java:54)\n\tat >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWr >> apper.execute(CitrixRequestWrapper.java:122)\n\tat >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeReque >> st(CitrixResourceBase.java:1677)\n\tat >> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgen >> tAttache.java:315)\n\tat >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Mana >> gedContextRunnable.java:49)\n\tat >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call >> (DefaultManagedContext.java:56)\n\tat >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi >> thContext(DefaultManagedContext.java:103)\n\tat >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit >> hContext(DefaultManagedContext.java:53)\n\tat >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manage >> dContextRunnable.java:46)\n\tat >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> \n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac >> cess$201(ScheduledThreadPoolExecutor.java:178)\n\tat >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru<http://ScheduledFutureTask.ru> >> n(ScheduledThreadPoolExecutor.java:292)\n\tat >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja >> va:1145)\n\tat >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j >> ava:615)\n\tat java.lang.Thread.run(Thread.java:745)\n","wait":0}}] } >> 2016-05-24 12:10:17,171 DEBUG [c.c.a.m.AgentAttache] >> (DirectAgent-132:ctx-a97d60fa) (logid:8748bffe) Seq >> 136589-479633360314957887: No more commands found >> 2016-05-24 12:10:17,171 DEBUG [c.c.a.t.Request] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Seq 136589-479633360314957887: Received: { Ans: , >> MgmtId: 128437433941265, via: >> 136589(cloudstack-computing03-sar01.tech-corps.com<http://cloudstack-computing03-sar01.tech-corps.com>), >> Ver: v1, Flags: >> 110, { MigrateWithStorageAnswer } } >> 2016-05-24 12:10:17,171 ERROR >> [o.a.c.s.m.XenServerStorageMotionStrategy] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Migration with storage of vm VM[User|i-3-271-VM] >> failed. Details: Exception: java.lang.NullPointerException >> Message: null >> Stack: java.lang.NullPointerException >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:86) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:54) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122) >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677) >> at >> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> 2016-05-24 12:10:17,171 ERROR >> [o.a.c.s.m.XenServerStorageMotionStrategy] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) copy failed >> com.cloud.utils.exception.CloudRuntimeException: Error while migrating >> the vm VM[User|i-3-271-VM] to host Host[-136589-Routing]. Exception: >> java.lang.NullPointerException >> Message: null >> Stack: java.lang.NullPointerException >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:86) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:54) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122) >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677) >> at >> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> at >> org.apache.cloudstack.storage.motion.XenServerStorageMotionStrategy.migrateVmWithVolumesWithinCluster(XenServerStorageMotionStrategy.java:212) >> at >> org.apache.cloudstack.storage.motion.XenServerStorageMotionStrategy.copyAsync(XenServerStorageMotionStrategy.java:116) >> at >> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:90) >> at >> org.apache.cloudstack.storage.volume.VolumeServiceImpl.migrateVolumes(VolumeServiceImpl.java:1160) >> at >> org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1002) >> at >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:2274) >> at >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:4638) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) >> at >> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4741) >> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) >> at >> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> 2016-05-24 12:10:17,183 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Unsupported data object (VOLUME, >> org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@142a92a2), >> no need to delete from object in store ref table >> 2016-05-24 12:10:17,183 DEBUG [o.a.c.e.o.VolumeOrchestrator] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Failed to migrated vm VM[User|i-3-271-VM] along with >> its volumes. com.cloud.utils.exception.CloudRuntimeException: Error >> while migrating the vm VM[User|i-3-271-VM] to host >> Host[-136589-Routing]. Exception: java.lang.NullPointerException >> Message: null >> Stack: java.lang.NullPointerException >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:86) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xen610.XenServer610MigrateWithStorageCommandWrapper.execute(XenServer610MigrateWithStorageCommandWrapper.java:54) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122) >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677) >> at >> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> 2016-05-24 12:10:17,183 INFO [c.c.v.VirtualMachineManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Migration was unsuccessful. Cleaning up: >> VM[User|i-3-271-VM] >> 2016-05-24 12:10:17,184 WARN [o.a.c.alerts] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) alertType:: 17 // dataCenterId:: 1 // podId:: 1 // >> clusterId:: null // message:: Unable to migrate vm i-3-271-VM from host >> cloudstack-computing01-sar01.tech-corps.com<http://cloudstack-computing01-sar01.tech-corps.com> >> in zone zone1.sar and pod >> zone1.sar >> 2016-05-24 12:10:17,185 DEBUG [c.c.a.AlertManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Have already sent: 1 emails for alert type '17' -- >> skipping send email >> 2016-05-24 12:10:17,185 DEBUG [c.c.a.m.AgentAttache] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Seq 136589-479633360314957888: Routed from >> 128437433941265 >> 2016-05-24 12:10:17,185 DEBUG [c.c.a.t.Request] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Seq 136589-479633360314957888: Sending { Cmd , >> MgmtId: 128437433941265, via: >> 136589(cloudstack-computing03-sar01.tech-corps.com<http://cloudstack-computing03-sar01.tech-corps.com>), >> Ver: v1, Flags: >> 100011, >> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence >> ":false,"checkBeforeCleanup":false,"vmName":"i-3-271-VM","wait":0}}] } >> 2016-05-24 12:10:17,185 DEBUG [c.c.a.t.Request] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Seq 136589-479633360314957888: Executing: { Cmd , >> MgmtId: 128437433941265, via: >> 136589(cloudstack-computing03-sar01.tech-corps.com<http://cloudstack-computing03-sar01.tech-corps.com>), >> Ver: v1, Flags: >> 100011, >> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence >> ":false,"checkBeforeCleanup":false,"vmName":"i-3-271-VM","wait":0}}] } >> 2016-05-24 12:10:17,185 DEBUG [c.c.a.m.DirectAgentAttache] >> (DirectAgent-34:ctx-65ef585a) (logid:924b5b07) Seq >> 136589-479633360314957888: Executing request >> 2016-05-24 12:10:17,197 DEBUG [c.c.c.CapacityManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) VM state transitted from :Migrating to Running with >> event: OperationFailedvm's original host id: 136592 new host id: 136592 >> host id before state transition: 136589 >> 2016-05-24 12:10:17,200 DEBUG [c.c.c.CapacityManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Hosts's actual total CPU: 15960 and CPU after applying >> overprovisioning: 39900 >> 2016-05-24 12:10:17,200 DEBUG [c.c.c.CapacityManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Hosts's actual total RAM: 11549697280 and RAM after >> applying overprovisioning: 11549697024 >> 2016-05-24 12:10:17,200 DEBUG [c.c.c.CapacityManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) release cpu from host: 136589, old used: >> 3990,reserved: 0, actual total: 15960, total with overprovisioning: >> 39900; new used: 1995,reserved:0; movedfromreserved: >> false,moveToReserveredfalse >> 2016-05-24 12:10:17,200 DEBUG [c.c.c.CapacityManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) release mem from host: 136589, old used: >> 5368709120,reserved: 0, total: 11549697024; new used: >> 4294967296,reserved:0; movedfromreserved: false,moveToReserveredfalse >> 2016-05-24 12:10:17,201 DEBUG [c.c.a.m.DirectAgentAttache] >> (DirectAgent-34:ctx-65ef585a) (logid:8748bffe) Seq >> 136589-479633360314957888: Response Received: >> 2016-05-24 12:10:17,202 DEBUG [c.c.a.t.Request] >> (DirectAgent-34:ctx-65ef585a) (logid:8748bffe) Seq >> 136589-479633360314957888: Processing: { Ans: , MgmtId: >> 128437433941265, via: >> 136589(cloudstack-computing03-sar01.tech-corps.com<http://cloudstack-computing03-sar01.tech-corps.com>), >> Ver: v1, Flags: >> 10, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM >> does not exist","wait":0}}] } >> 2016-05-24 12:10:17,202 DEBUG [c.c.a.m.AgentAttache] >> (DirectAgent-34:ctx-65ef585a) (logid:8748bffe) Seq >> 136589-479633360314957888: Unable to find listener. >> 2016-05-24 12:10:17,218 ERROR [c.c.v.VmWorkJobHandlerProxy] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Invocation exception, caused by: >> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm >> VM[User|i-3-271-VM] along with its volumes. >> 2016-05-24 12:10:17,218 INFO [c.c.v.VmWorkJobHandlerProxy] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875 ctx-619c9d4e) >> (logid:8748bffe) Rethrow exception >> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm >> VM[User|i-3-271-VM] along with its volumes. >> 2016-05-24 12:10:17,218 DEBUG [c.c.v.VmWorkJobDispatcher] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Done with run of VM work job: com.cloud.vm.VmWorkMigrateWithStorage for >> VM 271, job origin: 4874 >> 2016-05-24 12:10:17,218 ERROR [c.c.v.VmWorkJobDispatcher] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Unable to complete AsyncJobVO {id:4875, userId: 17, accountId: 3, >> instanceType: null, instanceId: null, cmd: >> com.cloud.vm.VmWorkMigrateWithStorage, cmdInfo: >> rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZVdpdGhTdG9yYWdlsew9z6UxtXM >> CAANKAApkZXN0SG9zdElkSgAJc3JjSG9zdElkTAAMdm9sdW1lVG9Qb29sdAAPTGphdmEvdX >> RpbC9NYXA7eHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZ >> EoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hw >> AAAAAAAAAAMAAAAAAAAAEQAAAAAAAAEPdAAZVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbAA >> AAAAAAhWNAAAAAAACFZBzcgARamF2YS51dGlsLkhhc2hNYXAFB9rBwxZg0QMAAkYACmxvYW >> RGYWN0b3JJAAl0aHJlc2hvbGR4cD9AAAAAAAAQdwgAAAAQAAAAAHg, cmdVersion: 0, >> status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, >> initMsid: 128437433941265, completeMsid: null, lastUpdated: null, >> lastPolled: null, created: Tue May 24 12:10:16 MSK 2016}, job >> origin:4874 >> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm >> VM[User|i-3-271-VM] along with its volumes. >> at >> org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1007) >> at >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:2274) >> at >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:4638) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) >> at >> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4741) >> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) >> at >> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> 2016-05-24 12:10:17,220 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Complete async job-4875, jobStatus: FAILED, resultCode: 0, result: >> rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGl >> vbgAAAABWTU9yAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcH >> Rpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATa >> mF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Ro >> cm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2t >> UcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeG >> NlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AEJGYWlsZWQgdG8gbWlncmF0Z >> WQgdm0gVk1bVXNlcnxpLTMtMjcxLVZNXSBhbG9uZyB3aXRoIGl0cyB2b2x1bWVzLiB1cgAe >> W0xqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnQ7AkYqPDz9IjkCAAB4cAAAABZzcgAbamF >> 2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50YQnFmiY23YUCAARJAApsaW5lTnVtYmVyTAAOZG >> VjbGFyaW5nQ2xhc3NxAH4ABUwACGZpbGVOYW1lcQB-AAVMAAptZXRob2ROYW1lcQB-AAV4c >> AAAA-90AD1vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZW5naW5lLm9yY2hlc3RyYXRpb24uVm9s >> dW1lT3JjaGVzdHJhdG9ydAAXVm9sdW1lT3JjaGVzdHJhdG9yLmphdmF0AA5taWdyYXRlVm9 >> sdW1lc3NxAH4ADAAACOJ0ACZjb20uY2xvdWQudm0uVmlydHVhbE1hY2hpbmVNYW5hZ2VySW >> 1wbHQAHlZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwuamF2YXQAHW9yY2hlc3RyYXRlTWlnc >> mF0ZVdpdGhTdG9yYWdlc3EAfgAMAAASHnEAfgAScQB-ABNxAH4AFHNxAH4ADP____50ACRz >> dW4ucmVmbGVjdC5OYXRpdmVNZXRob2RBY2Nlc3NvckltcGx0AB1OYXRpdmVNZXRob2RBY2N >> lc3NvckltcGwuamF2YXQAB2ludm9rZTBzcQB-AAwAAAA5cQB-ABdxAH4AGHQABmludm9rZX >> NxAH4ADAAAACt0AChzdW4ucmVmbGVjdC5EZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsd >> AAhRGVsZWdhdGluZ01ldGhvZEFjY2Vzc29ySW1wbC5qYXZhcQB-ABtzcQB-AAwAAAJedAAY >> amF2YS5sYW5nLnJlZmxlY3QuTWV0aG9kdAALTWV0aG9kLmphdmFxAH4AG3NxAH4ADAAAAGt >> 0ACJjb20uY2xvdWQudm0uVm1Xb3JrSm9iSGFuZGxlclByb3h5dAAaVm1Xb3JrSm9iSGFuZG >> xlclByb3h5LmphdmF0AA9oYW5kbGVWbVdvcmtKb2JzcQB-AAwAABKFcQB-ABJxAH4AE3EAf >> gAlc3EAfgAMAAAAZnQAIGNvbS5jbG91ZC52bS5WbVdvcmtKb2JEaXNwYXRjaGVydAAYVm1X >> b3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9ic3EAfgAMAAACKnQAP29yZy5hcGFjaGU >> uY2xvdWRzdGFjay5mcmFtZXdvcmsuam9icy5pbXBsLkFzeW5jSm9iTWFuYWdlckltcGwkNX >> QAGEFzeW5jSm9iTWFuYWdlckltcGwuamF2YXQADHJ1bkluQ29udGV4dHNxAH4ADAAAADF0A >> D5vcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0 >> UnVubmFibGUkMXQAG01hbmFnZWRDb250ZXh0UnVubmFibGUuamF2YXQAA3J1bnNxAH4ADAA >> AADh0AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYX >> VsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YXQABGNhb >> GxzcQB-AAwAAABndABAb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5p >> bXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dHEAfgA1dAAPY2FsbFdpdGhDb250ZXh0c3EAfgA >> MAAAANXEAfgA4cQB-ADV0AA5ydW5XaXRoQ29udGV4dHNxAH4ADAAAAC50ADxvcmcuYXBhY2 >> hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxA >> H4AMXEAfgAyc3EAfgAMAAAB9nEAfgAscQB-AC1xAH4AMnNxAH4ADAAAAdd0AC5qYXZhLnV0 >> aWwuY29uY3VycmVudC5FeGVjdXRvcnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmp >> hdmFxAH4ANnNxAH4ADAAAAQZ0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdA >> APRnV0dXJlVGFzay5qYXZhcQB-ADJzcQB-AAwAAAR5dAAnamF2YS51dGlsLmNvbmN1cnJlb >> nQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5X >> b3JrZXJzcQB-AAwAAAJndAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN >> 1dG9yJFdvcmtlcnEAfgBHcQB-ADJzcQB-AAwAAALpdAAQamF2YS5sYW5nLlRocmVhZHQAC1 >> RocmVhZC5qYXZhcQB-ADJzcgAmamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZ >> Uxpc3T8DyUxteyOEAIAAUwABGxpc3RxAH4AB3hyACxqYXZhLnV0aWwuQ29sbGVjdGlvbnMk >> VW5tb2RpZmlhYmxlQ29sbGVjdGlvbhlCAIDLXvceAgABTAABY3QAFkxqYXZhL3V0aWwvQ29 >> sbGVjdGlvbjt4cHNyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleH >> AAAAAAdwQAAAAAeHEAfgBTeAAAEJp3CAAAAAAAAAAAeA >> 2016-05-24 12:10:17,220 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Publish async job-4875 complete on message bus >> 2016-05-24 12:10:17,220 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Wake up jobs related to job-4875 >> 2016-05-24 12:10:17,220 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Update db status for job-4875 >> 2016-05-24 12:10:17,221 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Wake up jobs joined with job-4875 and disjoin all subjobs created from >> job- 4875 >> 2016-05-24 12:10:17,230 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Done executing com.cloud.vm.VmWorkMigrateWithStorage for job-4875 >> 2016-05-24 12:10:17,231 INFO [o.a.c.f.j.i.AsyncJobMonitor] >> (Work-Job-Executor-3:ctx-601b1307 job-4874/job-4875) (logid:8748bffe) >> Remove job-4875 from job monitoring >> 2016-05-24 12:10:17,252 ERROR [c.c.a.ApiAsyncJobDispatcher] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Unexpected >> exception while executing >> org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVol >> umeCmd >> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm >> VM[User|i-3-271-VM] along with its volumes. >> at >> org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:1007) >> at >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:2274) >> at >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:4638) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) >> at >> com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4741) >> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) >> at >> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at >> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> 2016-05-24 12:10:17,253 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Complete >> async job-4874, jobStatus: FAILED, resultCode: 530, result: >> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[ >> ],"errorcode":530,"errortext":"Failed to migrated vm >> VM[User|i-3-271-VM] along with its volumes. "} >> 2016-05-24 12:10:17,254 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Publish >> async job-4874 complete on message bus >> 2016-05-24 12:10:17,254 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Wake up >> jobs related to job-4874 >> 2016-05-24 12:10:17,254 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Update db >> status for job-4874 >> 2016-05-24 12:10:17,255 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Wake up >> jobs joined with job-4874 and disjoin all subjobs created from job- >> 4874 >> 2016-05-24 12:10:17,263 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Done >> executing >> org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVol >> umeCmd for job-4874 >> 2016-05-24 12:10:17,263 INFO [o.a.c.f.j.i.AsyncJobMonitor] >> (API-Job-Executor-3:ctx-6ceed1a1 job-4874) (logid:8748bffe) Remove >> job-4874 from job monitoring ^C >> >> >> Regards, >> IT engineer >> Farheap, Russia >> Ivan Derbenev >> >> >> >> >> >> DISCLAIMER >> ========== >> This e-mail may contain privileged and confidential information which is the >> property of Accelerite, a Persistent Systems business. It is intended only >> for the use of the individual or entity to which it is addressed. If you are >> not the intended recipient, you are not authorized to read, retain, copy, >> print, distribute or use this message. If you have received this >> communication in error, please notify the sender and delete all copies of >> this message. Accelerite, a Persistent Systems business does not accept any >> liability for virus infected mails. > > > > > > > DISCLAIMER > ========== > This e-mail may contain privileged and confidential information which is the > property of Accelerite, a Persistent Systems business. It is intended only > for the use of the individual or entity to which it is addressed. If you are > not the intended recipient, you are not authorized to read, retain, copy, > print, distribute or use this message. If you have received this > communication in error, please notify the sender and delete all copies of > this message. Accelerite, a Persistent Systems business does not accept any > liability for virus infected mails.
